Image sensor, camera module, electronic device and image processing method
By setting a deformation layer on the light-in-light side of the photosensitive layer of the image sensor and controlling its deformation using voltage, the problem of unadjustable light parameters in the prior art is solved, dynamic control of light parameters is realized, and shooting flexibility and image are improved. quality.
Patent Information
- Application Number
- CN202210738124.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-24
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-06-24
AI Technical Summary
The light parameters of the photosensitive layer in the existing image sensor cannot be adjusted, resulting in insufficient shooting flexibility.
The deformation layer is provided on the light-incoming side of the photosensitive layer of the image sensor, and the deformation layer is deformed in a preset direction by applying a voltage, thereby changing the light parameters of the incident light, including the amount of light incoming and the wavelength.
Dynamic control of light parameters is realized, shooting flexibility and image quality are improved, and color value inaccuracy caused by color restoration algorithms is avoided.
Smart Images

Figure CN115134498B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of electronic equipment, and specifically relates to an image sensor, a camera module, an electronic equipment and an image processing method. Background Art
[0002] At present, terminal devices are usually equipped with shooting modules to enable the terminal devices to have shooting functions. The shooting module includes an image sensor. Under the inherent structure of the prior art, the light parameters of the light entering the photosensitive layer in the image sensor are often fixed and cannot be adjusted. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide an image sensor, a camera module, an electronic device and an image processing method to achieve dynamic control of light parameters.
[0004] In a first aspect, an embodiment of the present application provides an image sensor, comprising a deformation layer arranged on the light incident side of a photosensitive layer, the deformation layer comprising a plurality of deformation members arranged at intervals, and when a voltage is applied to the deformation layer along a preset direction, the deformation layer deforms along the second direction to change the light parameters of the light incident to the photosensitive layer; wherein, under different voltages, the deformation degree of the deformation layer is different; and different voltage application directions correspond to changes in different light parameters.
[0005] In a second aspect, an embodiment of the present application provides a camera module, which includes the above-mentioned image sensor.
[0006] In a third aspect, an embodiment of the present application provides an electronic device, which includes the above-mentioned camera module.
[0007] In a fourth aspect, an embodiment of the present application provides an image processing method, which is applied to an electronic device, wherein the electronic device includes the above-mentioned image sensor, and the method includes: obtaining a target voltage; based on the target voltage, controlling the deformation layer to deform so as to change the light parameters of the light incident on the photosensitive layer; and generating a target image based on the photosensitive information of the photosensitive layer.
[0008] The image sensor provided by the embodiment of the present invention has a deformation layer disposed on the light-entering side of the photosensitive layer, wherein the deformation layer includes a plurality of deformation members arranged at intervals, and when a voltage is applied to the deformation layer along a preset direction, the deformation layer deforms along the preset direction to change the light parameters of the light incident to the photosensitive layer; wherein, under different voltages, the deformation degree of the deformation layer is different; and different voltage application directions correspond to changes in the deformation degree of different light parameters. In this way, by disposing a deformation layer on the light-entering side of the photosensitive layer and adjusting the deformation degree of the deformation layer, the light parameters of the light incident to the photosensitive layer can be dynamically adjusted, thereby realizing dynamic control of the light parameters. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 is a schematic diagram of an image sensor provided by an embodiment of the present invention;
[0010] Figure 2 is one of the cross-sectional schematic diagrams of the image sensor provided by the embodiment of the present invention;
[0011] Figure 3A It is one of the partial stereoscopic schematic diagrams of the image sensor provided by the embodiment of the present invention;
[0012] Figure 3B This is the second partial stereoscopic schematic diagram of the image sensor provided by the embodiment of the present invention;
[0013] Figure 3C This is a third partial stereoscopic schematic diagram of an image sensor provided by an embodiment of the present invention;
[0014] Figure 4 is a second cross-sectional schematic diagram of an image sensor provided by an embodiment of the present invention;
[0015] Figure 5 is one of the flow charts of the image processing method provided by the embodiment of the present invention;
[0016] Figure 6 This is the second flowchart of the image processing method provided by the embodiment of the present invention;
[0017] Figure 7 is a timing diagram for realizing global exposure by using a deformable layer provided by an embodiment of the present invention;
[0018] Figure 8 is a structural diagram of an electronic device provided by an embodiment of the present invention;
[0019] Fig. 9 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention.
[0020] Description of reference numerals:
[0021] 01-photosensitive layer, 02-deformation layer, 03-incident light, 04-lens layer, 05-filter layer, 06-metal layer;
[0022] 021-first deformation layer, 022-second deformation layer, 023-third deformation layer;
[0023] 0221 - positive electrode set on the third side, 0222 - negative electrode set on the fourth side, 0211 - positive electrode set on the first side, 0212 - negative electrode set on the second side. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0025] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0026] The technical solution provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0027] Figure 1 is a schematic diagram of an image sensor provided by an embodiment of the present invention. Figure 1 , 03 represents the light incident on the image sensor. The image sensor includes a deformation layer 02 arranged on the light-incoming side of the photosensitive layer 01. The deformation layer 02 includes a plurality of deformation members arranged at intervals. When a voltage is applied to the deformation layer 02 along a preset direction, the deformation layer 02 is deformed along the second direction to change the light parameters of the light incident on the photosensitive layer 01; under different voltages, the deformation degree of the deformation layer 02 is different; different voltage application directions correspond to changes in different light parameters.
[0028] The photosensitive layer is used to collect light, and the deformable layer is arranged on the light-incoming side of the photosensitive layer to control the light parameters of the light incident on the photosensitive layer. Specifically, the deformable layer includes deformable members arranged at intervals, and the intervals between the deformable members are light-transmissive. The preset direction may be a direction parallel to the photosensitive layer, or a direction perpendicular to the photosensitive layer, which is not limited in the embodiment of the present invention. The light parameters may be the amount of light entering or the wavelength.
[0029] Specifically, when a voltage is applied to the deformable layer in a direction parallel to the photosensitive layer, the deformable layer will deform in the direction parallel to the photosensitive layer, and the width of the deformable element can be changed, that is, the interval between the deformable elements can be changed, so that the size of the light-transmitting area of the deformable layer can be changed. When a voltage is applied to the deformable layer in a direction perpendicular to the photosensitive layer, the deformable layer will deform in the direction perpendicular to the photosensitive layer, and the thickness or height of the deformable layer can be changed, so that the height of the spacing between the deformable elements can be changed when the spacing distance between the deformable elements remains unchanged.
[0030] Specifically, the above-mentioned deformable member can be made of piezoelectric material, that is, a material that satisfies the inverse piezoelectric effect, and the piezoelectric material can be deformed by applying a voltage. For example, single crystal, ceramic, thin film and other materials can be used as the above-mentioned deformable member. Among them, the inverse piezoelectric effect refers to the deformation caused by the application of voltage.
[0031] In summary, the image sensor provided by the embodiment of the present invention has a deformation layer disposed on the light-entering side of the photosensitive layer, and the deformation layer includes a plurality of deformation members arranged at intervals. When a voltage is applied to the deformation layer along a preset direction, the deformation layer deforms along the second direction to change the light parameters of the light incident to the photosensitive layer; wherein, under different voltages, the deformation degree of the deformation layer is different; and different voltage application directions correspond to changes in different light parameters. In this way, by disposing a deformation layer on the light-entering side of the photosensitive layer and adjusting the deformation degree of the deformation layer, the light parameters of the light incident to the photosensitive layer can be dynamically adjusted, thereby realizing dynamic control of the light parameters and improving the flexibility of shooting.
[0032] Optionally, the deformation layer 02 only includes the first deformation layer 021, or the deformation layer 02 only includes the second deformation layer 022, or the deformation layer 02 includes the first deformation layer 021 and the second deformation layer 022; the preset direction includes a first direction and a second direction; the first direction is the stacking direction of the deformation layer 02 and the photosensitive layer 01, and the second direction is the arrangement direction of the multiple deformation members, and the light parameters include wavelength and light input amount; when the first deformation layer 021 is deformed along the first direction, the first deformation layer 021 is used to change the wavelength of light incident to the photosensitive layer 01; when the second deformation layer 022 is deformed along the second direction, the first deformation layer 021 is used to change the light input amount of light incident to the photosensitive layer 01.
[0033] When the deformable layer includes a first deformable layer and a second deformable layer, the first deformable layer can be deformed along a first direction, and the second deformable layer can be deformed along a second direction. Specifically, the first direction refers to the stacking direction of the deformable layer and the photosensitive layer, which can also be understood as a direction perpendicular to the photosensitive layer. The second direction refers to the arrangement direction of the plurality of deformable members, which can be understood as a direction parallel to the photosensitive layer.
[0034] Specifically, when the second deformable layer is deformed in a direction parallel to the photosensitive layer, the width of the deformable elements in the second deformable layer can be changed, that is, the interval between the deformable elements can be changed, thereby changing the size of the light-transmitting area of the deformable layer and controlling the amount of light incident on the photosensitive layer.
[0035] When the first deformable layer is deformed in a direction perpendicular to the photosensitive layer, the thickness or height of the deformable layer can be changed. Since there are gaps between the deformable parts, the first deformable layer is equivalent to a one-dimensional grating. When light passes through the first deformable layer, it is equivalent to passing through a slit, that is, similar to the structure of passing through an FP cavity (optical resonant cavity). When the thickness of the first deformable layer is constant, only a few wavelengths of incident light can make the standing wave wavelength meet the matching conditions, and transmittance enhancement peaks will be formed at these wavelength positions in the entire transmission spectrum. In other words, the wavelengths that can be transmitted can be screened by controlling the thickness of the deformable layer. For example, the thickness of the first deformable layer can be controlled so that at this thickness, only light with a wavelength of 700nm can pass.
[0036] Specifically, when a voltage is applied to the first deformable layer along the first direction, the first deformable layer will deform along the first direction, ie, a direction perpendicular to the photosensitive layer, thereby changing the thickness of the first deformable layer and thus changing the wavelength of light that can be transmitted.
[0037] Since the image sensor usually adopts Bayer Filter (Bayer color filter array), only one color component can pass through each pixel acquisition point in this image sensor, and in order to obtain the complete color value of a pixel point, a color restoration algorithm (Demosaicing Algorithm) is usually used to estimate and restore the other two color components. However, the final image obtained using this color restoration algorithm is prone to errors, that is, it is impossible to obtain more accurate color values, which will cause the final image to be quite different from the original image. In one implementation, the first deformation layer can be controlled to deform along the first direction to control only red light (R), green light (G) and blue light (B) to be incident on the photosensitive layer respectively, so that each pixel point obtains accurate RGB color information, thereby improving the quality of the image. Among them, the interval formed between each deformation member in the first deformation layer can correspond to the pixel unit row in the photosensitive layer, thereby ensuring to a certain extent that each pixel unit can obtain RGB color information uniformly.
[0038] In the image sensor provided by the embodiment of the present invention, the deformation layer only includes the first deformation layer, or the deformation layer only includes the second deformation layer, or the deformation layer includes the first deformation layer and the second deformation layer; the preset direction includes the first direction and the second direction; the first direction is the stacking direction of the deformation layer and the photosensitive layer, and the second direction is the arrangement direction of the multiple deformation members, and the light parameters include the wavelength and the amount of light entering; when the first deformation layer is deformed along the first direction, the first deformation layer is used to change the wavelength of the light incident to the photosensitive layer; when the second deformation layer is deformed along the second direction, the first deformation layer is used to change the amount of light incident to the photosensitive layer. In this way, by setting the first deformation layer and the second deformation layer, the wavelength and the amount of light incident to the photosensitive layer can be controlled respectively, thereby improving the control accuracy to a certain extent.
[0039] Optionally, a positive electrode and a negative electrode are respectively arranged on the first side and the second side of each deformable element in the first deformable layer; the first side and the second side are two opposite sides in the first direction; a positive electrode and a negative electrode are respectively arranged on the third side and the fourth side of each deformable element in the second deformable layer; the third side and the fourth side are two opposite sides in the second direction; the image sensor also includes a first power supply unit, and the positive and negative interfaces of the first power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the first side and the second side; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the first side and the second side along the first direction, so that the first deformable layer is deformed along the first direction; the image sensor also includes a second power supply unit, and the positive and negative interfaces of the second power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side along the second direction, so that the second deformable layer is deformed along the second direction.
[0040] Wherein, both the positive electrode and the negative electrode may include connecting wires, and the positive electrode and the negative electrode may be connected to the positive and negative interfaces of the power supply unit through their respective connecting wires. Voltage may be applied to the deformable member through the positive and negative electrodes, so that the degree of deformation of each deformable member may be changed. Specifically, the first side surface and the second side surface refer to the surfaces distributed along the first direction. When the first direction is perpendicular to the direction of the photosensitive layer, the first side surface and the second side surface may be the top and bottom of the deformable member. The third side surface and the fourth side surface refer to the surfaces distributed along the second direction. When the second direction is parallel to the direction of the photosensitive layer, the third side surface and the fourth side surface may be the left and right sides of the deformable member. Wherein, the first power supply unit is used to supply power to the positive and negative electrodes arranged on the first side surface and the second side surface so that the deformable layer is deformed. Specifically, since the positive and negative interfaces of the first power supply unit are connected to the positive and negative electrodes distributed in the first direction, the first power supply unit supplies power along the first direction. Correspondingly, the positive and negative interfaces of the second power supply unit are connected to the positive and negative electrodes distributed in the second direction. Therefore, the second power supply unit supplies power along the second direction.
[0041] In an image sensor provided by an embodiment of the present invention, a positive electrode and a negative electrode are respectively arranged on the first side and the second side of the deformable element in the first deformable layer; the first side and the second side are sequentially distributed along the first direction; a positive electrode and a negative electrode are respectively arranged on the third side and the fourth side of the deformable element in the second deformable layer; the third side and the fourth side are sequentially distributed along the second direction; the image sensor further comprises a first power supply unit, the positive and negative interfaces of the first power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the first side and the second side; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the first side and the second side along the first direction, so that the first deformable layer is deformed along the first direction; the image sensor further comprises a second power supply unit, the positive and negative interfaces of the second power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side along the second direction, so that the second deformable layer is deformed along the second direction. In this way, by setting positive and negative electrodes on the deformable member and supplying power to the positive and negative electrodes through the power supply unit, the deformation degree of the deformable layer can be more conveniently controlled, thereby adjusting the light parameters of the light incident on the photosensitive layer and improving the adjustment efficiency.
[0042] Optionally, in the above-mentioned image sensor, the deformable layer 02 includes a third deformable layer 023, and the preset direction includes a first direction and a second direction; the first direction is the stacking direction of the deformable layer 02 and the photosensitive layer 01, and the second direction is the arrangement direction of the multiple deformable members, and the light parameters include wavelength and amount of light entering; when the third deformable layer 023 is deformed along the first direction, the third deformable layer 023 is used to change the wavelength of light incident to the photosensitive layer; when the third deformable layer 023 is deformed along the second direction, the third deformable layer 023 is used to change the amount of light incident to the photosensitive layer.
[0043] Among them, a third deformable layer can be set, and the third deformable layer can be deformed along the first direction and the second direction. That is to say, the thickness and spacing size of the deformable elements in the third deformable layer can be adjusted. That is, the third deformable layer can be used to adjust both the amount of light entering the photosensitive layer and the wavelength of light incident on the photosensitive layer.
[0044] The image sensor provided by the embodiment of the present invention includes a third deformable layer through the deformable layer, the preset direction includes a first direction and a second direction; the first direction is the stacking direction of the deformable layer and the photosensitive layer, the second direction is the arrangement direction of the multiple deformable members, and the light parameters include wavelength and light input amount; when the third deformable layer is deformed along the first direction, the third deformable layer is used to change the wavelength of the light incident to the photosensitive layer; when the third deformable layer is deformed along the second direction, the third deformable layer is used to change the light input amount of the light incident to the photosensitive layer. In this way, only one deformable layer is needed to achieve the effect of changing the wavelength and changing the light input amount, which can reduce the volume and production cost of the above-mentioned image sensor to a certain extent.
[0045] Optionally, the first side and the third side of each deformable element in the third deformable layer 023 are provided with a positive electrode, and the second side and the fourth side of each deformable element in the third deformable layer 023 are provided with a negative electrode; the first side and the second side are two opposite sides in the first direction, and the third side and the fourth side are two opposite sides in the second direction; the image sensor also includes a first power supply unit, and the positive and negative interfaces of the first power supply unit are respectively connected to the positive electrode and the negative electrode respectively provided on the first side and the second side; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively provided on the first side and the second side along the first direction, so that the first deformable layer is deformed along the first direction; the image sensor also includes a second power supply unit, and the positive and negative interfaces of the second power supply unit are respectively connected to the positive electrode and the negative electrode respectively provided on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively provided on the third side and the fourth side along the second direction, so that the second deformable layer is deformed along the second direction.
[0046] Among them, voltage can be applied to the deformable member through the positive and negative electrodes, so that the deformation degree of the deformable member can be changed. Specifically, the first side surface and the second side surface refer to the surfaces distributed along the first direction. When the first direction is perpendicular to the photosensitive layer, the first side surface and the second side surface can be the top and bottom of the deformable member. The third side surface and the fourth side surface refer to the surfaces distributed along the second direction. When the second direction is parallel to the photosensitive layer, the third side surface and the fourth side surface can be the left and right surfaces of the deformable member.
[0047] The first power supply unit is used to supply power to the positive and negative electrodes arranged on the first side and the second side so that the deformation layer is deformed. Specifically, since the positive and negative interfaces of the first power supply unit are connected to the positive and negative electrodes distributed in the first direction, the first power supply unit supplies power along the first direction, and correspondingly, the positive and negative interfaces of the second power supply unit are connected to the positive and negative electrodes distributed in the second direction, so the second power supply unit supplies power along the second direction.
[0048] In an image sensor provided by an embodiment of the present invention, a positive electrode is disposed on the first side and the third side of each deformable element in the third deformable layer, and a negative electrode is disposed on the second side and the fourth side of each deformable element in the third deformable layer; the first side and the second side are two opposite sides in the first direction, and the third side and the fourth side are two opposite sides in the second direction; the image sensor further comprises a first power supply unit, a positive and negative interface of the first power supply unit is respectively connected to the positive electrode and the negative electrode respectively disposed on the first side and the second side; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively disposed on the first side and the second side along the first direction, so that the first deformable layer is deformed along the first direction; the image sensor further comprises a second power supply unit, a positive and negative interface of the second power supply unit is respectively connected to the positive electrode and the negative electrode respectively disposed on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively disposed on the third side and the fourth side along the second direction, so that the second deformable layer is deformed along the second direction. By arranging positive and negative electrodes on the deformable member and supplying power along the first and second directions respectively through the first and second power supply units, the deformation degree of the deformable layer can be more conveniently controlled, thereby adjusting the light parameters of the light incident on the photosensitive layer and improving the adjustment efficiency.
[0049] Optionally, the shape and size of each deformable element in the deformable layer 02 and the spacing between every two deformable elements are the same; the photosensitive layer 01 includes a plurality of pixel units; and the spacing between every two deformable elements is less than or equal to the width of a pixel unit.
[0050] When the shapes, sizes and spacings of the above-mentioned deformable elements are the same, the above-mentioned deformable layer is equivalent to a one-dimensional grating. Specifically, when a voltage is applied to the one-dimensional grating, the deformation degree of each deformable element is the same. The pixel unit refers to the pixel collection point in the photosensitive layer. The pixel collection point can form a pixel array. The photosensitive layer is based on the pixel unit for light sensing.
[0051] In an embodiment of the present invention, the shape, size and spacing between each deformable element in the deformable layer are the same; the photosensitive layer includes a plurality of pixel units; the spacing between each two deformable elements is less than or equal to the width of a pixel unit. In this way, it is only necessary to apply a voltage to the entire deformable layer to make each deformable element produce the same degree of deformation, and there is no need to apply different voltages to each deformable element respectively, thereby improving the efficiency of controlling the deformation degree of the deformable layer. At the same time, the spacing between the deformable elements is less than or equal to the width of a pixel unit, which can ensure to a certain extent that the changes in the light parameters of the light collected by each pixel unit are the same.
[0052] Optionally, the image sensor further includes a lens layer 04 , and the deformation layer 02 is disposed between the lens layer 04 and the photosensitive layer 01 .
[0053] Among them, the lens layer 04 can be located on the light-incoming side of the deformable layer, and the lens layer (Micro Lens) can include multiple tiny lenses, so that the light-sensitive area of the photosensitive layer is no longer determined by the opening area of the photosensitive sheet, but by the surface area of the microlens, thereby improving the aperture ratio in terms of specifications while taking into account the size of a single pixel, thereby improving the sensitivity. Furthermore, the image sensor can also include a filter layer 05 and a metal layer 06. The filter layer 05 can be located between the deformable layer 02 and the photosensitive layer 01, and the metal layer (metal wiring) can be located on the light-outgoing side of the photosensitive layer. For example, Figure 2 is one of the cross-sectional schematic diagrams of the image sensor provided by the embodiment of the present invention, such as Figure 2 As shown, the image sensor is stacked with a metal layer 06, a photosensitive layer 01, a filter layer 05, a second deformation layer 022 and a lens layer 04. The filter layer 05 can be used to filter light according to wavelength, collect color information, and obtain color information of each pixel unit, thereby ensuring the quality of image acquisition. The filter layer 05 can be composed of a single color red (R) / green (G) / blue (B) filter placed on each pixel unit at intervals. Alternatively, an RGBW pixel arrangement can also be used.
[0054] Specifically, when a voltage is applied to the second deformable layer 022 along the second direction, it will deform along the second direction, that is, in a direction parallel to the photosensitive layer 01, thereby changing the width of the deformable member, that is, changing the interval between the deformable members, thereby changing the size of the light-transmitting area of the deformable layer, and then changing the amount of light entering the photosensitive layer. Specifically, the wider the deformable members, the smaller the amount of light entering, and correspondingly, the narrower the deformable members, the greater the amount of light entering. Specifically, by changing the amount of light entering the photosensitive layer, dynamic control of the amount of light entering can be achieved as needed.
[0055] Figure 3A is one of the partial stereoscopic schematic diagrams of the image sensor provided by the embodiment of the present invention, Figure 3A 01, the filter layer 05 and the second deformable layer 022 are shown. A positive electrode 0221 and a negative electrode 0222 are respectively disposed on the third side surface and the fourth side surface of the deformable element of the second deformable layer 022. Figure 3B FIG. 2 is a partial stereoscopic schematic diagram of an image sensor provided by an embodiment of the present invention. Figure 3B The photosensitive layer 01 and the first deformable layer 021 are shown in FIG. A positive electrode 0211 and a negative electrode 0212 are respectively disposed on the first side surface and the second side surface of the deformable element of the first deformable layer 021 . Figure 3C FIG. 3 is a third partial stereoscopic schematic diagram of an image sensor provided by an embodiment of the present invention. Figure 3C The photosensitive layer 01, the first deformable layer 021 and the second deformable layer 022 are shown in FIG. It should be noted that the electrodes are represented in the form of a grid. This is only a schematic form of the electrode. In practical applications, the form of the electrode is not unique, and the embodiment of the present invention does not limit this.
[0056] because Figure 3B as well as Figure 3C The image sensors shown in the figure all include a first deformable layer, and by controlling the first deformable layer to deform along a first direction, only red light, green light, and blue light can be controlled to be incident on the photosensitive layer, respectively, so that each pixel point obtains accurate RGB color information, thereby achieving the function of the filter layer. Therefore, Figure 3B as well as Figure 3C The image sensor shown does not need to be provided with a filter layer, thereby reducing the volume of the image sensor to a certain extent.
[0057] Figure 4 FIG. 2 is a second cross-sectional schematic diagram of an image sensor provided by an embodiment of the present invention. Figure 4 As shown, the image sensor includes a metal layer 06 , a photosensitive layer 01 , a first deformation layer 021 and a lens layer 04 which are stacked in sequence.
[0058] Figure 5 is one of the flowcharts of the image processing method provided by an embodiment of the present invention. The method is applied to an electronic device, and the electronic device includes the above-mentioned image sensor, such as Figure 5 As shown, the method includes:
[0059] Step 101: Obtain target voltage.
[0060] The target voltage refers to the voltage corresponding to the deformation degree of the deformation layer that meets the current demand. Specifically, the target voltage can be set by the user before issuing a photo taking instruction, or can be automatically determined according to a preset method.
[0061] Step 102: Based on the target voltage, control the deformable layer to deform so as to change light parameters of the light incident on the photosensitive layer.
[0062] The above parameter may be the amount of light entering or the wavelength. The above target voltage may be applied to the deformable layer along a specified direction to cause the above deformable layer to deform along the specified direction to change the amount of light entering the photosensitive layer or the wavelength of light incident on the photosensitive layer. The above specified direction may be a direction parallel to the photosensitive layer or a direction perpendicular to the photosensitive layer, which is not limited in the embodiment of the present invention.
[0063] Step 103: Generate a target image based on the photosensitivity information of the photosensitive layer.
[0064] The photosensitive information refers to the electrons accumulated in the photosensitive layer, which can be obtained by exposing the photosensitive layer. The exposure operation refers to the light passing through the light-transmitting area of the deformation layer and incident on the photosensitive layer, and the photosensitive layer begins to sense light and accumulate electrons. It should be noted that the amount of light entering the photosensitive layer at this time is the amount of light entering the light-transmitting area of the deformation layer.
[0065] Specifically, during the exposure process, the photosensitive layer converts photons into electrons based on the incident light. After the exposure operation is completed, the accumulated electrons are read out based on the readout circuit to obtain an exposure image, that is, a target image.
[0066] In summary, the image processing method provided by the embodiment of the present invention obtains a target voltage; based on the target voltage, controls the deformation of the deformable layer to change the light parameters of the light incident on the photosensitive layer; and generates a target image based on the photosensitive information of the photosensitive layer. In this way, by simply setting different target voltages, the deformation degree of the deformable layer can be adjusted, thereby adjusting the light parameters of the light incident on the photosensitive layer, achieving dynamic control of the light parameters and improving the flexibility of shooting.
[0067] Optionally, the deformation layer includes a first deformation layer, and the light parameter includes a wavelength; the operation of obtaining the target voltage may specifically include:
[0068] Step 201 : Based on a preset correspondence between wavelength and voltage, a first target voltage corresponding to a red light wavelength, a second target voltage corresponding to a green light wavelength, and a third target voltage corresponding to a blue light wavelength are obtained.
[0069] The first deformable layer may be a deformable layer that is deformed in a direction perpendicular to the photosensitive layer. The preset correspondence between the wavelength and the voltage refers to the correspondence between the voltage applied to the first deformable layer and the wavelength through which light can pass. Specifically, the correspondence is different for deformable layers of different sizes. The image sensor can be calibrated on the production line before it leaves the factory, that is, the correspondence can be obtained through experiments. The correspondence can be expressed based on a functional relationship obtained by calibration.
[0070] Specifically, based on the above correspondence, a first target voltage corresponding to a wavelength of 700nm, a second target voltage corresponding to a wavelength of 550nm, and a third target voltage corresponding to a wavelength of 430nm can be obtained respectively, so that red light, green light, and blue light can pass through the deformation layer and enter the photosensitive layer in turn.
[0071] The above operation of controlling the deformation of the deformable layer based on the target voltage to change the light parameters of the light incident on the photosensitive layer may specifically include:
[0072] Step 202: According to a preset order, the first target voltage, the second target voltage and the third target voltage are applied to the deformable members in the first deformable layer in sequence along a first direction to adjust the deformation degree of the first deformable layer so that the wavelength of light incident to the photosensitive layer changes to a red light wavelength, a green light wavelength and a blue light wavelength in sequence; wherein the first direction is a stacking direction of the first deformable layer and the photosensitive layer.
[0073] Among them, the above-mentioned preset order refers to the order of applying voltage, which can be the first, second, and third target voltages, or the second, first, and third target voltages, or the third, second, and first target voltages. Specifically, a default order can be set, or it can be set by the user, and the embodiment of the present invention does not limit this. The stacking direction of the first deformable layer and the photosensitive layer can be understood as a direction perpendicular to the photosensitive layer. Specifically, when the deformable layer is deformed in a direction perpendicular to the photosensitive layer, the thickness or height of the deformable layer can be changed. Since there is a gap between each deformable member, the first deformable layer is equivalent to a one-dimensional grating. When light passes through the first deformable layer, it is equivalent to passing through a slit, that is, similar to the structure of passing through an FP cavity (optical resonant cavity). When the thickness of the deformable layer is constant, only a few wavelengths of incident light can make the standing wave wavelength meet the matching condition, and a transmittance enhancement peak will be formed at these wavelength positions on the entire transmission spectrum. In other words, the wavelength that can be transmitted can be screened by controlling the thickness of the deformable layer. The deformable member in the above-mentioned image sensor can be made of piezoelectric material, that is, the thickness of the deformable layer can be changed by changing the voltage applied to the deformable member.
[0074] For example, the first target voltage can be applied to the deformable layer along the first direction to make the light (red light) with a wavelength of 700nm incident on the photosensitive layer, and the photosensitive layer collects red component information. Secondly, the second target voltage is applied to the deformable layer along the first direction to make the light (green light) with a wavelength of 550nm incident on the photosensitive layer, and the photosensitive layer collects green component information. Finally, the third target voltage is applied to the deformable layer along the first direction to make the light (blue light) with a wavelength of 430nm incident on the photosensitive layer, and the photosensitive layer collects blue component information. In this way, through multiple switching controls, each pixel unit in the photosensitive layer can truly filter the component information of RGB color, thereby obtaining a better color response. Among them, a pixel unit can be regarded as a pixel. In the embodiment of the present invention, the first deformable layer made of piezoelectric material can control the thickness deformation of the formed one-dimensional grating by changing the applied voltage, so that it can dynamically switch between different transmission spectra, so that the three wavelengths of red, blue and green can be directly detected, thereby obtaining sharper and more realistic colors.
[0075] In an embodiment of the present invention, the deformation layer includes a first deformation layer, the light parameters include wavelength, and based on a preset correspondence between wavelength and voltage, a first target voltage corresponding to the red light wavelength, a second target voltage corresponding to the green light wavelength, and a third target voltage corresponding to the blue light wavelength are obtained; based on the target voltage, the deformation of the deformation layer is controlled to change the light parameters of the light incident to the photosensitive layer, including: in a preset order, the first target voltage, the second target voltage and the third target voltage are applied to the deformation member in the first deformation layer in sequence along a first direction to adjust the deformation degree of the first deformation layer so that the wavelength of the light incident to the photosensitive layer is changed to the red light wavelength, the green light wavelength and the blue light wavelength in sequence; wherein the first direction is the stacking direction of the first deformation layer and the photosensitive layer. In this way, by controlling the deformation layer to deform three times along the stacking direction with the photosensitive layer, the red, green and blue component information can be obtained respectively, so that each pixel in the photosensitive layer can directly obtain the color value (RGB value) of the image, without using the color restoration algorithm (Demosaicing Algorithm) in the prior art to calculate the RGB value, avoiding the problem of inaccurate color information collected and improving the image quality. The RGB value can be obtained based on a preset processing end, wherein the processing end can be an AP / ISP end.
[0076] Optionally, the deformation layer includes a second deformation layer, and the light parameter includes an amount of incoming light; the operation of obtaining the target voltage may specifically include:
[0077] Step 301: applying a first voltage to the deformable elements in the second deformable layer along a second direction to adjust the deformation degree of the second deformable layer to a first deformation degree; the second direction is the arrangement direction of the plurality of deformable elements, and different deformation degrees correspond to different amounts of incoming light.
[0078] Among them, the second deformation layer can be a deformation layer that deforms in a direction parallel to the photosensitive layer. When the light parameter is the amount of light entering, the amount of light entering corresponding to the first deformation degree in the initial state can be the maximum amount of light entering, that is, the amount of light entering corresponding to the width of each deformation member in the above-mentioned second deformation layer when the width is the narrowest. Correspondingly, the first deformation degree refers to the deformation degree corresponding to the width of each deformation member in the above-mentioned second deformation layer when the width is the narrowest. At this time, the deformation degree is usually 0. After subsequent adjustments, the first deformation degree can gradually increase. Among them, the first voltage refers to the voltage corresponding to the first deformation degree. Specifically, the above-mentioned first voltage can be obtained through the preset correspondence between the deformation degree and the voltage.
[0079] Step 302: Capture a reference image through the image sensor; and determine a shooting exposure based on the reference image.
[0080] The reference image refers to an image used to obtain a reference value after the photosensitive layer performs a photosensitization operation under the first deformation degree, and the reference image will not be output and displayed. The above-mentioned shooting exposure refers to the exposure when obtaining the above-mentioned reference image. Specifically, the exposure is positively correlated with the amount of light entering the photosensitive layer.
[0081] Specifically, when the width of each deformable member in the second deformation layer is the narrowest, the photosensitive layer can be exposed to obtain a reference image first, and the brightness of the image can be obtained based on the reference image, that is, the shooting exposure. Optionally, the electronic device includes a processing module, which can statistically calculate the brightness of each pixel in the reference image to obtain the current image brightness, and determine the shooting exposure of the reference image based on the current image brightness. Among them, the processing module may include an AP / ISP end, and the AP / ISP end may determine the current exposure based on a preset automatic exposure algorithm (Automatic Exposure Control, AEC). Exemplarily, the current light flux can be determined based on the current image brightness and the current first deformation degree. The light flux is positively correlated with the current image brightness and negatively correlated with the deformation degree of the second deformation layer. The product of the current light flux, the current exposure time and the current sensitivity is calculated to obtain the shooting exposure. Among them, the shooting exposure can also be called the shooting exposure value.
[0082] For example, when the adjustment is made for the first time, the first deformation degree may be 100% of the maximum deformation degree, then a voltage V1 may be applied to the deformable member in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to 100%. Wherein, V1 may be a voltage corresponding to 100% of the maximum deformation degree in the preset corresponding relationship between the deformation degree and the voltage, and the specific value of V1 may be measured in advance, and the specific value of V1 corresponding to the second deformation layer of different materials and different sizes may be different.
[0083] Further, if the shooting exposure determined based on the reference image collected is greater than the preset exposure threshold value when the maximum deformation degree is 100%, then the first deformation degree can be updated, for example, the first deformation degree is updated to 95%, and accordingly, a voltage V2 can be applied to the deformable member in the second deformation layer along the second direction, the deformation degree of the second deformation layer is adjusted to 95%, and the reference image is collected again through the image sensor. If the shooting exposure determined based on the reference image collected this time does not exceed the preset exposure threshold value, then the voltage V2 corresponding to the deformation degree of 95% can be determined as the target voltage. Among them, V2 can be the voltage corresponding to the deformation degree of 95% in the preset corresponding relationship between the deformation degree and the voltage. The specific value of V2 can be measured in advance. The specific value of V2 corresponding to the second deformation layer of different materials and different sizes can be different.
[0084] Step 303: When the shooting exposure does not exceed the preset exposure threshold, based on a preset correspondence between the deformation degree and the voltage, determine a first voltage corresponding to the first deformation degree as a target voltage.
[0085] The preset exposure threshold may be a preset exposure reference value, which may be set by the user, or may be an average exposure range set by the image sensor before leaving the factory. For example, the preset exposure threshold may be set to 200±10%, indicating that when the exposure is between 180 and 220, the brightness of the image is within a normal range, that is, there is neither overexposure nor underexposure.
[0086] Specifically, when the shooting exposure does not exceed the preset exposure threshold, it means that the brightness of the image is within the normal range, and there is no need to adjust the amount of light entering the photosensitive layer, that is, there is no need to adjust the deformation degree of the second deformation layer, and the image can be directly taken based on the current first voltage and the first deformation degree to obtain the image. Therefore, in the embodiment of the present invention, the voltage corresponding to the current first deformation degree can be determined as the target voltage so that the shooting exposure meets the requirements.
[0087] In an embodiment of the present invention, the deformation layer includes a second deformation layer, and the light parameters include the amount of light entering; a first voltage is applied to the deformation member in the second deformation layer along a second direction to adjust the deformation degree of the second deformation layer to the first deformation degree; a reference image is collected by the image sensor; based on the reference image, a shooting exposure is determined; when the exposure is less than or equal to a preset exposure threshold, based on a preset correspondence between the deformation degree and the voltage, a first voltage corresponding to the first deformation degree is determined as a target voltage; wherein the second direction is the arrangement direction of the multiple deformation members, and different deformation degrees correspond to different amounts of light entering. In this way, adjustments are made based on the collected reference image to obtain a suitable target voltage, thereby ensuring the accuracy of the target voltage to a certain extent, ensuring that the target voltage is more suitable for the current environment, and at the same time, retaining as much light entering as possible for the photosensitive layer without overexposure.
[0088] Optionally, the above deformation degree is based on the photosensitive information of the photosensitive layer, and before generating the target image, the embodiment of the present invention may further include:
[0089] Step 401: When there is band noise in the reference image, set the shooting exposure time to an integer multiple of a preset period.
[0090] Among them, shooting exposure time refers to the current shooting parameters, which can be understood as the shutter speed or the length of time the photosensitive layer is sensitive to light. The faster the shutter speed, the shorter the shooting exposure time, and thus the smaller the exposure. Among them, the presence of banding noise in the image refers to the presence of light and dark stripes in the image, which usually appears when shooting scenes with artificial light sources. Since the artificial light source fluctuates in a fixed period, when the photosensitive layer begins to sense light line by line, the collected brightness is different, resulting in inconsistent exposure of adjacent lines in the final image, which can also be called the power frequency flicker (Flicker) phenomenon. Specifically, the AP / ISP end can use a noise detection algorithm to detect whether there is banding noise in the image.
[0091] Specifically, taking 50Hz alternating current as an example, the artificial light source usually fluctuates at a frequency of 100Hz, that is, the fluctuation period of the artificial light source is 1 / 100s. Therefore, the exposure time can be set to an integer multiple of the artificial light source period, that is, N*1 / 100s, which is an integer multiple of 10ms. In this way, since the energy of the artificial light source is fixed within a period, the brightness collected by each row of the photosensitive layer within the exposure period is the same, and the industrial frequency flicker phenomenon can be avoided. Among them, the preset period refers to the artificial light source period of 1 / 100s. Specifically, the shooting exposure time can be pre-set to 20ms, or the user can set the value of N to obtain the shooting exposure time. The embodiment of the present invention does not limit this.
[0092] In the embodiment of the present invention, when there is band noise in the reference image, the shooting exposure time is set to an integer multiple of a preset period, thereby avoiding the problem of power frequency flicker.
[0093] Optionally, after determining the shooting exposure based on the reference image, the embodiment of the present invention may further include:
[0094] Step 501: When the shooting exposure exceeds a preset exposure threshold, determine a target light input amount based on the preset exposure threshold, the image brightness of the reference image, the shooting exposure time, and the shooting sensitivity.
[0095] Among them, the above-mentioned shooting exposure time and shooting sensitivity are both current shooting parameters. Shooting sensitivity can also be called photosensitivity gain, which refers to the sensitivity of the photosensitive layer to light, that is, the ability of the photosensitive layer to convert photons into electrons. The higher the shooting sensitivity, the higher the exposure.
[0096] Specifically, when the shooting exposure exceeds the preset exposure threshold, it means that the image brightness is overexposed or underexposed. At this time, it is necessary to adjust the exposure to keep the brightness of the image within the normal range to avoid overexposure or too dark image. Specifically, since the exposure is positively correlated with the shooting exposure time, shooting sensitivity and light intake, the target light flux can be obtained by dividing the preset exposure threshold by the current exposure time and the current sensitivity. Based on the target light flux and the current image brightness, the target light intake is obtained. For example, the value obtained by dividing the target light flux by the current image brightness is used as the target light intake. In this way, the shooting exposure corresponding to the calculated target light intake can be made not to exceed the preset exposure threshold.
[0097] Step 502: Based on a preset correspondence between the amount of light entering and the degree of deformation, determine a second degree of deformation corresponding to the target amount of light entering.
[0098] The preset correspondence between the amount of light entering and the degree of deformation refers to the correspondence between the degree of deformation of the deformation layer 0-100% and the specific amount of light entering. Specifically, the higher the degree of deformation, the lower the amount of light entering. Specifically, the image sensor can be calibrated on the production line before leaving the factory, that is, the above correspondence can be obtained through experiments. The above correspondence can be expressed based on the functional relationship obtained by calibration.
[0099] Step 503: Determine a second voltage corresponding to the second deformation degree.
[0100] The second voltage corresponding to the second deformation degree can be obtained through the preset correspondence between the deformation degree and the voltage. Specifically, the preset correspondence between the deformation degree and the voltage can also be obtained by calibrating the image sensor before it leaves the factory, and the correspondence can be expressed based on the function relationship obtained by calibration.
[0101] Step 504: Apply the second voltage to the deformable element in the second deformable layer along the second direction to adjust the deformation degree of the second deformable layer to the second deformation degree to adjust the shooting exposure.
[0102] Specifically, after obtaining the second voltage, the second voltage can be applied to the deformable member through the power supply unit in the processing module, so that the deformation degree of the second deformable layer reaches the second deformation degree, and the light input amount of the photosensitive layer can reach the above-mentioned target light input amount. It should be noted that, since the brightness in the environment will change, even if the target light input amount is calculated based on the preset exposure threshold value and the deformation degree is adjusted based on the target light input amount, the brightness of the current image may change due to the change in ambient brightness, resulting in the next round still not satisfying the situation where the exposure does not exceed the preset exposure threshold value.
[0103] Among them, since the first deformation degree usually refers to the deformation degree corresponding to the time when the width of each deformable member in the second deformation layer is the narrowest, the deformation degree is usually 0 at this time, and there is usually no situation where the shooting exposure is less than the preset exposure threshold, so the shooting exposure exceeding the preset exposure threshold usually refers to the shooting exposure being greater than the preset exposure threshold. At this time, the second deformation degree can be made less than the first deformation degree, and the second deformation degree determined each time can be gradually reduced. At this time, the second deformation degree can be understood as a new first deformation degree, and the operations of steps 301 to 303 are performed again until the shooting exposure does not exceed the preset exposure threshold. Accordingly, the voltage corresponding to the current first deformation degree can be determined as the target voltage, and the current first deformation degree is the second deformation degree determined for the last time. Among them, the amplitude of each reduction of the second deformation degree can be positively correlated with the excess value of the current shooting exposure exceeding the preset exposure threshold. The greater the difference between the second deformation degree and the first deformation degree, the greater the reduction in the shooting exposure. In other words, the specific reduction in the shooting exposure is positively correlated with the difference between the second deformation degree and the first deformation degree. Alternatively, the specific reduction amount each time may be fixed, for example, reduced according to a preset value.
[0104] In an embodiment of the present invention, when the shooting exposure exceeds the preset exposure threshold, the target light intake is determined based on the preset exposure threshold, the image brightness of the reference image, the shooting exposure time, and the shooting sensitivity; based on the preset correspondence between the light intake and the deformation degree, the second deformation degree corresponding to the target light intake is determined; the second voltage corresponding to the second deformation degree is determined; and the second voltage is applied to the deformable member in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the second deformation degree to adjust the shooting exposure. In this way, the target light intake is calculated based on the preset exposure threshold, and the deformation degree corresponding to the target light intake is obtained through the preset correspondence between the light intake and the deformation degree, and the corresponding deformation degree is reset to the first deformation degree, so that in the case of the deformation degree corresponding to the target light intake, the preset exposure threshold can be met more quickly, thereby reducing the number of adjustments as much as possible.
[0105] Figure 6 FIG. 2 is a second flowchart of the image processing method provided by an embodiment of the present invention. Figure 6 As shown:
[0106] Step 211: applying a first voltage to the deformable element in the second deformable layer along a second direction to adjust the deformation degree of the second deformable layer to a first deformation degree, wherein the first voltage is a voltage corresponding to the first deformation degree.
[0107] Step 212: Capture a reference image based on an image sensor.
[0108] Step 213: Detect whether there is band noise in the reference image.
[0109] Step 214: If there is band noise, adjust the shooting exposure time to an integer multiple of the preset period, and execute step 212 again.
[0110] Step 215: If there is no band noise, determine the shooting exposure based on the reference image.
[0111] Step 216, determining whether the shooting exposure exceeds a preset exposure threshold;
[0112] Step 217 : If the exposure value does not exceed the preset exposure threshold, determine the first voltage as the target voltage and directly execute step 221 .
[0113] Step 218: When the preset exposure threshold is exceeded, detecting whether the shooting sensitivity reaches the minimum sensitivity.
[0114] Step 219: If the minimum sensitivity is not reached, adjust the sensitivity so that the shooting exposure does not exceed the preset exposure threshold.
[0115] Step 220: When the minimum sensitivity is reached, the first deformation degree is adjusted so that the exposure does not exceed the preset exposure threshold, and the above steps are executed again until the shooting exposure is not greater than the preset exposure threshold.
[0116] Step 221 : Generate a target image at a target voltage.
[0117] Specifically, the sensitivity of the photosensitive layer usually has an adjustable range, that is, there is a maximum sensitivity and a minimum sensitivity. Since the integer multiples of the above-mentioned preset period are usually greater than the original exposure time, when the shooting exposure time is adjusted to an integer multiple of the preset period, the adjusted exposure will be greater than the original exposure, thereby affecting the brightness of the image. At this time, when the sensitivity has not reached the minimum value, that is, when the sensitivity can be reduced, the shooting exposure can be reduced by reducing the sensitivity.
[0118] In actual application scenarios, in order to ensure the dark imaging effect, the camera module with a fixed aperture will try its best to increase the aperture of the module to increase the amount of light entering, and match it with a very high conversion gain (Conversion Gain) to obtain the highest possible low-light signal value. Among them, the conversion gain represents the ability to convert photons into electrons. However, this is very easy to overexpose in high-brightness scenes, that is, the dynamic range of the highlight part will be sacrificed. And too high photosensitivity will make it easier to enter a smaller exposure time under artificial light, that is, an exposure time that is less than the power frequency cycle, resulting in a power frequency flicker problem. After the above steps 211 to 221, a target image without flicker problems and with the current exposure meeting the requirements can be obtained.
[0119] Optionally, the operation of generating a target image based on the photosensitive information of the photosensitive layer may further include:
[0120] Step 601: Control the image sensor to perform exposure to obtain the light-sensitive information.
[0121] Among them, exposure refers to the light passing through the deformation layer and entering the photosensitive layer, and the photosensitive layer begins to sense light and accumulate electrons.
[0122] Step 602: Generate a target image based on the light-sensitive information.
[0123] The photosensitive information may be the electrons converted by the photosensitive layer based on the incident light during the exposure process. After the exposure is completed, the accumulated electrons are read out based on the readout circuit to obtain the exposure image, that is, the target image.
[0124] Before the deformation of the deformation layer is controlled based on the target voltage, the method may further include:
[0125] Step 603, applying a third voltage to the deformable element of the second deformable layer in the deformable layer along the second direction to adjust the deformation degree of the second deformable layer to a third deformation degree, wherein the amount of light input corresponding to the third deformation degree is 0; controlling all pixel units in the photosensitive layer to reset; the second direction is the arrangement direction of the multiple deformable elements.
[0126] Among them, the third deformation degree refers to the case where the deformation degree is 100%, that is, the interval between each deformable member in the above-mentioned second deformation layer is 0, at this time, the amount of light corresponding to the photosensitive layer is 0, and the photons that can be collected by the photosensitive layer are 0. Among them, a pixel unit in the photosensitive layer can be regarded as a pixel point (pixel), which can also be called a pixel collection point. The pixel units can form a pixel array, that is, the photosensitive layer is based on each pixel unit for light collection. Among them, reset refers to clearing the original electrons of each pixel unit row in the photosensitive layer and starting to read new photons to convert them into electrons. Optionally, the image sensor in the embodiment of the present invention can be a complementary metal oxide semiconductor active pixel sensor (CMOS Active pixelsensor), and each pixel unit row can be reset row by row. Specifically, it can be reset row by row based on the reset mechanism of rolling shutter exposure. Among them, rolling shutter exposure refers to exposing each pixel point row by row until all pixels are exposed. Since the electrons are read out row by row in rolling shutter exposure, when using rolling shutter exposure to shoot high-speed moving objects, problems such as image distortion are prone to occur.
[0127] Correspondingly, there is also global exposure, which means that all pixels are exposed and terminated at the same time. Understandably, global exposure does not have the side effects of rolling shutter exposure such as image distortion, but global exposure often requires more processing circuits and has a higher implementation cost. And because the space is compressed, the number of pixels is small, which has a negative impact on the clarity of the image.
[0128] Specifically, the image sensor in the embodiment of the present invention may adopt rolling shutter exposure. Each pixel unit row enters the photosensitive state and starts to sense light after the reset operation. However, since the amount of light entering the photosensitive layer is 0 at this time, the photons read by each pixel unit row are actually 0.
[0129] The operation of controlling the deformation of the deformation layer based on the target voltage may specifically include:
[0130] Step 604: When all pixel units in the photosensitive layer have been reset, based on the target voltage, control the second deformation layer in the deformation layer to deform.
[0131] Specifically, after all pixel units have completed the reset operation, that is, after all pixel units have entered the photosensitive state, a target voltage is applied to the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the deformation degree corresponding to the target voltage, that is, the amount of light entering the photosensitive layer is not 0. At this time, since all pixel unit rows have completed the reset operation, they can read photons at the same time and convert them into electrons waiting for output.
[0132] After controlling the image sensor to perform exposure, the method further includes:
[0133] Step 605: When the exposure time of at least one pixel unit row is the preset exposure time, apply the target voltage to the deformable elements in the second deformable layer along the second direction to adjust the deformation degree of the second deformable layer to the third deformation degree again.
[0134] Among them, the preset exposure time can be the above-mentioned adjusted exposure time, that is, an integer multiple of the preset period. At least one pixel unit row that meets the preset exposure time can be: the first pixel unit row to complete the reset operation. After the first pixel unit row completes the reset operation, timing can be performed. When the above-mentioned preset exposure time is reached, the second deformation layer is controlled to deform along the second direction to the above-mentioned third deformation degree, so that all pixel unit rows cannot read photons at the same time. Specifically, since each pixel unit row is photosensitive and read out row by row, that is to say, the time when each pixel unit row ends the photosensitive state is inconsistent, therefore, in order to ensure that all pixel unit rows end the photosensitive state at the same time, it can be achieved by adjusting the amount of light entering the photosensitive layer to 0. For example, Figure 7 is a timing diagram of realizing global exposure by using a deformable layer provided by an embodiment of the present invention, such as Figure 7 As shown, R represents resetting the pixel unit row, and O represents reading the accumulated electrons in the pixel unit row. The part before point a indicates that the deformation degree of the above-mentioned deformation layer is 100%, that is, the amount of light entering the photosensitive layer is 0, and all pixel unit rows start to be sensitive to light at the same time after point a. The part after point b indicates that the deformation degree of the above-mentioned deformation layer becomes 100% again, that is, the amount of light entering the photosensitive layer is 0, and all pixel unit rows end the sensitivity to light at the same time at point b. That is, the effect of a mechanical shutter is achieved. And the photosensitivity time of all pixel rows can be controlled through the deformation layer, so as to realize simultaneous exposure and simultaneous termination of exposure of all pixel rows. In this way, the hardware based on rolling shutter exposure can achieve the effect of global exposure, thereby avoiding problems such as image distortion, and does not require a high implementation cost.
[0135] It should be noted that, when the image sensor supports global reset, a global reset may be configured once to implement a reset operation on all pixel unit rows, thereby reducing frame rate sacrifice.
[0136] In an embodiment of the present invention, the image sensor is controlled to be exposed to obtain the photosensitive information; a target image is generated based on the photosensitive information; before the deformation layer is controlled to be deformed based on the target voltage, a third voltage is applied to the deformation member of the second deformation layer in the deformation layer along the second direction to adjust the deformation degree of the second deformation layer to a third deformation degree, and the amount of light input corresponding to the third deformation degree is 0; all pixel units in the photosensitive layer are controlled to be reset; the second direction is the arrangement direction of the multiple deformation members; the deformation layer is controlled to be deformed based on the target voltage, including: when all pixel units in the photosensitive layer have been reset, the second deformation layer in the deformation layer is controlled to be deformed based on the target voltage; after the image sensor is controlled to be exposed, the method further includes: when the exposure time of at least one pixel unit row is a preset exposure time, the target voltage is applied to the deformation member in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the third deformation degree again. In this way, by ensuring that all pixel unit rows are exposed and terminated at the same time, problems such as image distortion or inconsistent exposure caused by inconsistent exposure time of each pixel unit row can be avoided. At the same time, since global exposure can be achieved by simply setting a deformation layer, high production costs are not required and image clarity is guaranteed.
[0137] Optionally, before generating the target image based on the photosensitive information of the photosensitive layer, the embodiment of the present invention may further include the following operations:
[0138] Step 701, obtaining a dark current value and bad pixel distribution information corresponding to a corrected image; the corrected image is obtained by adjusting the deformation degree of the second deformation layer to a third deformation degree, and the amount of light input corresponding to the third deformation degree is 0.
[0139] Among them, the third deformation degree refers to the deformation degree of 100%, that is, the interval between each deformable element in the above-mentioned second deformation layer is 0, at this time the amount of light entering the photosensitive layer is 0, and the photons that can be collected by the photosensitive layer are 0. The corrected image refers to the image obtained when the second deformation layer is the third deformation degree, that is, the corrected image is the image obtained in the dark state. Specifically, the corrected image can be obtained by controlling the photosensitive layer to start exposure at the third deformation degree and reading the accumulated electrons.
[0140] Among them, dark current refers to the tiny current generated on the photosensitive layer when the amount of light entering the photosensitive layer is 0. Dark current is one of the main noise sources of image sensors such as photosensitive coupling elements. Accordingly, the dark current value refers to the size of the above-mentioned tiny current. The dark current value of the above-mentioned correction image can also be understood as obtaining background data through dark frame calibration. Among them, bad pixels refer to some pixel units due to process defects or errors in the process of converting light signals into electrical signals, which leads to inaccurate pixel values or even pixel loss in the image. These pixel units are called bad pixels. Correspondingly, the bad pixel distribution information refers to the specific location of the pixel unit with errors or defects in the photosensitive layer. The bad pixel distribution information can also include the coordinates of the bad line distribution. It can be understood that the bad line refers to the pixel row with errors or defects. Specifically, the above-mentioned dark current value can be obtained by detecting the noise existing in each pixel point or pixel area of the correction image, and the above-mentioned bad pixel distribution information can be obtained by detecting whether the pixel value in each pixel point or pixel area of the correction image is accurate. Among them, the dark current value and the bad pixel distribution information can be obtained by statistics pixel by pixel or pixel area by pixel at the AP / ISP end.
[0141] It is understandable that the above dark current and bad pixels will have a bad impact on the image quality and will destroy the clarity and integrity of the image. Therefore, in the embodiment of the present invention, the image can be corrected based on the dark current value and the bad pixel distribution information.
[0142] After the target image is generated based on the photosensitive information of the photosensitive layer, the following steps may be specifically included:
[0143] Step 702: Perform image correction on the target image based on the dark current value and the bad pixel distribution information.
[0144] Specifically, after obtaining the target image, the target image can be subjected to pedestal compensation based on the dark current value obtained in the above step S701, that is, the pixel values generated by the above dark current are sequentially subtracted based on the pixel units, thereby obtaining pixel values without dark current. Also, the corresponding pixel units are found based on the above bad pixel distribution information for compensation, thereby obtaining an image without the influence of dark current and bad pixels. Optionally, the corresponding pixel units can be found based on the above bad pixel distribution information and compensated using an spdc (static bad pixel compensation) algorithm. For example, after obtaining the above bad pixel distribution information, for the above bad pixel coordinates or bad line coordinates, the pixel values of the pixel units adjacent to the above bad pixel coordinates or bad line coordinates can be used as the pixel values of the pixel units indicated by the above bad pixel distribution information.
[0145] It should be noted that, since the image sensor may be affected by temperature, humidity and working hours, the dark current value and bad pixel distribution information may be unstable. Therefore, the above steps 701-702 can be performed every time a photo is taken, so that a higher quality target image can be obtained every time a photo is taken. Optionally, step 701 can also be performed before the image sensor leaves the factory to obtain the initial dark current value and bad pixel distribution information, so that when the user takes subsequent photos, the initial dark current value and bad pixel distribution information can be directly used to correct the image, which can improve the image quality to a certain extent while reducing the processing volume of the photo-taking link.
[0146] In the embodiment of the present invention, by obtaining the dark current value and bad pixel distribution information corresponding to the correction image; the correction image is obtained when the deformation degree of the second deformation layer is adjusted to the third deformation degree, and the amount of light input corresponding to the third deformation degree is 0; after the target image is generated based on the photosensitive information of the photosensitive layer, the method further includes: performing image correction on the target image based on the dark current value and the bad pixel distribution information. In this way, by obtaining the real-time dark current value and bad pixel distribution information of the image sensor, problems such as inaccurate pixel values caused by the presence of dark current can be solved, and the influence of bad pixels on image clarity can be reduced, thereby improving the image quality to a certain extent.
[0147] Alternatively, if Figure 8 As shown, the embodiment of the present application further provides an electronic device 800, including a processor 8001, a memory 8002 and an image sensor 8003 of the aforementioned embodiment, wherein the memory 8002 stores a program or instruction that can be run on the processor 8001, and the program or instruction implements the steps of the method of the aforementioned embodiment when executed by the processor 8001. The same technical effect can be achieved, and it will not be repeated here to avoid repetition.
[0148] It should be noted that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.
[0149] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0150] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, and a processor 910 and other components.
[0151] Those skilled in the art will appreciate that the electronic device 900 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 910 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0152] Among them, the sensor 905 may include an image sensor, which includes a deformation layer arranged on the light-entering side of the photosensitive layer, and the deformation layer includes a plurality of deformation members arranged at intervals; when a voltage is applied to the deformation layer along a preset direction, the deformation layer is deformed along the preset direction to change the light parameters of the light incident on the photosensitive layer; wherein, under different voltages, the deformation degree of the deformation layer is different; different voltage application directions correspond to changes in different light parameters. In this way, by setting a deformation layer on the light-entering side of the photosensitive layer and adjusting the deformation degree of the deformation layer, the light parameters of the light incident on the photosensitive layer can be dynamically adjusted, thereby realizing dynamic control of the light parameters.
[0153] Optionally, the deformation layer in the above-mentioned image sensor includes only the first deformation layer, or the deformation layer includes only the second deformation layer, or the deformation layer includes the first deformation layer and the second deformation layer; the preset direction includes the first direction and the second direction; the first direction is the stacking direction of the deformation layer and the photosensitive layer, and the second direction is the arrangement direction of the multiple deformation members, and the light parameters include wavelength and light input amount; when the first deformation layer is deformed along the first direction, the first deformation layer is used to change the wavelength of the light incident to the photosensitive layer; when the second deformation layer is deformed along the second direction, the first deformation layer is used to change the light input amount of the light incident to the photosensitive layer. In this way, the wavelength and light input amount of the light incident to the photosensitive layer can be controlled respectively by setting the first deformation layer and the second deformation layer.
[0154] Optionally, a positive electrode and a negative electrode are respectively arranged on the first side and the second side of each deformable element in the first deformable layer of the above-mentioned image sensor; the first side and the second side are two opposite sides in the first direction; a positive electrode and a negative electrode are respectively arranged on the third side and the fourth side of each deformable element in the second deformable layer; the third side and the fourth side are two opposite sides in the second direction; the image sensor also includes a first power supply unit, the positive and negative interfaces of the first power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the first side and the second side; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the first side and the second side along the first direction, so that the first deformable layer is deformed along the first direction; the image sensor also includes a second power supply unit, the positive and negative interfaces of the second power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side along the second direction, so that the second deformable layer is deformed along the second direction. In this way, by arranging positive and negative electrodes on the deformable member, the deformation degree of the deformable layer can be more conveniently controlled, thereby adjusting the light parameters of the light incident on the photosensitive layer and improving the adjustment efficiency.
[0155] Optionally, the deformable layer in the above-mentioned image sensor includes a third deformable layer, and the preset direction includes a first direction and a second direction; the first direction is the stacking direction of the deformable layer and the photosensitive layer, and the second direction is the arrangement direction of the multiple deformable members, and the light parameters include wavelength and light input amount; when the third deformable layer is deformed along the first direction, the third deformable layer is used to change the wavelength of the light incident to the photosensitive layer; when the third deformable layer is deformed along the second direction, the third deformable layer is used to change the light input amount of the light incident to the photosensitive layer. In this way, only one deformable layer is needed to achieve the effect of changing the wavelength and changing the light input amount, which can reduce the volume and production cost of the above-mentioned image sensor to a certain extent, thereby saving space of the above-mentioned electronic equipment and reducing production costs.
[0156] Optionally, the first side and the third side of each deformable element in the third deformable layer in the above-mentioned image sensor are provided with a positive electrode, and the second side and the fourth side of each deformable element in the third deformable layer are provided with a negative electrode; the first side and the second side are two opposite sides in the first direction, and the third side and the fourth side are two opposite sides in the second direction; the image sensor further comprises a first power supply unit, the positive and negative interfaces of the first power supply unit are respectively connected to the positive electrode and the negative electrode respectively provided on the first side and the second side; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively provided on the first side and the second side along the first direction, so that the first deformable layer is deformed along the first direction; the image sensor further comprises a second power supply unit, the positive and negative interfaces of the second power supply unit are respectively connected to the positive electrode and the negative electrode respectively provided on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively provided on the third side and the fourth side along the second direction, so that the second deformable layer is deformed along the second direction. The deformation degree of the deformable layer can be more conveniently controlled, thereby adjusting the light parameters of the light incident on the photosensitive layer, and improving the adjustment efficiency.
[0157] Optionally, the shape, size and spacing between each deformable element in the deformable layer of the above-mentioned image sensor are the same; the photosensitive layer includes a plurality of pixel units; the spacing between each two deformable elements is less than or equal to the width of a pixel unit. It is only necessary to apply a voltage to the entire deformable layer to make each deformable element produce the same degree of deformation, without applying different voltages to each deformable element, thereby improving the efficiency of controlling the degree of deformation of the deformable layer. At the same time, the spacing between the deformable elements is less than or equal to the width of a pixel unit, which can ensure that the changes in the light parameters of the light collected by each pixel unit are the same.
[0158] Optionally, the image sensor may further include a lens layer, and the deformation layer is disposed between the lens layer and the photosensitive layer, so as to increase the aperture ratio in terms of specifications, thereby improving the photosensitivity.
[0159] The processor 910 is used to obtain a target voltage; based on the target voltage, control the deformation layer to deform so as to change the light parameters of the light incident on the photosensitive layer; and generate a target image based on the photosensitivity information of the photosensitive layer.
[0160] In the image processing method provided by the embodiment of the present invention, the electronic device obtains a target voltage; based on the target voltage, controls the deformation of the deformation layer to change the light parameters of the light incident to the photosensitive layer; based on the photosensitive information of the photosensitive layer, generates a target image; in response to a photographing instruction, obtains a target voltage; based on the target voltage, controls the deformation of the deformation layer along a specified direction to make the light parameters of the light incident to the photosensitive layer reach preset parameters; performs an exposure operation based on the photosensitive layer; after the exposure operation is completed, generates a target image based on the exposure result of the exposure operation. In this way, only different target voltages need to be set to adjust the deformation degree of the deformation layer, thereby adjusting the light parameters of the light incident to the photosensitive layer, and realizing dynamic control of the light parameters.
[0161] Optionally, the deformation layer includes a first deformation layer, and the light parameters include wavelength; the processor 910 is also used to obtain a first target voltage corresponding to a red light wavelength, a second target voltage corresponding to a green light wavelength, and a third target voltage corresponding to a blue light wavelength based on a preset correspondence between wavelength and voltage; based on the target voltage, the deformation of the deformation layer is controlled to change the light parameters of the light incident to the photosensitive layer, including: in accordance with a preset order, the first target voltage, the second target voltage and the third target voltage are applied to the deformation member in the first deformation layer in sequence along a first direction to adjust the deformation degree of the first deformation layer so that the wavelength of the light incident to the photosensitive layer is changed to a red light wavelength, a green light wavelength and a blue light wavelength in sequence; wherein the first direction is the stacking direction of the first deformation layer and the photosensitive layer.
[0162] In an embodiment of the present invention, the deformation layer includes a first deformation layer, the light parameter includes a wavelength, and based on a preset correspondence between wavelength and voltage, a first target voltage corresponding to a red light wavelength, a second target voltage corresponding to a green light wavelength, and a third target voltage corresponding to a blue light wavelength are obtained; based on the target voltages, controlling the deformation of the deformation layer to change the light parameters of the light incident to the photosensitive layer includes: applying the first target voltage, the second target voltage, and the third target voltage to the deformation members in the first deformation layer in a first direction in sequence according to a preset order to adjust the deformation degree of the first deformation layer so that the wavelength of the light incident to the photosensitive layer changes to a red light wavelength, a green light wavelength, and a blue light wavelength in sequence; wherein the first direction is the stacking direction of the first deformation layer and the photosensitive layer. The preset parameters include red light wavelength, green light wavelength and blue light wavelength; and based on the preset wavelength-voltage correspondence, a first target voltage corresponding to the red light wavelength, a second target voltage corresponding to the green light wavelength and a third target voltage corresponding to the blue light wavelength are respectively obtained; based on the target voltage, the deformation of the deformation layer is controlled to change the light parameters of the light incident to the photosensitive layer, including: in accordance with a preset order, the first target voltage, the second target voltage and the third target voltage are applied to the deformation member in the first deformation layer in sequence along a first direction to adjust the deformation degree of the first deformation layer so that the wavelength of the light incident to the photosensitive layer is changed to the red light wavelength, the green light wavelength and the blue light wavelength in sequence; wherein the first direction is the stacking direction of the first deformation layer and the photosensitive layer. In this way, by controlling the deformable layer to deform three times along the stacking direction with the photosensitive layer to obtain red, green and blue component information respectively, each pixel in the photosensitive layer can directly obtain the color value (RGB value) of the image, without using the color restoration algorithm (Demosaicing Algorithm) in the prior art to calculate the RGB value, thereby avoiding the problem of inaccurate color information collected and improving the image quality.
[0163] Optionally, the deformation layer includes a second deformation layer, and the light parameters include the amount of light entering. The processor 910 is also used to apply a first voltage to the deformable elements in the second deformation layer along a second direction to adjust the deformation degree of the second deformation layer to the first deformation degree; collect a reference image through the image sensor; determine the shooting exposure based on the reference image; when the shooting exposure does not exceed a preset exposure threshold, based on a preset correspondence between the deformation degree and the voltage, determine the first voltage corresponding to the first deformation degree as the target voltage; wherein the second direction is the arrangement direction of the multiple deformable elements, and different deformation degrees correspond to different amounts of light entering.
[0164] In an embodiment of the present invention, the deformation layer includes a second deformation layer, the light parameter includes the amount of light entering, and the preset parameter is the first target amount of light entering; a first voltage is applied to the deformation strip deformation member in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the first deformation degree; a reference image is collected by the image sensor; based on the reference image, the shooting exposure is determined; when the exposure is less than or equal to the preset exposure threshold, based on the preset correspondence between the deformation degree and the voltage, the first voltage corresponding to the first deformation degree is determined as the target voltage; wherein the second direction is the arrangement direction of the multiple deformation members, and different deformation degrees correspond to different amounts of light entering. In this way, adjustments are made based on the collected reference image to obtain a suitable target voltage, thereby ensuring the accuracy of the target voltage to a certain extent, ensuring that the target voltage is more suitable for the current environment, and at the same time, retaining as much light entering as possible for the photosensitive layer without overexposure.
[0165] Optionally, the processor 910 is further configured to set the shooting exposure time to an integer multiple of a preset period when there is band noise in the reference image.
[0166] In the embodiment of the present invention, when there is band noise in the reference image, the shooting exposure time is set to an integer multiple of a preset period, thereby avoiding the problem of power frequency flicker.
[0167] Optionally, the processor 910 is also used to determine the target light input amount based on the preset exposure threshold, the image brightness of the reference image, the shooting exposure time and the shooting sensitivity when the shooting exposure exceeds a preset exposure threshold; determine a second deformation degree corresponding to the target light input amount based on a preset correspondence between the light input amount and the deformation degree; determine a second voltage corresponding to the second deformation degree; and apply the second voltage to the deformation element in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the second deformation degree to adjust the shooting exposure.
[0168] In an embodiment of the present invention, when the shooting exposure exceeds the preset exposure threshold, the target light intake is determined based on the preset exposure threshold, the image brightness of the reference image, the shooting exposure time, and the shooting sensitivity; based on the preset correspondence between the light intake and the deformation degree, the second deformation degree corresponding to the target light intake is determined; the second voltage corresponding to the second deformation degree is determined; and the second voltage is applied to the deformable member in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the second deformation degree to adjust the shooting exposure. In this way, the target light intake is calculated based on the preset exposure threshold, and the deformation degree corresponding to the target light intake is obtained through the preset correspondence between the light intake and the deformation degree, and the corresponding deformation degree is reset to the first deformation degree, so that in the case of the deformation degree corresponding to the target light intake, the preset exposure threshold can be met more quickly, thereby reducing the number of adjustments as much as possible.
[0169] Optionally, the processor 910 is also used to control the image sensor to be exposed to obtain the photosensitive information; generate a target image based on the photosensitive information; based on the target voltage, before the deformation layer is controlled to deform, a third voltage is applied to the deformation element of the second deformation layer in the deformation layer along the second direction to adjust the deformation degree of the second deformation layer to a third deformation degree, and the amount of light input corresponding to the third deformation degree is 0; control all pixel units in the photosensitive layer to reset; the second direction is the arrangement direction of the multiple deformation elements; when all pixel units in the photosensitive layer have been reset, based on the target voltage, control the second deformation layer in the deformation layer to deform; after controlling the image sensor to be exposed, when the exposure time of at least one pixel unit row is a preset exposure time, a target voltage is applied to the deformation element in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the third deformation degree again.
[0170] In an embodiment of the present invention, the image sensor is controlled to be exposed to obtain the photosensitive information; a target image is generated based on the photosensitive information; before the deformation layer is controlled to be deformed based on the target voltage, a third voltage is applied to the deformation member of the second deformation layer in the deformation layer along the second direction to adjust the deformation degree of the second deformation layer to a third deformation degree, and the amount of light input corresponding to the third deformation degree is 0; all pixel units in the photosensitive layer are controlled to be reset; the second direction is the arrangement direction of the multiple deformation members; the deformation layer is controlled to be deformed based on the target voltage, including: when all pixel units in the photosensitive layer have been reset, the second deformation layer in the deformation layer is controlled to be deformed based on the target voltage; after the image sensor is controlled to be exposed, the method further includes: when the exposure time of at least one pixel unit row is a preset exposure time, a target voltage is applied to the deformation member in the second deformation layer along the second direction to adjust the deformation degree of the second deformation layer to the third deformation degree again. In this way, by ensuring that all pixel unit rows are exposed and terminated at the same time, problems such as image distortion or inconsistent exposure caused by inconsistent exposure time of each pixel unit row can be avoided. At the same time, since global exposure can be achieved by simply setting a deformation layer, high production costs are not required and image clarity is guaranteed.
[0171] Optionally, the processor 910 is also used to obtain a dark current value and bad pixel distribution information corresponding to a corrected image before generating a target image based on the photosensitive information of the photosensitive layer; the corrected image is obtained when the deformation degree of the second deformation layer in the deformation layer is adjusted to a third deformation degree, and the amount of light input corresponding to the third deformation degree is 0; after generating the target image based on the photosensitive information of the photosensitive layer, image correction is performed on the target image based on the dark current value and the bad pixel distribution information.
[0172] In the embodiment of the present invention, by obtaining the dark current value and bad pixel distribution information corresponding to the correction image; the correction image is obtained when the deformation degree of the second deformation layer is adjusted to the third deformation degree, and the amount of light input corresponding to the third deformation degree is 0; after the target image is generated based on the photosensitive information of the photosensitive layer, the method further includes: performing image correction on the target image based on the dark current value and the bad pixel distribution information. In this way, by obtaining the real-time dark current value and bad pixel distribution information of the image sensor, problems such as inaccurate pixel values caused by the presence of dark current can be solved, and the influence of bad pixels on image clarity can be reduced, thereby improving the image quality to a certain extent.
[0173] It should be understood that in the embodiment of the present application, the input unit 904 may include a graphics processor (Graphics Processing Unit, GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 907 includes a touch panel 9071 and at least one of other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0174] The memory 909 can be used to store software programs and various data, including but not limited to application programs and operating systems. The memory 909 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 909 may include a volatile memory or a non-volatile memory, or the memory 909 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 909 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0175] The processor 910 may include one or more processing units; optionally, the processor 910 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 910.
[0176] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method of the aforementioned embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0177] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0178] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0179] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0180] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0181] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0182] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0183] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image sensor, characterized in that: The image sensor comprises a deformable layer arranged on the light-incoming side of the photosensitive layer, and the deformable layer comprises a plurality of deformable elements arranged at intervals; When a voltage is applied to the deformable layer along a preset direction, the deformable layer is deformed along the preset direction to change the light parameters of the light incident on the photosensitive layer; The light parameters include wavelength and light input amount; the preset direction includes a first direction and a second direction; the first direction is the stacking direction of the deformable layer and the photosensitive layer, and the second direction is the arrangement direction of the plurality of deformable elements; Wherein, under different voltages, the deformation degree of the deformation layer is different; Different voltage application directions correspond to changes in different light parameters; and when the deformable layer is deformed along the second direction, the intervals between the deformable members are changed to change the amount of light incident on the photosensitive layer.
2. The image sensor according to claim 1, characterized in that The deformation layer includes a first deformation layer and a second deformation layer; When the first deformable layer is deformed along the first direction, the first deformable layer is used to change the wavelength of light incident to the photosensitive layer; When the second deformable layer is deformed along the second direction, the first deformable layer is used to change the amount of light incident on the photosensitive layer.
3. The image sensor according to claim 2, characterized in that A positive electrode and a negative electrode are respectively arranged on the first side surface and the second side surface of each deformable element in the first deformable layer; the first side surface and the second side surface are two opposite surfaces in the first direction; A positive electrode and a negative electrode are respectively arranged on the third side surface and the fourth side surface of each deformable element in the second deformable layer; the third side surface and the fourth side surface are two opposite surfaces in the second direction; The image sensor further includes a first power supply unit, wherein the positive and negative interfaces of the first power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the first side surface and the second side surface; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the first side surface and the second side surface along the first direction, so that the first deformable layer is deformed along the first direction; The image sensor also includes a second power supply unit, the positive and negative interfaces of the second power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side along the second direction, so that the second deformation layer is deformed along the second direction.
4. The image sensor according to claim 1, characterized in that The deformation layer comprises a third deformation layer, When the third deformable layer is deformed along the first direction, the third deformable layer is used to change the wavelength of light incident on the photosensitive layer; When the third deformable layer is deformed along the second direction, the third deformable layer is used to change the amount of light incident on the photosensitive layer.
5. The image sensor according to claim 4, characterized in that: A positive electrode is disposed on the first side and the third side of each deformable element in the third deformable layer, and a negative electrode is disposed on the second side and the fourth side of each deformable element in the third deformable layer; The first side surface and the second side surface are two opposite surfaces in the first direction, and the third side surface and the fourth side surface are two opposite surfaces in the second direction; The image sensor further includes a first power supply unit, wherein the positive and negative interfaces of the first power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the first side surface and the second side surface; the first power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the first side surface and the second side surface along the first direction, so that the third deformation layer is deformed along the first direction; The image sensor also includes a second power supply unit, the positive and negative interfaces of the second power supply unit are respectively connected to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side; the second power supply unit is used to supply power to the positive electrode and the negative electrode respectively arranged on the third side and the fourth side along the second direction, so that the third deformation layer is deformed along the second direction.
6. The image sensor according to claim 1, characterized in that: The shape and size of each deformable element in the deformable layer and the distance between every two deformable elements are the same; The photosensitive layer includes a plurality of pixel units; the distance between every two deformable members is less than or equal to the width of a pixel unit.
7. The image sensor according to claim 1, characterized in that: The image sensor further includes a lens layer, and the deformation layer is arranged between the lens layer and the photosensitive layer.
8. A camera module, characterized in that: Comprising the image sensor as claimed in any one of claims 1 to 7.
9. An electronic device, characterized in that: Comprising the camera module as claimed in claim 8.
10. An image processing method, applied to an electronic device, characterized in that: The electronic device comprises an image sensor as claimed in any one of claims 1 to 7, and the method comprises: Get the target voltage; Based on the target voltage, controlling the deformable layer to deform so as to change light parameters of the light incident on the photosensitive layer; A target image is generated based on the photosensitive information of the photosensitive layer.
11. The method according to claim 10, characterized in that The deformation layer includes a first deformation layer, and the light parameter includes a wavelength; The obtaining of the target voltage comprises: Based on the preset correspondence between wavelength and voltage, a first target voltage corresponding to the red light wavelength, a second target voltage corresponding to the green light wavelength, and a third target voltage corresponding to the blue light wavelength are obtained; The step of controlling the deformation of the deformation layer based on the target voltage to change the light parameters of the light incident on the photosensitive layer includes: applying the first target voltage, the second target voltage, and the third target voltage to the deformable member in the first deformable layer in sequence along a first direction in a preset order to adjust the deformation degree of the first deformable layer so that the wavelength of the light incident on the photosensitive layer changes to a red light wavelength, a green light wavelength, and a blue light wavelength in sequence; The first direction is the stacking direction of the first deformation layer and the photosensitive layer.
12. The method according to claim 10, characterized in that The deformation layer includes a second deformation layer, and the light parameter includes an amount of incoming light; The obtaining of the target voltage comprises: Applying a first voltage to the deformable element in the second deformable layer along a second direction to adjust the deformation degree of the second deformable layer to a first deformation degree; Acquiring a reference image by means of the image sensor; Determining shooting exposure based on the reference image; In a case where the shooting exposure does not exceed a preset exposure threshold, based on a preset correspondence relationship between the deformation degree and the voltage, determining a first voltage corresponding to the first deformation degree as a target voltage; The second direction is the arrangement direction of the plurality of deformable elements, and different deformation degrees correspond to different amounts of incoming light.
13. The method according to claim 12, characterized in that Before generating a target image based on the photosensitive information of the photosensitive layer, the method further includes: When there is band noise in the reference image, the shooting exposure time is set to an integer multiple of a preset period.
14. The method according to claim 12, characterized in that After determining the shooting exposure based on the reference image, the method further includes: When the shooting exposure exceeds a preset exposure threshold, determining a target light input amount based on the preset exposure threshold, the image brightness of the reference image, the shooting exposure time, and the shooting sensitivity; Based on a preset correspondence between the amount of incoming light and the degree of deformation, determining a second degree of deformation corresponding to the target amount of incoming light; determining a second voltage corresponding to the second deformation degree; The second voltage is applied to the deformable element in the second deformable layer along the second direction to adjust the deformation degree of the second deformable layer to the second deformation degree, so as to adjust the shooting exposure.
15. The method according to claim 10, characterized in that The step of generating a target image based on the photosensitive information of the photosensitive layer comprises: Controlling the image sensor to perform exposure to obtain the light-sensitive information; Based on the light-sensitive information, generating a target image; Before controlling the deformation of the deformation layer based on the target voltage, the method further includes: Applying a third voltage to the deformable element of the second deformable layer in the deformable layer along the second direction to adjust the deformation degree of the second deformable layer to a third deformation degree, wherein the amount of light entering the third deformation degree corresponds to 0; controlling all pixel units in the photosensitive layer to reset; the second direction is the arrangement direction of the plurality of deformable elements; The step of controlling the deformation layer to deform based on the target voltage includes: When all pixel units in the photosensitive layer have been reset, controlling the second deformable layer in the deformable layer to deform based on the target voltage; After controlling the image sensor to perform exposure, the method further includes: When the exposure time of at least one pixel unit row is the preset exposure time, the target voltage is applied to the deformable elements in the second deformable layer along the second direction to adjust the deformation degree of the second deformable layer to the third deformation degree again.
16. The method according to claim 12, characterized in that Before generating a target image based on the photosensitive information of the photosensitive layer, the method further includes: Obtaining a dark current value and bad pixel distribution information corresponding to a corrected image; the corrected image is obtained when the deformation degree of the second deformation layer is adjusted to a third deformation degree, and the amount of light input corresponding to the third deformation degree is 0; After generating the target image based on the photosensitive information of the photosensitive layer, the method further includes: Based on the dark current value and the bad pixel distribution information, image correction is performed on the target image.
