Image sensor, imaging module and electronic device

Abstract

The present application discloses an image sensor, an imaging module, and an electronic device, which relate to the technical field of image processing. The image sensor includes a pixel array and a plurality of amplifiers, the pixel array includes a plurality of pixel groups arranged along a row direction and / or a column direction, each pixel group includes a plurality of pixels of different colors, each amplifier corresponds to a pixel, an input terminal of a target amplifier is connected to an output terminal of the target pixel, and an output terminal of the target amplifier is connected to a column line on which the target pixel is located, the target amplifier is any one of the plurality of amplifiers, and the target pixel is a pixel corresponding to the target amplifier.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This invention claims priority to a Chinese patent application filed with the China Patent Office on November 30, 2020, bearing application number 202011373308.2 and entitled "Image Sensor, Imaging Module and Electronic Device," the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of image processing, and specifically to an image sensor, an imaging module, and an electronic device. [Background technology]

[0003] With the popularity of mobile terminal products, the market competition for mobile terminal products is also becoming increasingly fierce, especially in smartphone photography, for example, in improving the signal-to-noise ratio of smartphone images.

[0004] In the related art, when increasing the signal-to-noise ratio of an image, on the one hand, the actual signal-to-noise ratio of the image sensor is increased as much as possible, and on the other hand, noise is suppressed as much as possible when the raw image output by the image sensor is processed using an image algorithm in an ISP (Image Signal Processor).

[0005] However, when the image algorithm in the ISP is adopted to process the raw image, more noise is often introduced, which leads to image degradation. Summary of the Invention [Problem to be solved by the invention]

[0006] The present application aims to provide an image sensor, an imaging module, and an electronic device for reducing noise that causes image degradation due to the adoption of an image algorithm in an ISP to process raw images. [Means for solving the problem]

[0007] In order to solve the above technical problems, the present application is realized as follows.

[0008] According to a first aspect, an embodiment of the present application proposes an image sensor, the image sensor including a pixel array and a plurality of amplifiers, the pixel array includes a plurality of pixel groups arranged along a row direction and / or a column direction, each of the pixel groups includes a plurality of pixels of different colors, each of the amplifiers corresponds one-to-one to each of the pixels, an input end of a target amplifier is connected to an output end of the target pixel, and an output end of the target amplifier is connected to a column line in which the target pixel is located, the target amplifier is any one of the plurality of amplifiers, and the target pixel is a pixel corresponding to the target amplifier.

[0009] According to a second aspect, an embodiment of the present application proposes an imaging module, which includes the image sensor according to the first aspect, and a lens group in the imaging module is disposed corresponding to the image sensor.

[0010] According to a third aspect, an embodiment of the present application proposes an electronic device, said electronic device including the image sensor according to the first aspect. Effect of the Invention

[0011] In an embodiment of the present application, an amplifier is added to the image sensor to realize amplification processing for each pixel channel, so that the image amplification operation in the ISP is realized in the image sensor located at a more upstream stage, thereby reducing noise components from an earlier stage, suppressing image noise, and improving the final image quality signal-to-noise ratio and user experience. [Brief description of the drawings]

[0012] The above and / or additional aspects and advantages of the present application will become apparent and easier to understand from the following detailed description of the embodiments taken in conjunction with the drawings, in which: [Figure 1a] 2 is a schematic diagram of pixels included in a pixel group according to an exemplary embodiment of the present application; [Figure 1b] 2 is a schematic diagram of pixels included in a pixel group according to an exemplary embodiment of the present application; [Diagram 2] FIG. 1 is a structural schematic diagram of an image sensor according to an exemplary embodiment of the present application; [Diagram 3] FIG. 2 is an exploded schematic diagram of an imaging module according to an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following describes in detail the embodiments of the present application, examples of which are shown in the drawings, in which the same or similar reference numerals throughout represent the same or similar elements, or elements having the same or similar functions. The embodiments described below with reference to the drawings are illustrative and are used only to interpret the present application, and should not be understood as limitations on the present application. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without any creative effort belong to the scope of protection of the present application.

[0014] In the specification and claims of this application, the terms "first" and "second" features may include, explicitly or implicitly, one or more of these features. In the description of this application, unless otherwise specified, "plurality" means two or more. Note that in the specification and claims, "and / or" refers to at least one of the connected objects, and the character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0015] In the description of the present application, it should be understood that the orientations or positional relationships indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. are orientations or positional relationships shown based on the drawings, and are merely for the convenience and simplification of the description of the present application, and do not indicate or imply that the referred devices or elements must have a specific orientation and be configured and operated in a specific orientation, and therefore should not be understood as limitations on the present application.

[0016] In the description of the present application, it should be explained that unless otherwise clearly defined or limited, the terms "attached", "connected" and "connected" should be understood in a broad sense, for example, it may be a fixed connection, a removable connection, or an integral connection. It may be a mechanical connection or an electrical connection. It may be a direct connection, an indirect connection through an intermediate medium, or an internal communication between two elements. Those skilled in the art can understand the specific meaning of the above terms in the present application according to the specific situation.

[0017] An exemplary embodiment of the present application provides an image sensor, which is a semiconductor usable to record changes in light, where the image sensor may be a Complementary Metal Oxide Semiconductor (CMOS) image sensor, a Charge Coupled Device (CCD) image sensor, or a quantum thin film image sensor, etc.

[0018] In this embodiment, the image sensor may include a pixel array and an amplifier, the pixel array may include a plurality of pixel groups (e.g., at least two pixel groups) arranged along a row direction and / or a column direction, each of the pixel groups may include a plurality of different color pixels, e.g., red-sensitive pixels, green-sensitive pixels, blue-sensitive pixels, etc., and each of the amplifiers may correspond one-to-one to each of the pixels.

[0019] Here, the input end of the target amplifier is connected to the output end of the target pixel, and the output end of the target amplifier is connected to the column line where the target pixel is located, the target amplifier is one of the plurality of amplifiers, and the target pixel is a pixel corresponding to the target amplifier. As a result, the pixel channel corresponding to the target pixel can be controlled, and the image sensor has a function of realizing image processing (e.g., luminance uniformity (lens shading processing, etc.) by realizing individual amplification of each pixel channel. In other words, the present application adds an amplifier to the image sensor, thereby realizing the image amplification operation in the ISP in the image sensor located at a more front stage, reducing noise components from a more front stage, suppressing image noise, and improving the final image quality signal-to-noise ratio and user experience.

[0020] In this embodiment, the target pixel may be, but is not limited to, a red-sensitive pixel, a blue-sensitive pixel, or a green-sensitive pixel. As can be understood, in each of the pixel groups, the different color pixels included therein may be composed of at least one sub-pixel. For example, assuming that the pixel group includes a red-sensitive pixel, a green-sensitive pixel, and a blue-sensitive pixel, taking the red-sensitive pixel as an example, the red-sensitive pixel may include a total of four red-sensitive sub-pixels R1, R2, R3, and R4 as shown in FIG. 1a, or may include only one red-sensitive pixel R1 as shown in FIG. 1b, and this embodiment is not limited thereto. In addition, in the pixel array, the arrangement manner of the different color pixels in each of the pixel groups may be the same or different.

[0021] Furthermore, the amplifier may be, but is not limited to, an operational amplifier, etc. The gain of the amplifier may be set according to actual needs, for example, when the shooting environment changes, the gain of the amplifier can be adjusted to ensure image quality, that is, the gain of the amplifier is adjustable.

[0022] In one implementation, depending on the amplification factor requirements of each pixel channel, one amplifier may only amplify the output of a target pixel (e.g., a red-sensitive pixel, a blue-sensitive pixel, or a green-sensitive pixel) in a certain row of the pixel array, or multiple amplifiers may amplify the output of target pixels in multiple rows, respectively, or multiple amplifiers corresponding to pixels of different colors may simultaneously amplify the output of target pixels of multiple different colors, which this embodiment is not limited to.

[0023] Based on the above embodiment, as one possible implementation, each of the pixel groups may include a first pixel, a second pixel and a third pixel, and correspondingly, the amplifier may include a first amplifier, a second amplifier and a third amplifier.

[0024] Here, the first amplifier has an input end connected to the output end of the first pixel and an output end connected to the column line where the first pixel is located, the second amplifier has an input end connected to the output end of the second pixel and an output end connected to the column line where the second pixel is located, and the third amplifier has an input end connected to the output end of the third pixel and an output end connected to the column line where the third pixel is located. Thus, by adding amplifiers corresponding to different pixels to the image sensor to amplify the outputs of different pixel channels, the image sensor has the function of controlling each pixel channel respectively, and further realizes image processing functions such as image brightness uniformization processing, so as to improve the final image quality signal-to-noise ratio and user experience. As can be seen, in this implementation, the first pixel, the second pixel, and the third pixel correspond to red-sensitive pixels, green-sensitive pixels, and blue-sensitive pixels, respectively, that is, the pixel group consisting of the first pixel, the second pixel, and the third pixel is a three-channel pixel group.

[0025] As another possible scheme, each of the pixel groups may include a first pixel, a second pixel, and a third pixel, the second pixel may include a first sub-pixel and a second sub-pixel, and correspondingly, the amplifier may include a first amplifier, a second amplifier, and a third amplifier, and the second amplifier may include a first sub-amplifier and a second sub-amplifier.

[0026] Here, the first amplifier has an input end connected to the output end of the first pixel and an output end connected to the column line in which the first pixel is located, the first sub-amplifier has an input end connected to the output end of the first sub-pixel and an output end connected to the column line in which the first sub-pixel is located, the second sub-amplifier has an input end connected to the output end of the second sub-pixel and an output end connected to the column line in which the second sub-pixel is located, and the third amplifier has an input end connected to the output end of the third pixel and an output end connected to the column line in which the third pixel is located. It can be understood that in this implementation, the first pixel and the third pixel correspond to red and blue light-sensitive pixels, respectively, and the first sub-pixel and the second sub-pixel both correspond to green light-sensitive pixels, that is, the pixel group consisting of the first pixel, the first sub-pixel, the second sub-pixel and the third pixel is a four-channel pixel group.

[0027] 2, the connection method of the amplifiers in the image sensor will be described by taking a four-channel pixel group as an example. If the first pixel included in the pixel group 20 is a red-sensitive pixel (R), the first and second sub-pixels are both green-sensitive pixels (Gr, Gb), and the third pixel is a blue-sensitive pixel (B), the output of the R pixel is connected to the input terminal of a first amplifier 11, the output of which is amplified through the first amplifier and then connected to the column line in which the R pixel is located, the output of the Gr pixel is connected to the input terminal of a first sub-amplifier 12, the output of which is amplified through the first sub-amplifier and then connected to the column line in which the Gr pixel is located, and the output of the Gb pixel is connected to the input terminal of a second sub-amplifier 13, the output of which is amplified through the first sub-amplifier and then connected to the column line in which the Gr pixel is located, and the output of the Gb pixel is connected to the input terminal of a second sub-amplifier 14. The output of the B pixel is connected to the input terminal of the third amplifier 14, the output of which is amplified through the second sub-amplifier and then connected to the column line where the Gb pixel is located, and the output of the B pixel is connected to the input terminal of the third amplifier 14, the output of which is amplified through the third amplifier and then connected to the column line where the B pixel is located, and finally the subsequent column amplification (column amplification is the amplification of all signals in the image sensor, and this column amplifier can only amplify all signals as a whole, and cannot amplify a channel individually) and conversion by the analog-to-digital converter (ADC) are completed by the column amplifier 30. In this case, the first amplifier, the first sub-amplifier, the second sub-amplifier, and the third amplifier control pixel channels corresponding to different colors, respectively, so that the image sensor can realize image processing functions such as brightness uniformity processing.

[0028] The amplification factors of the first amplifier, the second amplifier (or the first sub-amplifier, the second sub-amplifier), and the third amplifier described above may be set according to actual needs, and different examples will be described below.

[0029] In Example 1, when the ISP performs processing using the Lens Shading module, it is considered that the image luminance becomes darker from the center to the four corners due to the characteristics of lens shading. For this reason, in this embodiment, a luminance gain matrix required for lens shading compensation is calculated, and the luminance gain matrix is ​​further mapped to each of the amplifiers to realize the lens shading compensation function at the image sensor end. That is, in this embodiment, the amplification factor of each of the amplifiers may be determined based on a luminance gain matrix, and the luminance gain matrix is ​​obtained by an image processor (ISP) performing a luminance uniformity calculation on the raw image output by the image sensor. By realizing the lens shading compensation function at the image sensor end, it is possible to reduce noise components from an earlier stage, avoid lens shading compensation processing at the ISP end, suppress image noise, and improve the final image quality signal-to-noise ratio and user experience.

[0030] In one implementation, the implementation process may include the following: first, obtain a first raw image by an image sensor, then input the first raw image into an ISP processing module, and use a brightness uniformity algorithm to process the first raw image to obtain a brightness gain matrix; then set the brightness gain matrix as the amplification factor of each amplifier in the image sensor, and finally use the image sensor again to obtain a second raw image. In this case, the second raw image is an image after brightness uniformity processing, that is, the automatic brightness uniformity processing function at the image sensor end is realized. When the environment changes, the above steps can be repeated to complete the dynamic refresh of the brightness uniformity processing of the image sensor.

[0031] In example 2, when the first sub-pixel and the second sub-pixel are both green-sensitive pixels (e.g., Gr, Gb), the amplification factor of the first sub-amplifier is different from the amplification factor of the second sub-amplifier. For example, when the Gr channel and the Gb channel are processed with different amplification factors, the resolution of the image sensor is sacrificed, but the dynamic range of the image sensor can be increased. As can be seen, the closer the signals of the Gr channel and the Gb channel in the image sensor are, the higher the resolution of the image sensor is, and the better the image quality of the image signal output by the image sensor is.

[0032] Here, the fact that the amplification factor of the first sub-amplifier is different from the amplification factor of the second sub-amplifier may include that the amplification factor of the first sub-amplifier and the amplification factor of the second sub-amplifier have a multiplication relationship (difference in multiplication). For example, the amplification factor of the first sub-amplifier corresponding to the Gr channel and the amplification factor of the second sub-amplifier corresponding to the Gb channel may be set to have an integer multiple difference such as 2x, 3x, 4x, etc., but is not limited thereto. This effectively increases the dynamic range of the image sensor.

[0033] It should be noted that the multiple relationship may be determined according to a target shooting scene, and the target shooting scene may include a portrait shooting scene and / or a sky shooting scene, etc., and the present embodiment is not limited thereto.

[0034] For example, take a face and sky scene as an example, which is a face and sky scene that is often encountered when a user takes a selfie using the front camera of a smart product such as a mobile phone. Generally, the Gb channel needs to use 1x gain to ensure that the sky is properly exposed, but the face is underexposed. In this case, if the Gr channel and the Gb channel can support processing of different amplification factors, for example, the sky part (i.e., the sky scene) can be processed using 1x gain (1x gain) of the Gr channel, but the face part can be processed using 4x gain (1x gain) of the Gb channel, that is, the amplification factor of the first sub-amplifier and the amplification factor of the second sub-amplifier have a 4x relationship, so that both the sky part and the face part can be properly exposed, improving the image quality of the output image of the image sensor and ensuring the user experience.

[0035] It should be noted that in each of the above-mentioned implementations, the amplification factors of the multiple target amplifiers are the same or different, the multiple target amplifiers correspond to pixels of the same color, and the target amplifiers can be the first amplifier, the second amplifier (including the first sub-amplifier or the second sub-amplifier) ​​or the third amplifier. For example, taking the first amplifier as an example, the amplification factors of the multiple first amplifiers can be the same or different, and can be specifically set according to actual needs, and this embodiment is not limited thereto.

[0036] In addition to the image sensor provided in each of the above implementations, in one possible implementation, the image sensor may further include a micro-lens and a color filter, where the micro-lens (micro-lens / lens sheet) is used to focus light onto each photodiode in the pixel array, the color filter filters the incident light and allows only light of a wavelength band corresponding to the color of the color filter to pass through, and the photodiode generates an electric charge when it receives light and converts the light into an electrical signal.

[0037] In addition, an exemplary embodiment of the present application further provides an imaging module, which includes an image sensor according to the above-mentioned embodiment, and a lens group in the imaging module is installed corresponding to the image sensor so that a light ray passing through the lens group reaches the image sensor. As can be understood, the image sensor included in this embodiment has the same or similar technical features as the image sensor according to the above-mentioned embodiment, so that in order to avoid repetition, it will not be further described here.

[0038] Exemplarily, as shown in FIG. 3, the imaging module may further include, for example, a protective film 201, a lens 202, a voice coil motor 203, a support member 204, an infrared filter 205, an image sensor 206 according to the present application, a flexible printed circuit board (FPC) 207, and a connection member 208.

[0039] Here, the operational flow of the imaging module may include the following steps (1) to (4).

[0040] (1) The lens is used for focusing and focusing. The lens is wrapped and fixed in a voice coil motor, and the upper and lower ends of the voice coil motor are linked to elastic sheets. When focusing, an electromagnetic force is generated in the motor by energizing it, and this force is finally balanced with the elastic force of the elastic sheet. The position of the motor can be controlled by the magnitude of the energization, and the motor then pushes the lens to the focusing position.

[0041] The voice coil motor includes an upper cover, an upper spring sheet, a lower spring sheet, a housing, a coil, a magnet, a moving member, a base, and a terminal. Here, the upper cover plays a role of protecting the motor. The upper spring sheet generates a biasing force on the motor when it is deformed, and the sum of the biasing force by the lower spring sheet balances with the electromagnetic force. The housing is a main frame of the motor fixing part, and has a magnetic conductivity, which can improve the effective utilization rate of the magnet. When a current is passed through the coil, an upward thrust is generated by the magnetic field action of the magnet, and other members of the moving part are interlocked. The magnet generates a magnetic field, and the current-carrying coil generates an electromagnetic force by the magnetic field action, and the moving member carrier mounts the lens to interlock. The lower spring sheet generates a biasing force on the motor when it is deformed, and the sum of the biasing force by the upper spring sheet balances with the electromagnetic force. The base and the motor are directly assembled on the flexible printed circuit board, and the mobile phone supplies power to the motor through the terminal.

[0042] (2) Photosensitive by the image sensor 206 Light from the scene is projected onto a color filter (IR filter) in the image sensor 206, and the IR filter removes unnecessary light, thereby preventing false color / moire in the image sensor 206 and improving its effective resolution and color restoration. After passing through the IR filter, the light can be sensed by the image sensor 206.

[0043] After being exposed to light, the image sensor 206 converts the optical signal into an electrical signal, amplifies it, and converts it into a digital signal by an ADC to form a raw image, which is output to the image processor ISP.

[0044] (3) Process the image and feed it back to the sensor for processing (2) For the raw image obtained, auto exposure (AE), auto focus (AF), AWB, OB, lens sBy performing processing using an ISP algorithm module such as shading, a luminance gain matrix required for obtaining correct luminance uniformity processing is obtained, and the luminance gain matrix is ​​set as the amplification gain (i.e., amplification factor) of the R channel (corresponding to the first amplifier), the Gr channel (corresponding to the first amplifier), the Gb channel (corresponding to the second sub-amplifier), and the B channel (corresponding to the third amplifier) ​​in the image sensor 206, and the setting of the luminance uniformity processing function at the image sensor 206 end is completed. If the environment changes, the above steps can be repeated to complete the dynamic refresh of the luminance uniformity processing of the image sensor 206.

[0045] The dynamic range of the sensor can also be increased by setting the amplification factor of the Gr channel (corresponding to the first sub-amplifier) ​​and the amplification factor of the Gb channel (corresponding to the second sub-amplifier) ​​to be the difference in the amplification factor.

[0046] (4) Process the image and compress it The raw image output by the image sensor 206 is re-input to the ISP processing, where it undergoes operations such as sharpening, noise reduction, color calibration, etc., and is then compressed into a JPEG image for storage or transmission to a display.

[0047] An exemplary embodiment of the present application further provides an electronic device, which includes an image sensor 206 according to the above-mentioned embodiments, and the image sensor 206 included in this embodiment has the same or similar technical features as the image sensor 206 according to the above-mentioned embodiments, so that in order to avoid repetition, it will not be further described here.

[0048] In the description of this specification, reference to the terms "one embodiment," "several embodiments," "exemplary embodiments," "example," "specific example," or "several examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the description of this specification, exemplary expressions for the above terms do not necessarily refer to the same embodiment or example. And, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0049] Although embodiments of the present application have been shown and described, as will be understood by those skilled in the art, various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is limited by the claims and their equivalents.

Claims

1. An image sensor comprising a pixel array and a plurality of amplifiers, the pixel array including a plurality of pixel groups arranged along a row direction and / or a column direction, each of the pixel groups including a plurality of pixels of different colors, each of the amplifiers corresponding to each of the pixels in a one-to-one correspondence; an input terminal of the amplifier is connected to an output terminal of the pixel, and an output terminal of the amplifier is connected to a column line in which the pixel is located; the plurality of amplifiers includes a second amplifier, the second amplifier including a first sub-amplifier and a second sub-amplifier; Each of the pixel groups includes a second pixel, the second pixel including a first sub-pixel and a second sub-pixel; the first sub-amplifier has an input terminal connected to an output terminal of the first sub-pixel and an output terminal connected to a column line on which the first sub-pixel is located; the second sub-amplifier has an input terminal connected to an output terminal of the second sub-pixel and an output terminal connected to a column line on which the second sub-pixel is located; the first sub-pixel and the second sub-pixel are both green-sensitive pixels, and an amplification factor of the first sub-amplifier is different from an amplification factor of the second sub-amplifier; An image sensor, wherein an amplification factor of the first sub-amplifier and an amplification factor of the second sub-amplifier have a multiplication relationship, the multiplication relationship being determined according to a target photographing scene.

2. the plurality of amplifiers include a first amplifier and a third amplifier, and each of the pixel groups includes a first pixel and a third pixel; the first amplifier has an input terminal connected to an output terminal of the first pixel and an output terminal connected to a column line in which the first pixel is located; 2. The image sensor according to claim 1, wherein the third amplifier has an input terminal connected to an output terminal of the third pixel and an output terminal connected to a column line in which the third pixel is located.

3. The image sensor of claim 1 , wherein the target scene comprises a portrait scene and / or a sky scene.

4. 3. The image sensor according to claim 1, wherein the plurality of amplifiers include a plurality of target amplifiers corresponding to pixels of the same color, and the amplification factors of the plurality of target amplifiers corresponding to the pixels of the same color are the same.

5. 3. The image sensor of claim 1, wherein the amplification factor of each of the amplifiers is determined based on a brightness gain matrix, the brightness gain matrix being obtained by an image processor ISP performing a brightness uniformity calculation on a raw image output by the image sensor.

6. 3. The image sensor according to claim 1, wherein the amplification factor of each of the amplifiers is adjustable.

7. 7. An imaging module comprising: a lens group; and the image sensor according to claim 1, wherein the lens group is disposed in correspondence with the image sensor.

8. An electronic device, comprising the image sensor according to claim 1 .

Images