Image processor, camera module, electronic equipment and photographing method

Abstract

The invention discloses an image processor, a camera module, electronic equipment and a photographing method. The image processor comprises at least one control unit, at least two selection units and a pixel array, the pixel array comprises at least two pixel blocks; each selection unit is connected with one control unit, and each selection unit is respectively connected with the control ends of at least one row of pixel units or at least one column of pixel units in the pixel array; wherein different selection units are connected with pixel units in different rows or pixel units in different columns in the pixel array; under the condition that the control unit receives a first control signal, pixel blocks corresponding to at least one selection unit in the pixel array are controlled to be opened through the at least one selection unit; and under the condition that the control unit receives a second control signal, controlling the pixel block corresponding to at least one selection unit in the pixel array to be closed through the at least one selection unit.

Description

Technical Field

[0001] The present application belongs to the field of camera technology, and specifically relates to an image processor, a camera module, an electronic device and a photographing method. Background Art

[0002] In a photo scene, it is often encountered that the brightness of multiple objects in the same photo scene is different. For example, the same photo scene contains grass and sky, but the grass is darker than the sky. Since one exposure time is usually used to take a photo, this may result in that only a part of the objects in the photo meet the exposure time requirements. For example, in the same scene, the grass and sky require a longer exposure time to capture more details because the grass area is a dark area, but the sky area requires a short exposure time due to the strong light. If the exposure time of the grass is met, the details of the sky area will be lost. At present, in order to ensure the photographic effect of all objects in the entire image, the commonly used solution is to take multiple frames of photos at different exposure times in the same photo scene, and then obtain a photo through fusion processing. This method requires superimposing complex algorithms for processing, resulting in low photography efficiency and poor image quality. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image processor, a camera module, an electronic device and a photographing method, which can improve photographing efficiency and imaging quality.

[0004] In a first aspect, an embodiment of the present application provides an image sensor, comprising: at least one control unit, at least two selection units and a pixel array; the pixel array comprises at least two pixel blocks;

[0005] Each of the selection units is connected to one of the control units, and each of the selection units is respectively connected to a control end of at least one row of pixel units or at least one column of pixel units in the pixel array; wherein different selection units are connected to pixel units in different rows or different columns of pixel units in the pixel array;

[0006] Wherein, when the control unit receives the first control signal, the pixel block corresponding to the at least one selection unit in the pixel array is controlled to be turned on through at least one selection unit;

[0007] When the control unit receives the second control signal, the pixel block corresponding to the at least one selection unit in the pixel array is controlled to be turned off through the at least one selection unit;

[0008] Each pixel block includes 4*K pixel units, where K is a positive integer.

[0009] In a second aspect, an embodiment of the present application provides a camera module, comprising the image sensor as described above.

[0010] In a third aspect, an embodiment of the present application provides an electronic device, comprising the camera module as described above.

[0011] In a fourth aspect, an embodiment of the present application provides a photographing method, which is performed by the electronic device as described in the third aspect above, and the method includes:

[0012] Determining the exposure time of each pixel block in the pixel array according to an initial image of the photographed object;

[0013] sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block;

[0014] When all pixel units in each pixel block are turned off, reading pixel data of each pixel block;

[0015] An image of the photographed object is generated according to the pixel data of each pixel block.

[0016] In a fifth aspect, an embodiment of the present application provides a photographing device, which is executed by the electronic device as described in the third aspect above, and the device includes:

[0017] A determination module, used to determine the exposure time of each pixel block in the pixel array according to an initial image of the photographed object;

[0018] A first sending module, used for sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block;

[0019] A reading module, used for reading pixel data of each pixel block when all pixel units in each pixel block are turned off;

[0020] A processing module is used to generate an image of the photographed object according to the pixel data of each pixel block.

[0021] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the photographing method described in the fourth aspect are implemented.

[0022] In a seventh aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the photographing method described in the fourth aspect are implemented.

[0023] In an eighth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the photographing method described in the fourth aspect above.

[0024] In a ninth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the photographing method as described in the fourth aspect.

[0025] In the embodiment of the present application, at least one control unit and at least two selection units are provided in the image sensor, and each selection unit can be connected to at least one row of pixel units or at least one column of pixel units, that is, each selection unit can be connected to part of the pixel units in the pixel array, so that at least two selection units are provided in the image sensor, it is possible to divide all the pixel units in the pixel array into at least two pixel blocks for independent control. When the control unit receives a first control signal, the pixel block corresponding to the at least one selection unit in the pixel array can be controlled to turn on through the at least one selection unit; when the control unit receives a second control signal, the pixel block corresponding to the at least one selection unit in the pixel array can be controlled to turn off through the at least one selection unit. In this way, it is possible to independently control the turning on and off of different pixel blocks in the pixel array, and the turning on and off controls of any two pixel blocks do not affect each other, that is, at least two pixel blocks can be set in one pixel array, and different exposure times can be set for different pixel blocks. Therefore, when using the image sensor to take photos, different exposure times can be set for different pixel blocks to ensure that different exposure requirements of different subjects can be met in one frame of the photo, avoiding the problem of low photography efficiency and image quality caused by multi-frame image synthesis processing, thereby improving photography efficiency and image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic diagram of an image sensor according to some embodiments of the present application;

[0027] Figure 2 is a schematic diagram of the circuit structure of a pixel unit in some embodiments of the present application;

[0028] Figure 3A is a schematic diagram of a pixel block of some embodiments of the present application;

[0029] Figure 3B is a schematic diagram of a pixel block of some embodiments of the present application;

[0030] Figure 3C is a schematic diagram of a pixel block of some embodiments of the present application;

[0031] Figure 3D is a schematic diagram of a pixel block of some embodiments of the present application;

[0032] Figure 3E is a schematic diagram of a pixel block of some embodiments of the present application;

[0033] Figure 4 is a schematic diagram of an image sensor according to some embodiments of the present application;

[0034] Figure 5 is a schematic diagram of a shooting interface of some embodiments of the present application;

[0035] Figure 6 is a schematic diagram of pixel area division in some embodiments of the present application;

[0036] Figure 7 is a schematic diagram of pixel area division in some embodiments of the present application;

[0037] Figure 8 It is a schematic diagram of images captured by some embodiments of the present application.

[0038] Fig. 9 is a flow chart of a photographing method in some embodiments of the present application;

[0039] Fig.10 is a schematic diagram of a shooting interface of some embodiments of the present application;

[0040] Fig.11 is a schematic diagram of partitioning an initial image in some embodiments of the present application;

[0041] Fig.12 is a schematic diagram of a circuit structure for realizing photo taking control in some embodiments of the present application;

[0042] Fig.13 is a schematic diagram of images captured by some embodiments of the present application;

[0043] Fig.14 is a block diagram of a photographing device in some embodiments of the present application;

[0044] Fig.15 is a block diagram of an electronic device of some embodiments of the present application;

[0045] Fig.16 It is a schematic diagram of the hardware structure of an electronic device of some embodiments of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0047] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0048] like Figure 1 As shown, an embodiment of the present application provides an image sensor, including: at least one control unit 1, at least two selection units 2 and a pixel array 3; the pixel array 3 includes at least two pixel blocks.

[0049] Each selection unit 2 is connected to a control unit 1, and each selection unit 2 is respectively connected to the control end of at least one row of pixel units or at least one column of pixel units in the pixel array 3; wherein different selection units 2 are connected to pixel units in different rows or different columns of pixel units in the pixel array 3.

[0050] In which, when the control unit 1 receives a first control signal, the pixel block corresponding to the at least one selection unit 2 in the pixel array 3 is controlled to be turned on through at least one selection unit 2; when the control unit 1 receives a second control signal, the pixel block corresponding to the at least one selection unit 2 in the pixel array 3 is controlled to be turned off through at least one selection unit 2.

[0051] Each pixel block includes 4*K pixel units, where K is a positive integer.

[0052] Optionally, the image sensor is an optical device, which is a core component of a camera module or a camera, and is used to convert light signals into electrical signals. For example, the types of image sensors include, but are not limited to, charge coupled device (CCD) sensors, complementary metal oxide semiconductor (CMOS) sensors, etc. Among them, CMOS sensors have the advantages of low power consumption, low cost and fast reading speed compared to CCD sensors.

[0053] Working principle of CMOS sensor: CMOS sensor is manufactured by semiconductor technology, with silicon-based materials as substrate, and photodiodes capture photons and convert them into electrical signals. In order to capture color images, the surface of the sensor is covered with a color filter array (CFA). An optional arrangement is to include red (R), green (Gr, Gb), and blue (B) filters, with green occupying half of the area because the human eye is more sensitive to green. Microlenses can concentrate incident light, reduce optical crosstalk, and improve image quality.

[0054] Optionally, the pixel array 3 is composed of X rows and Y columns of pixel units, where X and Y are both positive integers greater than 1, and each pixel unit captures a light signal and converts it into an electrical signal. Figure 1 Each square in the figure represents a photodiode covered by a filter of a different color, for example, square R represents red, squares Gr and Gb represent green, and square B represents blue. The photodiodes may be arranged in a RGB Bayer arrangement, or in other arrangements, and the embodiments of the present application are not limited thereto.

[0055] like Figure 2 As shown, a schematic diagram of the circuit structure of a pixel unit is given, including: a photodiode (Photodiode, PD), a reset switch (Reset Transistor, RST), a transfer gate transistor (Transfer Gate, TG), a floating diffusion area (Floating Diffusion, FD), a source follower transistor (Source Follower, SF), and a row selector switch (Row Selector, RSL).

[0056] Among them, the photodiode PD is used to convert incident photons into charges or electrons. The charges generated in this process will be stored and used for subsequent signal processing; the reset switch RST is used to reset the floating diffusion area FD to a fixed voltage, such as usually resetting to the power supply voltage VDD. This process clears the previously stored charges in the floating diffusion area and prepares for the next charge accumulation; the transfer gate transistor TG is used to control the transfer of charges from the photodiode PD to the floating diffusion area FD. When the transfer gate transistor TG is turned on, the charges accumulated in the photodiode will be transferred to the floating diffusion area FD; the floating diffusion area FD is used to store the charges transferred from the photodiode. The charges are converted into voltage signals here and amplified by the source follower transistor SF; the source follower transistor SF is used to amplify the voltage signal of the floating diffusion area FD and output it to the next level circuit. The source follower transistor SF usually provides high input impedance and low output impedance; the row selector switch RSL is used to control the final signal output path. The row selector switch RSL determines when the amplified voltage signal is passed to the signal output terminal Vout.

[0057] Specifically, the working principle of the pixel unit is as follows:

[0058] Reset stage: When the reset switch RST is turned on, the floating diffusion area FD is reset to the voltage value of the power supply voltage VDD, and the charge signal is cleared.

[0059] Charge accumulation and transfer stage: The photodiode PD accumulates charges under light, and when the transfer gate transistor TG is turned on, the accumulated charges are transferred from the photodiode to the floating diffusion region FD.

[0060] Signal amplification and output stage: The charge in the floating diffusion area FD is amplified by the source follower transistor SF to generate a corresponding voltage signal; the row selector switch RSL controls whether to output the amplified signal to the next level processing circuit.

[0061] Optionally, each selection unit 2 may be connected to at least one row of pixel units or at least one column of pixel units, so that at least one row of pixel units or at least one column of pixel units corresponding to each selection unit 2 may be independently controlled to be turned on and off by the selection unit 2. For example, at least one row of pixel units or at least one column of pixel units corresponding to each selection unit may constitute a pixel block, or at least one row of pixel units or at least one column of pixel units corresponding to each selection unit may constitute at least two pixel blocks, etc. In this way, by providing at least two selection units 2 in the image sensor, it is possible to divide all pixel units in the pixel array into at least two pixel blocks for independent control.

[0062] by Figure 2Taking the circuit structure diagram of the pixel unit shown as an example, the selection unit 2 is connected to the control end of the pixel unit, which can be the selection unit 2 connected to the control end of the reset switch RST, the control end of the transmission gate transistor TG, and the control end of the row selector switch RSL in the pixel unit.

[0063] Optionally, when the number of control units 1 is 1, the control unit 1 is respectively connected to each selection unit 2; or, when the number of control units 1 is greater than 1, each control unit 1 may be respectively connected to a part of the selection units 2, and each control unit 1 is respectively connected to different selection units 2, that is, each selection unit 2 is connected to one control unit 1.

[0064] In the embodiment of the present application, at least one control unit 1 and at least two selection units 2 are provided in the image sensor, and each selection unit 2 can be connected to at least one row of pixel units or at least one column of pixel units, that is, each selection unit 2 can be connected to part of the pixel units in the pixel array 3. In this way, by providing at least two selection units 2 in the image sensor, all the pixel units in the pixel array can be divided into at least two pixel blocks for independent control. When the control unit 1 receives a first control signal, the pixel block corresponding to the at least one selection unit 2 in the pixel array 3 can be controlled to turn on through the at least one selection unit 2; when the control unit 1 receives a second control signal, the pixel block corresponding to the at least one selection unit 2 in the pixel array 3 can be controlled to turn off through the at least one selection unit 2. In this way, different pixel blocks in the pixel array 3 can be turned on and off, and the on and off controls of any two pixel blocks do not affect each other, that is, at least two pixel blocks can be set in one pixel array, and different exposure times of different pixel blocks can be set. Therefore, when using the image sensor to take photos, different exposure times of different pixel blocks can be set to be different, so that different exposure requirements of different shooting objects can be met in one frame of the photo, avoiding the problem of relatively low shooting efficiency and imaging quality caused by the use of multi-frame image synthesis processing, thereby improving the shooting efficiency and imaging quality.

[0065] Optionally, dynamic range is a measure of an image sensor's ability to capture the darkest and brightest light, expressed in decibels (dB) or exposure values ​​(EV). The larger the dynamic range, the richer the picture details.

[0066] Optionally, when the control unit 1 receives the third control signal, the at least one selection unit 2 controls the pixel block corresponding to the at least one selection unit 2 in the pixel array 3 to be reset.

[0067] The pixel block reset means that the charge of the photoelectric conversion element of each pixel unit in the pixel block is cleared. Figure 2Taking the circuit structure diagram of the pixel unit shown as an example, the charge in the photodiode PD can be cleared by the reset switch RST, and the floating diffusion area FD can be reset to a fixed voltage, such as resetting to the power supply voltage VDD. That is, the charge of the photoelectric conversion element of the pixel unit is cleared, which can specifically refer to clearing the charge stored in the photodiode PD and resetting the floating diffusion area to the power supply voltage VDD, in preparation for the accumulation of charge after photoelectric conversion by the photodiode PD.

[0068] In this embodiment, the pixel block is reset to clear the charge stored in each pixel unit in the pixel block, thereby avoiding errors in the output pixel data due to the residual charge in the pixel unit, avoiding affecting the imaging quality, and ensuring the imaging effect.

[0069] Optionally, the at least one control unit 1 includes: a first control unit 11 and a second control unit 12; the at least two selection units 2 include: at least two row selection units 21 and at least two column selection units 22;

[0070] The first control unit 11 is connected to each row selection unit 21 respectively, each row selection unit 21 is connected to the control end of N rows of pixel units in the pixel array 3 respectively, and different row selection units 21 are connected to different rows of pixel units in the pixel array 3;

[0071] The second control unit 12 is connected to each column selection unit 22 respectively, each column selection unit 22 is connected to the control end of the M columns of pixel units in the pixel array 3 respectively, and different column selection units 22 are connected to the pixel units in different columns in the pixel array 3;

[0072] Wherein, when the first control unit 11 and the second control unit 12 receive the first control signal, the first control unit 11 and the second control unit 12 control the pixel blocks corresponding to the at least one row selection unit 21 and the at least one column selection unit 2 in the pixel array 3 to be turned on through the at least one row selection unit 21 and the at least one column selection unit 22;

[0073] When the first control unit 11 and the second control unit 12 receive the second control signal, the first control unit 11 and the second control unit 12 control the pixel blocks corresponding to the at least one row selection unit 21 and the at least one column selection unit 2 in the pixel array 3 to be turned off through at least one row selection unit 21 and at least one column selection unit 22.

[0074] Optionally, the first control unit 11 and the second control unit 12 may adopt a field programmable gate array (FPGA), or may adopt other control circuits other than these, but the embodiments of the present application are not limited thereto.

[0075] In this embodiment, two control units are provided, namely, a first control unit 11 and a second control unit 12, wherein the first control unit 11 is used to transmit a first control signal to a row selection unit 21 to select a row pixel unit, and the second control unit 12 is used to transmit a first control signal to a column selection unit 22 to select a column pixel unit, that is, the row selection unit 21 and the column selection unit 22 are independently controlled by the two control units. In this way, when at least one row selection unit 21 receives the first control signal of the first control unit 11, and at least one column selection unit 22 receives the first control signal of the second control unit 12, the row pixel unit corresponding to the at least one row selection unit 21 and the column pixel unit corresponding to the column selection unit 22 constitute the pixel unit. The block is turned on. Similarly, when at least one row selection unit 21 receives the second control signal of the first control unit 11 and at least one column selection unit 22 receives the second control signal of the second control unit 12, the pixel block composed of the row pixel units corresponding to the at least one row selection unit 21 and the column pixel units corresponding to the column selection unit 22 is turned off. Therefore, two control units, at least two row selection units 21 and at least two column selection units 22 can divide part of the column pixel units or part of the row pixel units in the pixel array into one pixel block, thereby improving the flexibility of the pixel block division method of the pixel array, so that the exposure time of the pixel block containing part of the column pixel units or part of the row pixel units in the pixel array can be individually controlled.

[0076] Optionally, a pixel block consisting of row pixel units corresponding to at least one row selection unit 21 and column pixel units corresponding to the column selection unit 22 includes at least 4 pixel units, that is, each pixel block includes 4*K pixel units, where K is a positive integer.

[0077] For example, the 4*K pixel units can be arranged in a row. For example, when K is 1, as an optional arrangement, Figure 3A As shown; or these 4*K pixel units can be arranged in a row, for example, when K is 2, as an optional arrangement, such as Figure 3B As shown; or these 4*K pixel units can be arranged in a rectangular shape, for example, when K is 4, as an optional arrangement method, Figure 3C As shown; or these 4*K pixel units can be arranged in other irregular shapes, such as Figure 3D , Figure 3E As shown, the embodiments of the present application are not limited thereto.

[0078] Optionally, the number of the row selection units 21 is greater than or equal to 12, and the number of the column selection units 22 is greater than or equal to 9.

[0079] Optionally, N is less than or equal to 341, and M is less than or equal to 341.

[0080] Taking the pixel array including 4096*3072 pixel units as an example, 4096 rows of pixel units can be divided into at least 12 parts, each part is controlled by a row selection unit 21 with at most 341 rows of pixel units, and 3072 columns of pixel units can be divided into at least 9 parts, each part is controlled by a column selection unit 22 with at most 341 columns of pixel units. That is, the entire pixel array 3 is divided into at least 12*9 pixel blocks, that is, each pixel block contains at most 341 columns and 341 rows of pixel units. The specific number of rows and columns of pixel units contained in each pixel block can be determined based on the total number of rows and columns of pixel units contained in the pixel array, and the number of row selection units and column selection units required for actual design, and the embodiments of the present application are not limited thereto.

[0081] It should be noted here that the total number of pixel units, the number of rows and the number of columns set in the pixel array in the embodiments of the present application are all exemplary descriptions, and the number of row selection units 21, the number of column selection units 22, the number of pixel unit rows controlled by each row selection unit 21, the number of pixel unit columns controlled by each column selection unit 22, etc. are also exemplary descriptions, and the embodiments of the present application are not limited to this.

[0082] Optionally, the row selection unit 21 includes: a first decoder 211 and a row multiplexing circuit 212; the first control unit 11 is connected to the row multiplexing circuit 212 through the first decoder 211, and the row multiplexing circuit 212 is connected to the control end of N rows of pixel units in the pixel array 3.

[0083] Among them, when the first control unit 11 receives the first control signal, it sends a first sub-signal to the first decoder 211 in at least one row selection unit 21; the first decoder 211 decodes the first sub-signal to obtain a second sub-signal, and sends the second sub-signal to the pixel block corresponding to the row multiplexing circuit 212 through the row multiplexing circuit 212; the second sub-signal is used to control the pixel block corresponding to the row multiplexing circuit 212 to turn on.

[0084] When the first control unit 11 receives the second control signal, it sends a third sub-signal to the first decoder 211 in at least one row selection unit 21; the first decoder 211 decodes the third sub-signal to obtain a fourth sub-signal, and sends the fourth sub-signal to the pixel block corresponding to the row multiplexing circuit 212 through the row multiplexing circuit 212; the fourth sub-signal is used to control the pixel block corresponding to the row multiplexing circuit 212 to be turned off.

[0085] In this embodiment, each row selection unit 21 may include a first decoder 211 and a row multiplexing circuit 212, and one row multiplexing circuit 212 may control N rows of pixel units. Optionally, the value of N may be greater than or equal to 2, so that at least two rows of pixel units may correspond to one first decoder 211, so as to reduce the number of the first decoders 211 provided.

[0086] Optionally, the first control signal and the second control signal received by the first control unit 11 can be used to determine the exposure time of each pixel block in the pixel array 3, so that the first control unit 11 can send a first sub-signal to each first decoder 211 based on the first control signal, and the first decoder 211 decodes to obtain a second sub-signal, and the second sub-signal is used to instruct the row multiplexing circuit 212 to select a specific row or rows of pixel units to turn on in the corresponding N rows of pixel units; accordingly, the first control unit 11 can send a third sub-signal to each first decoder 211 based on the second control signal, and the first decoder 211 decodes to obtain a fourth sub-signal, and the fourth sub-signal is used to instruct the row multiplexing circuit 212 to select a specific row or rows of pixel units to turn off in the corresponding N rows of pixel units. That is, the first sub-signal or the third sub-signal sent by the first control unit 11 is decoded by the first decoder 211 and then transmitted to the row multiplexing circuit 212, so that the row multiplexing circuit 212 selects a row of pixel units.

[0087] Optionally, the column selection unit 22 includes: a second decoder 221 and a column multiplexing circuit 222; the second control unit 12 is connected to the column multiplexing circuit 222 through the second decoder 221, and the column multiplexing circuit 222 is connected to the control end of the M columns of pixel blocks in the pixel array 3.

[0088] Wherein, when the second control unit 12 receives the first control signal, it sends a fifth sub-signal to the second decoder 221 in at least one column selection unit 22; the second decoder 221 decodes the fifth sub-signal to obtain a sixth sub-signal, and sends the sixth sub-signal to the pixel block corresponding to the column multiplexing circuit 222 through the column multiplexing circuit 222; the sixth sub-signal is used to control the pixel block corresponding to the column multiplexing circuit 222 to be turned on;

[0089] When the second control unit 12 receives the second control signal, it sends the seventh sub-signal to the second decoder 221 in at least one column selection unit 22; the second decoder 221 decodes the seventh sub-signal to obtain an eighth sub-signal, and sends the seventh sub-signal to the pixel block corresponding to the column multiplexing circuit 222 through the column multiplexing circuit 222; the seventh sub-signal is used to control the pixel block corresponding to the column multiplexing circuit 222 to be turned off.

[0090] In this embodiment, each column selection unit 22 may include a second decoder 221 and a column multiplexing circuit 222, and one column multiplexing circuit 222 may control M columns of pixel units. Optionally, the value of M may be greater than or equal to 2, so that at least two columns of pixel units may correspond to one second decoder 221, so as to reduce the number of second decoders 221 provided.

[0091] Optionally, the first control signal and the second control signal received by the second control unit 12 can be used to determine the exposure time of each pixel block in the pixel array 3, so that the second control unit 12 can send a fifth sub-signal to each second decoder 221 based on the first control signal, and the first decoder 211 decodes to obtain a sixth sub-signal, and the sixth sub-signal is used to indicate the column multiplexing circuit 222 to select which specific column or columns of pixel units to turn on in the corresponding M columns of pixel units; similarly, the second control unit 12 can send a seventh sub-signal to each second decoder 221 based on the second control signal, and the first decoder 211 decodes to obtain an eighth sub-signal, and the eighth sub-signal is used to indicate the column multiplexing circuit 222 to select which specific column or columns of pixel units to turn off in the corresponding M columns of pixel units. That is, the fifth sub-signal or the seventh sub-signal sent by the second control unit 12 is decoded by the second decoder 221 and then transmitted to the row-column multiplexing circuit 222, so that the column multiplexing circuit 222 selects the column pixel units.

[0092] It should be noted that, for a certain pixel unit, it is correspondingly connected to a row multiplexing circuit 212 and a column multiplexing circuit 222. When the row multiplexing circuit 212 outputs the second sub-signal to the pixel unit and the column multiplexing circuit 222 outputs the sixth sub-signal to the pixel unit, the pixel unit is turned on; correspondingly, when the row multiplexing circuit 212 outputs the fourth sub-signal to the pixel unit and the column multiplexing circuit 222 outputs the eighth sub-signal to the pixel unit, the pixel unit is turned off.

[0093] Alternatively, if Figure 4 As shown, the image sensor further includes: P analog-to-digital conversion units 4; P is the number of columns of pixel units in the pixel array 3. Each analog-to-digital conversion unit 4 is connected to a signal output terminal Vout of a column of pixel units in the pixel array 3 respectively.

[0094] When the pixel units in the corresponding column of the analog-to-digital conversion unit 4 are turned off, the analog-to-digital conversion unit 4 converts the analog signals output by the turned-off pixel units into digital signals.

[0095] Optionally, Figure 4 Pixel unit A1 represents the first pixel unit in row A, pixel unit A2 represents the second pixel unit in row A, pixel unit B1 represents the first pixel unit in row B, and pixel unit B2 represents the second pixel unit in row B. It should be noted that, Figure 4 Only two rows and two columns of pixel units are shown in the figure, and the number of rows and columns of pixel units in an actual pixel array is not limited thereto. Figure 4 Each pixel unit can be used Figure 2 The circuit structure shown will not be described in detail here.

[0096] In this embodiment, when the pixel unit in the pixel array 3 is turned on, the light signal is captured by its photodiode and converted into an electric charge, that is, the exposure begins. When the pixel unit is turned off, the analog electric signal of the signal output terminal Vout of the pixel unit is output to the analog-to-digital conversion unit 4, and the exposure ends. The analog-to-digital conversion unit 4 converts the received analog electric signal into a digital signal. For example, the converted digital signal can be transmitted to a subsequent image signal processor (Image Signal Processor, ISP) for processing, such as white balance, denoising, color restoration processing, etc., to finally generate an image of the photographed object. In the embodiment of the present application, the image of the photographed object is the finally generated image of the photographed object.

[0097] The working process of the image sensor in the embodiment of the present application is described below in combination with a specific application scenario:

[0098] Step 1: The automatic exposure system calculates the exposure time.

[0099] In an embodiment of the present application, before collecting image information based on an image sensor, an automatic exposure system (Auto Exposure, AE) can be used to calculate the exposure time required for different pixel blocks in the pixel array of the image sensor in advance according to the brightness distribution information of the shooting scene; for example, an initial image can be taken in advance of the object in the shooting scene, and the exposure time required for different pixel blocks in the pixel array of the image sensor can be calculated based on the brightness distribution information of the initial image.

[0100] For example: Figure 5 Taking the photo scene shown in the figure as an example, AE can be used to divide it into 336 pixel blocks in advance, such as Figure 6As shown, each rectangular area is a pixel block B. For example, the pixel blocks are numbered according to the number of rows, and each row is numbered in an increasing manner from left to right, that is, starting from the first row from top to bottom, the first row is pixel blocks 1 to 16 from left to right, and the second row is pixel blocks 17 to 32 from left to right, and so on, to obtain pixel blocks 1 to 336. Based on the control unit 1 and the selection unit 2 provided in the image sensor in the above embodiment, each pixel block can independently control the exposure time.

[0101] The smaller the pixel block, that is, the fewer pixel units each pixel block contains, the more refined its control area is. Figure 5 In the shooting scene shown, the pixel array is divided into 336 pixel blocks, which can be specifically set according to the actual shooting scene requirements, and the embodiments of the present application are not limited to this.

[0102] Optionally, the size and shape of the pixel block can be adjusted according to the pixel block divided by AE, such as Figure 7 As shown, each rectangular area is a pixel block B, and the sizes and shapes of the pixel blocks B can be the same or different. It should be noted that in the embodiment of the present application, each pixel block is set to be rectangular, and in actual applications, it can also be set according to the needs of the shooting scene. For example, the pixel block can also be set to an irregular shape based on the shooting object in the actual shooting scene, such as Figure 3D , Figure 3E As shown, or the size of each pixel block can be set to be the same or different, or the shape of each pixel block can be set to be the same or different, etc. The embodiments of the present application are not limited to this.

[0103] like Figure 6 As shown, consider that the shooting object corresponding to pixel blocks 1 to 144 is the sky, which belongs to the bright area and requires a shorter exposure time; and the shooting objects corresponding to pixel blocks 115 to 336 are mountains and grasslands, which belong to the dark area and require a longer exposure time; therefore, as an example, the exposure time of different areas can be set according to the following values:

[0104] Pixel blocks 1 to 144: require a shorter exposure time, such as setting the exposure time to 16ms;

[0105] Pixel blocks 115 to 336: require a longer exposure time, such as the exposure time is set to 32 ms.

[0106] Step 2: Reset all pixel units before taking pictures.

[0107] Pixel unit is adopted Figure 2Taking the circuit structure shown in FIG. 1 as an example, the first control unit 11 outputs the ninth sub-signal to each row multiplexing circuit 212 through the first decoder 211 according to the third control signal, and the second control unit 12 outputs the tenth sub-signal to each column multiplexing circuit 222 through the second decoder 221 according to the third control signal, and each row multiplexing circuit 212 and each column multiplexing circuit 222 control the reset switch RT and the transfer gate transistor GT in the corresponding pixel unit to be turned on at the same time. The charge in the photodiode PD in each pixel unit is cleared, and the floating diffusion area FD is reset to the voltage value of the power supply voltage VDD.

[0108] After the reset switch RT and the transfer gate transistor GT in each pixel unit are turned on for a period of time, the first control unit 11 outputs the eleventh sub-signal to each row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 outputs the twelfth sub-signal to each column multiplexing circuit 222 through the second decoder 221, and each row multiplexing circuit 212 and each column multiplexing circuit 222 control the reset switch RT and the transfer gate transistor GT in the corresponding pixel unit to be turned off at the same time, that is, the reset process of all pixel units is completed.

[0109] Step 3: Pixel blocks 1 to 336 begin to be exposed.

[0110] The first control unit 11 controls the row multiplexing circuit 212 to select rows through the first decoder 211, and the second control unit 12 controls the column multiplexing circuit 222 to select columns through the second decoder 221. The specific control method can be referred to the above embodiment, which will not be repeated here. The row multiplexing circuit 212 and the column multiplexing circuit 222 remain stable, and the photodiodes PD in all pixel units of the pixel blocks 1 to 336 begin to absorb photons, and charges gradually accumulate, that is, the pixel blocks 1 to 336 begin to be exposed.

[0111] Step 4: Pixel blocks 1 to 144 end exposure.

[0112] The exposure time of pixel blocks 1 to 144 is set to 16 ms; after 16 ms have passed since the exposure started in step 3, it is necessary to control pixel blocks 1 to 144 to end the exposure.

[0113] Specifically, the first control unit 11 controls the row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 controls the column multiplexing circuit 222 through the second decoder 221, and selects pixel blocks 1 to 144 to be turned off. The specific control method can be referred to the above embodiment, which will not be repeated here. For example, the reset switch RT and the transmission gate transistor GT in all pixel units in pixel blocks 1 to 144 are selected by the row multiplexing circuit 212 and the column multiplexing circuit 222 to perform a shutdown operation to ensure that they are no longer exposed. That is, all pixel units in pixel blocks 1 to 144 end exposure to avoid further charge accumulation and affect image quality.

[0114] Step 5: Pixel block 1-144 readout phase.

[0115] The first control unit 11 controls the row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 controls the column multiplexing circuit 222 through the second decoder 221. When the pixel blocks 1 to 144 are selected to be closed, all pixel units of the pixel blocks 1 to 144 end exposure and enter the pixel data readout phase.

[0116] Specifically, the first control unit 11 controls the row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 controls the column multiplexing circuit 222 through the second decoder 221, selects pixel blocks 1 to 144, controls the transfer gate transistors GT and the row selector switch RSL in all pixel units of pixel blocks 1 to 144 to turn on, so that the signals of pixel blocks 1 to 144 are output from the signal output terminal Vout, that is, the pixel data can be read.

[0117] Specifically, the charges in all pixel units of pixel blocks 1 to 144 are transferred to the floating diffusion region FD, and the analog electrical signals amplified by the source follower transistor SF are transmitted to the analog-to-digital conversion unit 4. The analog-to-digital conversion unit 4 converts the analog electrical signals into digital electrical signals so as to be transmitted to the subsequent image signal processing unit for processing, such as white balance, denoising, color restoration, etc., so as to generate an image.

[0118] Step 6: Pixel blocks 115 to 336 finish exposure and read out.

[0119] Since the exposure time of pixel blocks 115 to 336 is set to 32 ms, after pixel blocks 1 to 144 complete the exposure of 16 ms, pixel blocks 115 to 336 need to continue to be exposed until the total exposure time reaches 32 ms.

[0120] After pixel blocks 115 to 336 complete 32 ms of exposure, the first control unit 11 controls the row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 controls the column multiplexing circuit 222 through the second decoder 221 to select pixel blocks 115 to 336 to end exposure and perform readout. The specific operation is similar to closing pixel blocks 1 to 144 in steps 4 to 5 to end exposure and perform readout, and will not be repeated here.

[0121] Step 7: Generate image.

[0122] After pixel blocks 115 to 336 are exposed and read out, the pixel data read out from pixel blocks 1 to 336 are processed to obtain images of each photographed object that meet different exposure requirements, such as Figure 8 As shown, the corresponding details can be retained when the shooting object is the sky, and when the shooting object is the mountain and the grass, the embodiment of the present application has high shooting efficiency and imaging quality.

[0123] The image sensor in the above embodiment can enable different pixel blocks in the pixel array to independently control the exposure time according to the brightness distribution information of different photographed objects, while ensuring efficient signal readout and processing, thereby achieving precise imaging with a high dynamic range.

[0124] The shooting objects in the embodiments of the present application include but are not limited to: human objects, animal objects, landscape objects, object objects, etc., and the embodiments of the present application are not limited thereto. Figure 5 For example, the shooting objects may include the sky, mountains, grass and the like.

[0125] The embodiment of the present application provides a camera module, including the image sensor described above. The camera module can implement various embodiments of the image sensor described above and achieve the same technical effects, which will not be described here to avoid repetition.

[0126] The embodiment of the present application provides an electronic device, including the camera module as described above. The electronic device can implement various embodiments of the image sensor included in the camera module and can achieve the same technical effect, which will not be described here to avoid repetition.

[0127] The photographing method provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0128] like Fig. 9 As shown, an embodiment of the present invention provides a photographing method, which is performed by the above electronic device, and the method includes the following steps:

[0129] Step 91: Determine the exposure time of each pixel block in the pixel array according to the initial image of the photographed object.

[0130] Optionally, the initial image may be an image obtained by photographing the subject using the same exposure time for all pixel units in the image sensor; or, the initial image may be a preview screen containing the subject displayed on a photo preview interface, etc. The embodiments of the present application are not limited to this.

[0131] Optionally, each pixel block may include 4*K pixel units, where K is a positive integer. It should be noted that the number of pixel units included in each pixel block may be set as 4*K, and the number of K corresponding to each pixel block may be the same or different, that is, the number of pixel units included in different pixel blocks may be the same or different.

[0132] For example, the exposure time of different pixel blocks may be the same or different, for example, the pixel array may be divided into at least two pixel blocks, each pixel block corresponding to a different exposure time; or, the pixel array may be divided into at least two pixel areas, and these pixel areas may be divided into at least two groups according to the exposure time, for example, each group contains at least one pixel block, the exposure time of each pixel block in each group is the same, and the exposure time of pixel blocks in different groups is different, etc. The embodiments of the present application are not limited to this.

[0133] Step 92: Sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block.

[0134] Optionally, the first control signal is used to control the pixel block to be turned on, and the second control signal is used to control the pixel block to be turned off. In this way, the first control signal and the second control signal can control the corresponding pixel block to be exposed according to its corresponding exposure time.

[0135] Step 93: When all pixel units in each pixel block are turned off, read the pixel data of each pixel block.

[0136] Optionally, since the exposure time of different pixel blocks may be different, when each pixel block completes exposure, the pixel data of the pixel block is read.

[0137] Optionally, pixel data may refer to data output by a pixel unit. In the embodiment of the present application, the data output by a pixel unit may be an analog electrical signal.

[0138] Step 94: Generate an image of the photographed object according to the pixel data of the plurality of pixel blocks.

[0139] Optionally, when each pixel block ends exposure based on the second control signal and reads corresponding pixel data, the pixel data may be subsequently processed to generate an image of the photographed object, such as transmitting the analog electrical signal output from the pixel unit of the pixel block to the analog-to-digital conversion unit, and converting the analog electrical signal into a digital electrical signal through the analog-to-digital conversion unit. Further optionally, the digital electrical signal converted by the analog-to-digital conversion unit may also be processed by the ISP to generate an image.

[0140] In this embodiment, based on the initial image of the photographed object, the exposure time of at least two pixel blocks in the pixel array is determined, that is, each pixel block can independently set the exposure time, and according to the exposure time of each pixel block, the first control signal and the second control signal are sent to the control unit, so that each pixel block is exposed according to the corresponding exposure time. When each pixel block completes the exposure, the pixel data of the pixel block is read to generate the image of the photographed object. In this way, it can be ensured that different exposure requirements of different photographed objects can be met in one frame of photographed photos, avoiding the problems of low photography efficiency and relatively low imaging quality caused by the use of multi-frame image synthesis processing, thereby improving photography efficiency and imaging quality.

[0141] Optionally, determining the exposure time of each pixel block in the pixel array according to the initial image of the photographed object includes:

[0142] Dividing the pixel array into at least two pixel blocks according to brightness distribution information of the initial image;

[0143] Determine the exposure time of each pixel block according to the brightness value of the image area in the initial image corresponding to each pixel block;

[0144] The exposure time of each pixel block is negatively correlated with the brightness value of each pixel block.

[0145] Optionally, the brightness distribution information of the initial image refers to the distribution of brightness values ​​in all regions of the initial image. The brightness distribution information can be used to obtain important features of the initial image, such as overall brightness, brightness contrast, and color levels.

[0146] For example: when the pixel array is divided into at least two pixel blocks according to the brightness distribution information of the initial image, 4*K pixel units with the same brightness or a brightness difference less than a preset threshold in the corresponding initial image can be divided into the same pixel block based on AE; or, according to the brightness distribution information of the initial image, a preset number of pixel units satisfying 4*K can be divided into the same pixel block, etc. The embodiments of the present application are not limited to this.

[0147] For example, when determining the exposure time of each pixel block according to the brightness value of the image area in the initial image corresponding to each pixel block, a shorter exposure time may be set for the pixel block with a higher brightness value to avoid overexposure; a longer exposure time may be set for the pixel block with a lower brightness value to retain more image details, etc., that is, the exposure time of each pixel block is negatively correlated with the brightness value of each pixel block. In this way, different exposure times can be set for different pixel blocks to ensure that different exposure requirements for different objects in the picture can be met when taking a picture, thereby improving the shooting efficiency and image quality.

[0148] Optionally, the size and shape of the pixel block can be adjusted according to the pixel block divided by AE. The embodiment of the present application does not limit each pixel block to be rectangular, and can be set according to the actual shooting scene requirements. For example, the pixel block can also be set to an irregular shape based on the shooting object in the actual shooting scene, such as Figure 3D , Figure 3E As shown, or the size of each pixel block can be set to be the same or different, or the shape of each pixel block can be set to be the same or different, etc. The embodiments of the present application are not limited to this.

[0149] Optionally, the sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block includes:

[0150] Sending a first control signal to the control unit, the first control signal being used to control each pixel block to turn on; wherein each pixel block starts to be exposed from the moment it is turned on;

[0151] According to the exposure time of each pixel block, a second control signal is sent to the control unit, and the second control signal is used to control each pixel block to be turned off when the exposure of each pixel block reaches the exposure time.

[0152] For example, Figure 1 Taking the image sensor shown as an example, the corresponding pixel units can be uniformly turned on according to the pixel blocks corresponding to each selection unit. When a pixel block or some pixel blocks reach their corresponding exposure time, the corresponding pixel blocks are controlled to be turned off, that is, the exposure of the pixel blocks ends.

[0153] Taking the case where all pixel units in a pixel block are arranged in a rectangular shape as an example, when the control unit controls the corresponding pixel block to turn on according to the first control signal, the first row of pixel units in the pixel block is turned on, that is, the pixel block starts to be exposed. After the exposure of the first row of pixel units is completed, the pixel units in the row are turned off, and the second row of pixel units is turned on and starts to be exposed, and so on, until the cumulative exposure time of each row of pixel units reaches the exposure time set for the pixel block, that is, the exposure of the pixel block ends. It should be supplemented that if the cumulative exposure time of all rows of pixel units in the pixel block does not reach the exposure time set for the pixel block after all the rows of pixel units have completed one exposure, the first row of pixel units is started again for exposure, and so on, until the cumulative exposure time of each row of pixel units reaches the exposure time set for the pixel block, then the exposure of the pixel block ends.

[0154] In this embodiment, by uniformly starting exposure according to the pixel blocks corresponding to the respective selection units, and individually controlling the end of exposure of each pixel block according to its exposure time, calculations can be reduced and the implementation of control logic can be facilitated.

[0155] Optionally, the sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block includes:

[0156] According to the exposure time of each pixel block, a first control signal is sent to the control unit, wherein the first control signal is used to control the pixel blocks in the pixel array to be turned on in the order of exposure time from long to short; wherein the turn-on time interval of two pixel blocks with adjacent turn-on times is equal to the difference between the exposure times of the two pixel blocks;

[0157] A second control signal is sent to the control unit, where the second control signal is used to control each pixel block to be turned off.

[0158] For example, Figure 1 Taking the image sensor shown as an example, the corresponding pixel blocks can be turned on in the order of exposure time from long to short, for example, the pixel block with the longest exposure time is turned on first, and then the pixel block with the second longest exposure time is turned on according to the time difference between the second longest exposure time and the longest exposure time, and so on, until all pixel blocks start to be exposed, and then all pixel blocks can be controlled to be turned off uniformly through various selection units. Specifically, the exposure process of each pixel block can refer to the method of turning on exposure of pixel units row by row in the above embodiment, which will not be repeated here.

[0159] In this embodiment, by turning on the corresponding pixel blocks in the order of exposure time from long to short, all pixel blocks can be turned off uniformly to ensure the continuity of reading pixel information.

[0160] Optionally, before sending the first control signal and the second control signal to the control unit according to the exposure time of each pixel block, the method further includes:

[0161] A third control signal is sent to the control unit, wherein the third control signal is used to control the resetting of each pixel block.

[0162] The pixel block reset means that the charge of the photoelectric conversion element of each pixel unit in the pixel block is cleared.

[0163] In this embodiment, before controlling each pixel block to be exposed, all pixel units in the pixel block are reset to clear the charges stored in the pixel units, thereby avoiding errors in the output pixel data due to the residual charges in the pixel units, avoiding affecting the imaging quality, and ensuring the imaging effect.

[0164] The following describes the photographing method of the embodiment of the present application in combination with specific application scenarios:

[0165] Step 1: When the electronic device starts a camera application, a preview screen containing at least one shooting object in the shooting scene can be displayed on the shooting preview interface of the camera application. Fig.10 As shown, the camera is automatically exposed, that is, each object in the shooting scene is exposed according to the same exposure time to obtain an initial image.

[0166] Step 2: According to the initial image of the photographed object, the exposure time of each pixel block in the pixel array is determined.

[0167] Specifically, based on the initial image obtained in step 1 above, the brightness value of the brightest area and the brightness value of the darkest area in the initial image can be calculated based on AE, and the initial image can be divided into regions according to the calculated brightness values. Fig.11 As shown, the initial image is divided into two areas: the upper half is the sky area 111, and the exposure time is set to 10ms; the lower half is the ground and building area 112, and the exposure time is set to 30ms. Accordingly, based on the exposure time of the sky area 111 and the ground and building area 112, the exposure time of each pixel block in the image sensor corresponding to the sky area 111 and the ground and building area 112 is determined respectively, for example, the exposure time of at least one pixel block in the image sensor corresponding to the sky area 111 is set to 10ms, and the exposure time of at least one pixel block in the image sensor corresponding to the ground and building area 112 is set to 30ms.

[0168] Step 3: Sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block.

[0169] For example, the processor in the electronic device determines the exposure time of each pixel block based on step 2, and can send a first control signal to the image sensor. For example, when the control unit in the image sensor receives the first control signal, it controls each pixel block to perform regional exposure according to the corresponding exposure time. Fig.12 As shown, taking the initial image with a resolution of 4000*3000 as an example, the initial image is divided into two regions of interest (ROI) from top to bottom, including region of interest 1, i.e., sky region 111, and region of interest 2, i.e., ground and building region 112. Region of interest 1 corresponds to an exposure time of 10m, and region of interest 2 corresponds to an exposure time of 30ms. Each region of interest corresponds to a 4000*1500 pixel block, or each region of interest corresponds to at least two pixel blocks, wherein at least two pixel blocks corresponding to each region of interest constitute a 4000*1500 pixel array. This division method ensures that each region of interest can be processed independently based on different exposure time requirements, thereby optimizing the exposure effect of the overall picture.

[0170] Step 4: The control unit starts exposure according to the first control signal.

[0171] like Fig.12 As shown, when the first control unit 11 and the second control unit 12 receive the first control signal, the first control unit 11 controls each row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 controls each column multiplexing circuit 222 through the second decoder 221, and selects the corresponding pixel blocks to turn on. For example, in this embodiment, both pixel block 31 and pixel block 32 can be controlled to turn on, that is, exposure starts.

[0172] When both pixel block 31 and pixel block 32 start to be exposed, the photodiodes in the pixel units of pixel block 31 and pixel block 32 start to absorb photons and gradually accumulate charges. Specifically, the exposure method of each pixel unit in each pixel block can refer to the above-mentioned embodiment of the image sensor, which will not be repeated here.

[0173] Step 5: The control unit controls the pixel block 31 to end exposure according to the second control signal and reads out data.

[0174] Pixel block 31 corresponds to region of interest 1, and its exposure time is set to 10ms, and pixel block 32 corresponds to region of interest 1, and its exposure time is set to 30ms. When pixel block 31 and pixel block 32 start to be exposed and after the exposure time of 10ms, the control unit can control pixel block 31 to be turned off according to the second control signal, that is, pixel block 31 ends exposure in advance and reads out data, and pixel block 32 continues to be exposed.

[0175] Specifically, Fig.12 As shown, when the first control unit 11 and the second control unit 12 receive the second control signal, the first control unit 11 controls the corresponding row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 controls the corresponding column multiplexing circuit 222 through the second decoder 221, selects the pixel block 31 and controls the pixel block 31 to be turned off, that is, the pixel block 31 can end the exposure earlier than the pixel block 32, to prevent the further accumulation of charges of each pixel unit in the pixel block 31. The pixel units of row1 to row1500, a total of 1500 rows and col1 to col4000, a total of 4000 columns, corresponding to the pixel block 31, are read, that is, all the pixel units from the 1st row to the 1500th row are read.

[0176] Optionally, after reading the pixel data of the pixel block 31, the first control unit 11 can control the corresponding row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 can control the corresponding column multiplexing circuit 222 through the second decoder 221 to select the pixel block 31 and control the pixel block 31 to reset, ensuring that no pixel data is read out of the pixel block 31.

[0177] Step 6: The control unit controls the pixel block 32 to end exposure according to the second control signal and reads out data.

[0178] After pixel block 32 and pixel block 31 complete the 10 ms exposure, pixel block 32 continues to expose until the total exposure time reaches 30 ms, and the control unit can control pixel block 32 to turn off according to the second control signal, that is, pixel block 32 also ends exposure and reads out data.

[0179] Specifically, Fig.12 As shown, when the first control unit 11 and the second control unit 12 receive the second control signal, the first control unit 11 controls the corresponding row multiplexing circuit 212 through the first decoder 211, and the second control unit 12 controls the corresponding column multiplexing circuit 222 through the second decoder 221, selects the pixel block 32 and controls the pixel block 32 to be turned off, that is, the pixel block 32 ends exposure, and prevents further accumulation of charges of each pixel unit in the pixel block 32. The pixel units of row1501 to row3000, a total of 1500 rows, and col1 to col4000, a total of 4000 columns, corresponding to the pixel block 32, are read, that is, all the pixel units from the pixel unit in row 1501 to row 3000 are read.

[0180] Based on the pixel data read out from the pixel block 31 and the pixel data read out from the pixel block 32, an image is generated, such as Fig.11As shown, pixel block 31 corresponds to region of interest 1 in the image, namely, sky region 111, and pixel block 32 corresponds to region of interest 2 in the image, namely, ground and building region 112. The image obtained by shooting with the traditional scheme, that is, the image obtained by shooting different objects in the entire shooting scene with a uniform exposure time, as shown in FIG. Fig.13 As shown. Images taken with the traditional solution may cause the clouds and the sun in the sky to be overexposed, and their details are lost. However, with the solution of the embodiment of the present application, the photographed objects in the area of ​​interest 1, i.e., the sky area 111, and the area of ​​interest 2, i.e., the ground and building area 112, all retain photographic details. Obviously, the solution of the embodiment of the present application can improve the imaging quality and has a higher photographing efficiency than the method of multi-frame image synthesis.

[0181] In the embodiment of the present application, the image sensor is divided into a plurality of independent pixel blocks, and each pixel block can independently control the exposure time. For example, for a shooting object with low brightness, a longer exposure time can be used, while for a shooting object with high brightness, a short exposure time can be used to avoid overexposure and retain the details of the dark area, thereby solving the problem of poor imaging caused by the decline of optical quality and electromagnetic interference. And based on the real-time analysis of the brightness distribution in the shooting scene, the exposure time of each pixel block and the size and shape of the pixel block can be dynamically adjusted, for example, the exposure time is extended for a shooting object with low brightness, and the exposure time is shortened for a shooting object with high brightness to ensure the overall imaging quality. In this way, by adaptively controlling the exposure time of different pixel blocks in a single frame, the image fault problem caused by the multi-frame image synthesis processing is reduced, and the continuity and integrity of the image are ensured, and the imaging efficiency can also be improved. In addition, since the embodiment of the present application adopts a single frame to achieve high dynamic range imaging, its motion blur problem is greatly reduced, and no additional deblurring algorithm is required, and by extending the exposure time in the low-light area, the detail capture capability of the shooting object with low brightness is increased, and the signal-to-noise ratio of the overall image is improved.

[0182] The photographing method provided in the embodiment of the present application can be executed by a photographing device. In the embodiment of the present application, the photographing device provided in the embodiment of the present application is described by taking the photographing method executed by the photographing device as an example.

[0183] like Fig.14 As shown, the embodiment of the present application provides a photographing device 1400, which is executed by the above electronic device, and the device includes:

[0184] A determination module 1410, configured to determine an exposure time of each pixel block in the pixel array according to an initial image of the photographed object;

[0185] A first sending module 1420, configured to send a first control signal and a second control signal to the control unit according to the exposure time of each pixel block;

[0186] A reading module 1430, configured to read pixel data of each pixel block when all pixel units in each pixel block are turned off;

[0187] The processing module 1440 is used to generate the image of the photographed object according to the pixel data of each pixel block.

[0188] Optionally, the determining module 1410 includes:

[0189] A division unit, configured to divide the pixel array into at least two pixel blocks according to brightness distribution information of the initial image;

[0190] The determination unit is used to determine the exposure time of each pixel block according to the brightness value of the image area in the initial image corresponding to each pixel block.

[0191] Optionally, the first sending module 1420 includes:

[0192] A first sending unit, used to send a first control signal to the control unit, wherein the first control signal is used to control each pixel block to turn on; wherein each pixel block starts to be exposed from the moment of turning on;

[0193] The second sending unit is used to send a second control signal to the control unit according to the exposure time of each pixel block, and the second control signal is used to control each pixel block to be turned off when the exposure of each pixel block reaches the exposure time.

[0194] Optionally, the first sending module 1420 includes:

[0195] A third sending unit is used to send a first control signal to the control unit according to the exposure time of each pixel block, wherein the first control signal is used to control the pixel blocks in the pixel array to be turned on in the order of exposure time from long to short; wherein the turn-on time interval of two pixel blocks with adjacent turn-on times is equal to the difference between the exposure times of the two pixel blocks;

[0196] The fourth sending unit is used to send a second control signal to the control unit, where the second control signal is used to control each pixel block to be turned off.

[0197] Optionally, the photographing device 1400 further includes:

[0198] A second sending module, used for sending a third control signal to the control unit;

[0199] The third control signal is used to control each pixel block.

[0200] The photographing device in the embodiment of the present invention determines the exposure time of at least two pixel blocks in the pixel array based on the initial image of the photographed object, that is, each pixel block can independently set the exposure time, and sends a first control signal and a second control signal to the control unit according to the exposure time of each pixel block, so that each pixel block is exposed according to its corresponding exposure time. When each pixel block completes the exposure, the pixel data of the pixel block is read to generate the image of the photographed object. In this way, it can be ensured that different exposure requirements of different photographed objects can be met in one frame of photographed photos, avoiding the problem of low photographing efficiency and imaging quality caused by the use of multi-frame image synthesis processing, thereby improving photographing efficiency and imaging quality.

[0201] The audio data processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0202] The audio data processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0203] The audio data processing device provided in the embodiment of the present application can achieve Figures 1 to 13 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0204] Alternatively, if Fig.15As shown, an embodiment of the present application also provides an electronic device 1500, including a processor 1501 and a memory 1502, wherein the memory 1502 stores a program or instruction that can be executed on the processor 1501, and when the program or instruction is executed by the processor 1501, the various steps of the above-mentioned audio data processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0205] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0206] Fig.16 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0207] The electronic device 1600 includes but is not limited to: a radio frequency unit 1601, a network module 1602, an audio output unit 1603, an input unit 1604, a sensor 1605, a display unit 1606, a user input unit 1607, an interface unit 1608, a memory 1609, and a processor 1610 and other components.

[0208] Those skilled in the art will appreciate that the electronic device 1600 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1610 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.16 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0209] The processor 1610 is used for:

[0210] Determining the exposure time of each pixel block in the pixel array according to an initial image of the photographed object;

[0211] sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block;

[0212] When all pixel units in each pixel block are turned off, the pixel data of each pixel block is read;

[0213] An image of the photographed object is generated according to the pixel data of each pixel block.

[0214] Optionally, the processor 1610 is further configured to:

[0215] Dividing the pixel array into at least two pixel blocks according to the brightness distribution information of the initial image;

[0216] Determining the exposure time of each pixel block according to the brightness value of the image area in the initial image corresponding to each pixel block;

[0217] The exposure time of each pixel block is negatively correlated with the brightness value of each pixel block.

[0218] Optionally, the processor 1610 is further configured to:

[0219] Sending a first control signal to the control unit, wherein the first control signal is used to control each pixel block to turn on; wherein each pixel block starts to be exposed from the moment it is turned on;

[0220] A second control signal is sent to the control unit according to the exposure time of each pixel block, wherein the second control signal is used to control each pixel block to be turned off when the exposure of each pixel block reaches the exposure time.

[0221] Optionally, the processor 1610 is further configured to:

[0222] According to the exposure time of each pixel block, a first control signal is sent to the control unit, wherein the first control signal is used to control the pixel blocks in the pixel array to be turned on in the order of exposure time from long to short; wherein the turn-on time interval of two pixel blocks with adjacent turn-on times is equal to the difference between the exposure times of the two pixel blocks;

[0223] A second control signal is sent to the control unit, where the second control signal is used to control each pixel block to be turned off.

[0224] Optionally, the processor 1610 is further configured to:

[0225] sending a third control signal to the control unit;

[0226] The third control signal is used to control the resetting of each pixel block.

[0227] The electronic device in the embodiment of the present invention determines the exposure time of at least two pixel blocks in the pixel array based on the initial image of the photographed object, that is, each pixel block can independently set the exposure time, and sends a first control signal and a second control signal to the control unit according to the exposure time of each pixel block, so that each pixel block is exposed according to its corresponding exposure time. When each pixel block completes the exposure, the pixel data of the pixel block is read to generate the image of the photographed object. In this way, it can be ensured that different exposure requirements of different photographed objects can be met in one frame of photographed photos, avoiding the problem of low photographing efficiency and imaging quality caused by the use of multi-frame image synthesis processing, thereby improving photographing efficiency and imaging quality.

[0228] It should be understood that in the embodiment of the present application, the input unit 1604 may include a graphics processor (Graphics Processing Unit, GPU) 16041 and a microphone 16042, and the graphics processor 16041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1606 may include a display panel 16061, and the display panel 16061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1607 includes a touch panel 16071 and at least one of other input devices 16072. The touch panel 16071 is also called a touch screen. The touch panel 16071 may include two parts: a touch detection device and a touch controller. Other input devices 16072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0229] The memory 1609 can be used to store software programs and various data. The memory 1609 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1609 may include a volatile memory or a non-volatile memory, or the memory 1609 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1609 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0230] The processor 1610 may include one or more processing units; optionally, the processor 1610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1610.

[0231] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned photographing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0232] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0233] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned photographing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0234] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0235] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned photographing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0236] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0238] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image sensor, characterized in that: include: at least one control unit, at least two selection units and a pixel array; The pixel array includes at least two pixel blocks; Each selection unit is connected to a control unit, and each selection unit is respectively connected to a control end of at least one row of pixel units or at least one column of pixel units in the pixel array; wherein different selection units are connected to pixel units in different rows or different columns of pixel units in the pixel array; Wherein, when the control unit receives the first control signal, the pixel block corresponding to the at least one selection unit in the pixel array is controlled to be turned on through at least one selection unit; When the control unit receives the second control signal, the pixel block corresponding to the at least one selection unit in the pixel array is controlled to be turned off through the at least one selection unit; Each pixel block includes 4*K pixel units, where K is a positive integer.

2. The image sensor according to claim 1, characterized in that When the control unit receives a third control signal, the pixel block corresponding to the at least one selection unit in the pixel array is controlled to be reset through the at least one selection unit; The pixel block resetting means that the charge of the photoelectric conversion element of each pixel unit in the pixel block is cleared.

3. The image sensor according to claim 1, characterized in that The at least one control unit includes: a first control unit and a second control unit; the at least two selection units include: at least two row selection units and at least two column selection units; The first control unit is connected to each row selection unit respectively, each row selection unit is connected to the control end of N rows of pixel units in the pixel array respectively, and different row selection units are connected to different rows of pixel units in the pixel array; wherein N is a positive integer; The second control unit is connected to each column selection unit respectively, each column selection unit is connected to the control end of M columns of pixel units in the pixel array respectively, and different column selection units are connected to pixel units in different columns in the pixel array; wherein M is a positive integer; Wherein, when the first control unit and the second control unit receive the first control signal, the first control unit and the second control unit control the pixel blocks corresponding to the at least one row selection unit and the at least one column selection unit in the pixel array to turn on through the at least one row selection unit and the at least one column selection unit; When the first control unit and the second control unit receive a second control signal, the first control unit and the second control unit control the pixel blocks corresponding to the at least one row selection unit and the at least one column selection unit in the pixel array to turn off through the at least one row selection unit and the at least one column selection unit.

4. The image sensor according to claim 3, characterized in that: The number of the row selection units is greater than or equal to 12, and the number of the column selection units is greater than or equal to 9.

5. The image sensor according to claim 3, characterized in that: N is less than or equal to 341, and M is less than or equal to 341.

6. The image sensor according to claim 3, characterized in that: The row selection unit includes: a first decoder and a row multiplexing circuit; The first control unit is connected to the row multiplexing circuit through the first decoder, and the row multiplexing circuit is connected to the control terminals of N rows of pixel units in the pixel array; Wherein, when the first control unit receives the first control signal, it sends a first sub-signal to a first decoder in at least one row selection unit; the first decoder decodes the first sub-signal to obtain a second sub-signal, and sends the second sub-signal to a pixel block corresponding to the row multiplexing circuit through the row multiplexing circuit; the second sub-signal is used to control the pixel block corresponding to the row multiplexing circuit to turn on; When the first control unit receives the second control signal, it sends a third sub-signal to the first decoder in at least one row selection unit; the first decoder decodes the third sub-signal to obtain a fourth sub-signal, and sends the fourth sub-signal to the pixel block corresponding to the row multiplexing circuit through the row multiplexing circuit; the fourth sub-signal is used to control the pixel block corresponding to the row multiplexing circuit to be turned off.

7. The image sensor according to claim 3, characterized in that: The column selection unit includes: a second decoder and a column multiplexing circuit; The second control unit is connected to the column multiplexing circuit through the second decoder, and the column multiplexing circuit is connected to the control terminals of M columns of pixel units in the pixel array; Wherein, when the second control unit receives the first control signal, it sends a fifth sub-signal to a second decoder in at least one column selection unit; the second decoder decodes the fifth sub-signal to obtain a sixth sub-signal, and sends the sixth sub-signal to a pixel block corresponding to the column multiplexing circuit through the column multiplexing circuit; the sixth sub-signal is used to control the pixel block corresponding to the column multiplexing circuit to turn on; When the second control unit receives the second control signal, it sends a seventh sub-signal to a second decoder in at least one column selection unit; the second decoder decodes the seventh sub-signal to obtain an eighth sub-signal, and sends the seventh sub-signal to a pixel block corresponding to the column multiplexing circuit through the column multiplexing circuit; the seventh sub-signal is used to control the pixel block corresponding to the column multiplexing circuit to be turned off.

8. The image sensor according to claim 1, characterized in that: Also includes: P analog-to-digital conversion units, where P is the number of columns of pixel units in the pixel array; Each analog-to-digital conversion unit is respectively connected to a signal output end of a column of pixel units in the pixel array; Wherein, when the pixel units in the column corresponding to the analog-to-digital conversion unit are turned off, the analog-to-digital conversion unit converts the analog signals output by the turned-off pixel units into digital signals.

9. A camera module, characterized in that: Comprising the image sensor as claimed in any one of claims 1 to 8.

10. An electronic device, characterized in that: Comprising the camera module as claimed in claim 9.

11. A photographing method, executed by the electronic device according to claim 10, characterized in that: The method comprises: Determining the exposure time of each pixel block in the pixel array according to an initial image of the photographed object; sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block; When all pixel units in each pixel block are turned off, the pixel data of each pixel block is read; An image of the photographed object is generated according to the pixel data of each pixel block.

12. The photographing method according to claim 11, characterized in that: The step of determining the exposure time of each pixel block in the pixel array according to the initial image of the photographed object comprises: Dividing the pixel array into at least two pixel blocks according to the brightness distribution information of the initial image; Determining the exposure time of each pixel block according to the brightness value of the image area in the initial image corresponding to each pixel block; The exposure time of each pixel block is negatively correlated with the brightness value of each pixel block.

13. The photographing method according to claim 11, characterized in that: The step of sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block comprises: Sending a first control signal to the control unit, wherein the first control signal is used to control each pixel block to turn on; wherein each pixel block starts to be exposed from the moment it is turned on; A second control signal is sent to the control unit according to the exposure time of each pixel block, wherein the second control signal is used to control each pixel block to be turned off when the exposure time of each pixel block reaches the exposure time.

14. The photographing method according to claim 11, characterized in that: The step of sending a first control signal and a second control signal to the control unit according to the exposure time of each pixel block comprises: According to the exposure time of each pixel block, a first control signal is sent to the control unit, wherein the first control signal is used to control the pixel blocks in the pixel array to be turned on in the order of exposure time from long to short; wherein the turn-on time interval of two pixel blocks with adjacent turn-on times is equal to the difference between the exposure times of the two pixel blocks; A second control signal is sent to the control unit, where the second control signal is used to control each pixel block to be turned off.

15. The photographing method according to claim 11, characterized in that: Before sending the first control signal and the second control signal to the control unit according to the exposure time of each pixel block, the method further includes: sending a third control signal to the control unit; The third control signal is used to control the resetting of each pixel block.

Images