Image Sensor, Image Generation Method, and Storage Medium

Abstract

The present disclosure relates to an image sensor, an image generation method, and a storage medium. Among them, the image sensor includes a pixel array, an analog-to-digital conversion circuit, and an exposure timing control circuit. In the exposure timing control circuit, pixel values corresponding to each pixel in the currently output image frame of the pixel array are obtained, and based on the pixel values corresponding to each pixel in the currently output image frame, the exposure time for the pixel array to output the next image frame is determined. In the present disclosure, the pixel array outputs the next image frame based on the exposure time of the next image frame, thereby realizing the adjustment of the exposure time of the output image frame. Further, the image frames output based on the respective exposure times achieve the purpose of expanding the dynamic range.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to an image sensor, an image generation method, and a storage medium. Background Art

[0002] The dynamic range is an important indicator of the imaging quality of an image sensor, which refers to the range of illumination intensity distribution from the darkest shadow part to the brightest highlight part in an image. The larger the dynamic range, the wider the range of scene information that can be detected, making the image details richer. In related technologies, the dynamic range is adjusted by adjusting the exposure time. For example, through multiple sampling techniques, the same scene is sampled multiple times at different exposure times, and then the images obtained from multiple samplings are fused into an image with a larger dynamic range. However, since the images involved in the fusion are not generated by simultaneous exposure in the multiple sampling technique, there are motion artifacts in the generated image with a larger dynamic range. In related technologies, the dynamic range is also extended by Dual Conversion Gain (DCG) pixels, that is, when the image sensor operates in the High Dynamic Range (HDR) mode, only one exposure is performed but read out twice. Once, a high conversion gain (HCG) is used to capture the dark part information, and once, a low conversion gain (LCG) is used to capture the bright part information, and the dark part information and the bright part information are combined to output an image with a larger dynamic range. However, the performance of DCG pixels depends on the well capacity of the pixels in the image sensor, which limits the extension of the dynamic range. Therefore, the problem of how to adjust the exposure time to achieve the purpose of dynamic range extension remains to be solved. Summary of the Invention

[0003] To overcome the problems existing in related technologies, the present disclosure provides an image sensor, an image generation method, and a storage medium.

[0004] According to the first aspect of the embodiments of the present disclosure, an image sensor is provided, and the image sensor includes:

[0005] A pixel array for outputting image frames frame by frame; an analog-to-digital conversion circuit for performing analog-to-digital conversion on the pixel voltages generated by each pixel in the currently output image frame of the pixel array and then outputting pixel values; an exposure timing control circuit for obtaining the pixel values corresponding to each pixel in the currently output image frame of the pixel array, and determining the exposure time for the pixel array to output the next image frame according to the pixel values corresponding to each pixel in the currently output image frame.

[0006] In an implementation manner, the exposure timing control circuit includes:

[0007] A pixel scanning circuit scans out the maximum non-saturated pixel value from the pixel values corresponding to each pixel in the current output image frame; an exposure time determination circuit determines the exposure time for the pixel array to output the next image frame based on the maximum non-saturated pixel value; an exposure time feedback circuit is configured to feedback the exposure time of the next image frame to the pixel array.

[0008] In one embodiment, the exposure timing control circuit includes:

[0009] A pixel voltage detection circuit detects the pixel voltages generated by each pixel in the current output image frame of the pixel array. If there is a pixel whose pixel voltage is greater than a preset voltage threshold and the number of pixels whose pixel voltages are greater than the preset voltage threshold is greater than a pixel number threshold, it generates an identification signal for characterizing the adjustment of the exposure time of the next image frame; an exposure time feedback circuit is configured to, when the pixel voltage detection circuit generates the identification signal, feedback the exposure time of the next image frame to the pixel array.

[0010] In one embodiment, the pixel voltage detection circuit includes:

[0011] A threshold register is configured to store a preset voltage threshold and a pixel number threshold; an analog comparator is configured to compare the pixel voltages generated by each pixel in the current output image frame of the pixel array with the preset voltage threshold and count the number of pixels whose pixel voltages are greater than the preset voltage threshold. When the number of pixels whose pixel voltages are greater than the preset voltage threshold is greater than the pixel number threshold, it generates an identification signal for characterizing the adjustment of the exposure time of the next image frame.

[0012] In one embodiment, the image sensor includes: an image fusion circuit that fuses the image frames output frame by frame by the pixel array to obtain a fused preview original image frame.

[0013] According to a second aspect of the embodiments of the present disclosure, there is provided an image generation method applied to an image sensor. The image generation method includes:

[0014] Obtain the pixel values corresponding to each pixel in the current output image frame of the pixel array of the image sensor; determine the exposure time for the pixel array to output the next image frame based on the pixel values corresponding to each pixel in the current output image frame; control the pixel array to generate the next image frame based on the exposure time of the next image frame.

[0015] In one embodiment, determining the exposure time for the pixel array to output the next image frame based on the pixel values corresponding to each pixel in the current output image frame includes:

[0016] Scan out the maximum non-saturated pixel value from the pixel values corresponding to each pixel in the current output image frame; based on the maximum non-saturated pixel value, determine the exposure time for the pixel array to output the next image frame.

[0017] In one implementation, determining the exposure time for the pixel array to output the next image frame based on the maximum non-saturated pixel value includes:

[0018] Determine the minimum exposure time in the image frames already output by the pixel array; based on the ratio of the most significant bit of the maximum non-saturated pixel value to the minimum exposure time, determine an adjustment multiple value for adjusting the exposure time of the next image frame; based on the adjustment multiple value for adjusting the exposure time of the next image frame, determine the exposure time for the pixel array to output the next image frame.

[0019] In one implementation, before controlling the pixel array to generate the next image frame based on the exposure time of the next image frame, the image generation method includes:

[0020] Detect the pixel voltages generated by each pixel in the current output image frame of the pixel array. If there are pixels whose pixel voltages are greater than a preset voltage threshold and the number of pixels whose pixel voltages are greater than the preset voltage threshold is greater than a pixel number threshold, generate an identification signal for characterizing the adjustment of the exposure time of the next image frame; in the case of generating the identification signal, feedback the exposure time of the next image to the pixel array.

[0021] In one implementation, the image generation method includes: fusing the image frames output frame by frame by the pixel array to obtain a fused preview original image frame.

[0022] According to a third aspect of the embodiments of the present disclosure, there is provided an image sensor, including:

[0023] A processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute the image generation method described in any one of the implementations in the first aspect or the second aspect.

[0024] According to a fourth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enabling the mobile terminal to execute the image generation method described in any one of the implementations in the first aspect or the second aspect.

[0025] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: An image sensor including a pixel array, an analog-to-digital conversion circuit, and an exposure timing control circuit is provided. In the exposure timing control circuit, pixel values corresponding to each pixel in the currently output image frame of the pixel array are obtained, and based on the pixel values corresponding to each pixel in the currently output image frame, the exposure time of the pixel array for outputting the next image frame is determined. The pixel array outputs the next image frame based on the exposure time of the next image frame, thereby realizing the adjustment of the exposure time of the output image frame. Further, the image frames output based on each exposure time achieve the purpose of expanding the dynamic range.

[0026] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0028] Figure 1 is a schematic diagram of an image sensor shown according to an exemplary embodiment.

[0029] Figure 2 is a schematic diagram of an analog-to-digital conversion circuit shown according to an exemplary embodiment.

[0030] Figure 3 is a schematic diagram of an exposure timing control circuit shown according to an exemplary embodiment.

[0031] Figure 4 is a schematic diagram of a pixel scanning circuit shown according to an exemplary embodiment.

[0032] Figure 5 is a schematic diagram of an exposure time determination circuit shown according to an exemplary embodiment.

[0033] Figure 6 is a schematic diagram of an exposure timing control circuit shown according to an exemplary embodiment.

[0034] Figure 7 is a schematic diagram of a pixel voltage detection circuit shown according to an exemplary embodiment.

[0035] Figure 8 is a schematic diagram of an image sensor shown according to an exemplary embodiment.

[0036] Figure 9 is an overall schematic diagram of an image sensor shown according to an exemplary embodiment.

[0037] Figure 10It is a flowchart of an image generation method shown according to an exemplary embodiment.

[0038] Figure 11 It is a block diagram of a device provided with an image sensor shown according to an exemplary embodiment. Detailed implementation manners

[0039] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0040] Figure 1 It is a schematic diagram of an image sensor shown according to an exemplary embodiment. As Figure 1 shown, the image sensor 100 includes: a pixel array 101, an analog-to-digital conversion circuit 102, and an exposure timing control circuit 103.

[0041] The image sensor provided by the embodiments of the present disclosure can add an exposure timing control circuit 103 on the basis of a Complementary Metal Oxide Semiconductor (CMOS) transistor, and on the premise of adopting a drum-type exposure and a corresponding output mode, according to the maximum non-saturated pixel value within the current frame output by the pixel array 101 in the image sensor, dynamically adjust the exposure time of the pixel array 101 for outputting the next frame of image, avoiding the RAM storage space equivalent to the pixel array required for realizing pixel adaptive exposure and the response curve compression caused by multiple exposures. In addition, adding an exposure timing control circuit 103 on the basis of CMOS greatly reduces the control complexity and circuit area, and solves the problem that it is difficult to expand the dynamic range due to the well capacity limitation of the pixels when using DCG pixels.

[0042] In the embodiments of the present disclosure, the pixel array 101 is used to output image frames frame by frame. When an image sensor is used to capture an image, when the pixel array 101 receives the optical signal reflected by the object to be photographed, it outputs image frames frame by frame according to the exposure time. The pixel array 101 collects the optical signal, converts the collected optical signal into an electrical signal, and classifies the pixel array 101 according to the number of transistors. The pixel array 101 mainly includes three-transistor active pixel (3T-APS), four-transistor active pixel (4T-APS), five-transistor active pixel (5T-APS), etc. Those skilled in the art should understand that the number of transistors is not limited in this embodiment. Any form of pixel array 101, if applied to the corresponding image sensor in the embodiments of the present disclosure, should also be within the scope protected by the present disclosure. Since the image sensor structure provided in this embodiment has a simple logic and does not change the pixel array 101, with the continuous increase of the pixel array 101, it can still adaptively adjust the optical signal on the premise of ensuring the speed.

[0043] Figure 2 is a schematic diagram of an analog-to-digital conversion circuit shown according to an exemplary embodiment. As Figure 2 shown, the analog-to-digital conversion circuit 102 includes an amplifier 1021, a selector 1022, and an analog-to-digital converter 1023 (ADC). The input end of the amplifier 1021 is connected to each column of pixels in the pixel array 101. The output end of the amplifier 1021 is connected to the input end of the selector 1022. The output end of the selector 1022 is connected to the input end of the analog-to-digital converter 1023. The output end of the analog-to-digital converter 1023 is connected to the exposure timing control circuit 103. The analog-to-digital conversion circuit 102 is used to perform analog-to-digital conversion on the pixel voltages generated by each pixel in the currently output image frame of the pixel array 101 and then output pixel values. That is, the pixel voltages generated by each column of pixels in the currently output image frame of the pixel array 101 are sequentially input to the analog-to-digital converter 1023 through the amplifier 1021 and the selector 1022 connected to the corresponding column of the pixel array 101. The analog-to-digital converter 1023 performs analog-to-digital conversion on the pixel voltages output by the pixel array 101 and then outputs pixel values. By converting the pixel voltages generated by each pixel in the output image frame of the pixel array 101 into pixel values through the analog-to-digital conversion circuit 102, it is convenient to process the pixels subsequently and improve the processing efficiency.

[0044] In the embodiments of the present disclosure, an 8-bit analog-to-digital converter 1023 can be selected according to actual needs, or a 12-bit analog-to-digital converter 1023 can be selected. Those skilled in the art should understand that the number of bits of the analog-to-digital converter 1023 is not limited in this embodiment. Any form of analog-to-digital converter 1023, if applied to the corresponding image sensor in the embodiments of the present disclosure, should also be within the scope protected by the present disclosure.

[0045] In the embodiment of the present disclosure, the exposure timing control circuit 103 is configured to obtain pixel values corresponding to each pixel in the currently output image frame of the pixel array 101, and determine the exposure time of the pixel array 101 for outputting the next image frame according to the pixel values corresponding to each pixel in the currently output image frame. In other words, the exposure timing control circuit 103 obtains the pixel values corresponding to each pixel in the image frame output from the analog-to-digital converter 1023, searches for the maximum non-saturated pixel value among the pixel values corresponding to each pixel in the current image frame, and determines the minimum exposure time among the exposure times corresponding to the already output image frames. The exposure time of the pixel array 101 for outputting the next image frame is determined according to the maximum non-saturated pixel value and the minimum exposure time. The embodiment of the present disclosure gives full play to the pixel linear response characteristic through the exposure timing control circuit 103, and achieves light intensity adaptive exposure adjustment, solving the problem of motion artifacts caused by non-simultaneous exposure during multi-frame fusion when using the multiple sampling technique.

[0046] Figure 3 It is a schematic diagram of an exposure timing control circuit shown according to an exemplary embodiment. As Figure 3 shown, the exposure timing control circuit 103 in the image sensor 100 includes: a pixel scanning circuit 1031, an exposure time determination circuit 1032, and an exposure time feedback circuit 1033.

[0047] In the embodiment of the present disclosure, as Figure 4As shown, the pixel scanning circuit 1031 includes an output register 1031-2, a temporary register 1031-1, a comparator 1031-3, and a drive controller 1031-4. The output register 1031-2 receives the pixel values corresponding to the respective pixels in the image frame output by the analog-to-digital converter 1023. The maximum non-saturated pixel value is stored in the temporary register 1031-1, and the maximum non-saturated pixel value stored in the temporary register 1031-1 is the default value during initialization. When the output register 1031-2 receives the pixel value output by the analog-to-digital converter 1023, the comparator 1031-3 compares each pixel value in the output register 1031-2 with the maximum non-saturated pixel value stored in the temporary register 1031-1, and inputs the comparison result into the drive controller 1031-4. The drive controller 1031-4 determines whether to send an instruction to replace the maximum non-saturated pixel value to the output register 1031-2 according to the comparison result, that is, if the comparison result is that the pixel value in the output register 1031-2 is greater than the maximum non-saturated pixel value stored in the temporary register 1031-1, an instruction to replace the maximum non-saturated pixel value is sent to the output register 1031-2. If the output register 1031-2 receives the instruction to replace the maximum non-saturated pixel value, the received pixel value is input and stored as the maximum non-saturated pixel value. The pixel scanning circuit 1031 scans out the maximum non-saturated pixel value from the pixel values corresponding to the respective pixels of the image frame. The scanned maximum non-saturated pixel value is stored in the temporary register 1031-1 and output to the exposure time determination circuit 1032 connected to the pixel scanning circuit 1031 through the temporary register 1031-1. In this embodiment, the pixel scanning circuit 1031 performs pixel scanning work in real time. When the scanning of one frame of the image frame is completed, the maximum non-saturated pixel value is obtained, and the scanned maximum non-saturated pixel value is used as the basis for determining the exposure time of the next frame output by the pixel array 101. In this embodiment, the number of bits of the output register 1031-2, the temporary register 1031-1, and the comparator 1031-3 is not limited. According to the actual scenario requirements, the number of bits matching the aforementioned analog-to-digital converter 1023 can be selected.

[0048] In one implementation, the exposure time determination circuit 1032 is configured to receive the maximum non-saturated pixel value input by the pixel scanning circuit 1031 and determine the exposure time of the next frame of the image frame output by the pixel array based on the maximum non-saturated pixel value. As Figure 5As shown, the exposure time determination circuit 1032 includes a minimum exposure time register 1032-1, an exposure multiple register 1032-3, and an exposure decision circuit 1032-2. In this embodiment, the process of determining the exposure time for the pixel array 101 to output the next image frame based on the maximum non-saturated pixel value is described by the minimum exposure time register 1032-1, the exposure multiple register 1032-3, and the exposure decision circuit 1032-2. Among them, the minimum exposure time register 1032-1 is used to store the minimum exposure time in the image frames already output by the pixel array 101. The exposure multiple register 1032-3 is used to store the adjustment multiple value. The exposure decision circuit 1032-2 is used to count the exposure times in the image frames already output by the pixel array 101, determine the minimum exposure time among the exposure times in the image frames already output by the pixel array 101, and store the minimum exposure time in the minimum exposure time register 1032-1. The exposure decision circuit 1032-2 is used to receive the maximum non-saturated pixel value input by the pixel scan circuit 1031, and determine the adjustment multiple value for adjusting the exposure time of the next image frame based on the ratio of the most significant bit of the maximum non-saturated pixel value to the minimum exposure time. The exposure decision circuit 1032-2 is used to determine the exposure time for the pixel array 101 to output the next image frame based on the adjustment multiple and the exposure time of the current image frame. Exemplarily, the adjustment multiple value for adjusting the exposure time of the next image frame is 0.1 times. After reducing the exposure time of the current image frame output by the pixel array 101 by 0.1 times, it is used as the exposure time for the pixel array 101 to output the next image frame. The exposure time determination circuit 1032 outputs the exposure time of the next image to the exposure time feedback circuit 1033 connected to the exposure time determination circuit 1032.

[0049] In the embodiment of the present disclosure, the exposure time feedback circuit 1033 receives the exposure time of the next image input by the exposure time determination circuit 1032, and feeds back the exposure time of the next image to the pixel array 101.

[0050] In the embodiment of the present disclosure, as Figure 6 shown, the exposure timing control circuit 103 includes a pixel voltage detection circuit 1034. The pixel voltage detection circuit 1034 detects the pixel voltages generated by each pixel in the current image frame output by the pixel array 101. If there is a pixel whose pixel voltage is greater than the preset voltage threshold and the number of pixels whose pixel voltages are greater than the preset voltage threshold is greater than the pixel number threshold, it generates an identification signal for characterizing the adjustment of the exposure time of the next image frame. The pixel voltage detection circuit 1034 outputs the generated identification signal to the exposure time feedback circuit 1033 connected to the pixel voltage detection circuit 1034. When the exposure time feedback circuit 1033 receives the identification signal input by the pixel voltage detection circuit 1034, it feeds back the exposure time of the next image to the pixel array 101.

[0051] In the embodiments of the present disclosure, as Figure 7 shown, the pixel voltage detection circuit 1034 includes a threshold register 1034-1 and an analog comparator 1034-2. Among them, the threshold register 1034-1 is used to store a preset voltage threshold and a pixel number threshold. The analog comparator 1034-2 is used to compare the pixel voltages generated by each pixel in the currently output image frame of the pixel array 101 with the preset voltage threshold, and count the number of pixels whose pixel voltages are greater than the preset voltage threshold. When the number of pixels whose pixel voltages are greater than the preset voltage threshold is greater than the pixel number threshold, an identification signal for characterizing the exposure time adjustment of the next image frame is generated. In the embodiments of the present disclosure, the analog comparator 1034-2 is adopted. Compared with adopting a digital comparator, the area of the pixel voltage detection circuit 1034 in the image sensor is reduced and the delay is shortened. In addition, adopting the analog comparator 1034-2 compared with adopting an inverter avoids the problem that the accuracy does not meet the requirements when using the inverter to detect whether the pixel voltage is greater than the preset voltage threshold. In the embodiments of the present disclosure, the user sets the voltage threshold and the pixel number threshold according to the actual shooting scene, so as to directly adjust the exposure time of the image frame, and further expand the dynamic range.

[0052] In one implementation, when the number of pixels whose pixel voltages are greater than the preset voltage threshold is greater than the pixel number threshold, it indicates that the light intensity of the measured image no longer meets the user's requirements and the exposure time needs to be shortened. Therefore, an identification signal for characterizing the exposure time adjustment of the next image frame needs to be generated, and the exposure time feedback circuit 1033 connected to the pixel voltage detection circuit 1034 is instructed by this identification signal to adjust the exposure time of the next image frame.

[0053] In the embodiments of the present disclosure, as Figure 8 shown, the image sensor 100 includes an image fusion circuit 104.

[0054] The image fusion circuit 104 fuses the image frames output frame by frame by the pixel array 101 to obtain a fused preview raw (RAW) image frame, so as to implement digital reconstruction post-processing on multiple image frames, further improving the imaging quality and having high reliability and compatibility.

[0055] As can be seen from the above embodiments, the embodiments of the present disclosure start from the digital control and preprocessing circuit of the pixel array 101 in the image sensor, and propose an image sensor 100 with adaptive exposure adjustment and user-predefined exposure mode based on the existing drum exposure. The exposure time can range from 1 us to 65 ms. Under the conditions of a light response efficiency of 1.0 / lux-sec, a variable gain of 1, and an 8-bit ADC, the detected light intensity range can reach 0.11 ux - 106 lux. Among them, the exposure timing control circuit determines the exposure time of the next frame of the pixel array output image frame by obtaining the pixel values corresponding to the pixels in the current output image frame of the pixel array 101, and adjusts the exposure time of the next frame of the image frame when generating the identification signal, while when the identification signal is not generated, the next frame of the image frame is output according to the exposure time of the current output image frame, so as to achieve the light intensity adaptive exposure adjustment. In addition, during the adaptive exposure adjustment process, the user sets the voltage threshold and the pixel number threshold according to the shooting scene, which better meets the user's needs.

[0056] In order to more clearly illustrate the connection relationship between the various parts in the image sensor 100, in one embodiment, the above-mentioned parts are combined to obtain Figure 9Overall schematic diagram of the illustrated image sensor 100. Each pixel in the pixel array 101 converts the received optical signal into a pixel voltage. The pixel voltage performs two operations after passing through the amplifier 1021 (the amplifier 1021 is connected to the corresponding column in the pixel array 101 and is thus also referred to as a column amplifier later) and the multiplexer 1022 in the analog-to-digital conversion circuit 102. Through the first operation, the exposure time of the next image frame is obtained. Through the second operation, an identification signal for adjusting the exposure time of the next image frame is generated. In the case of generating the identification signal through the second operation, the exposure time of the next image frame obtained through the first operation is used as the exposure time for the pixel array to output the next image frame. Otherwise, the next image frame is output according to the exposure time of the current image frame. Among them, the first operation includes: the pixel voltage passes through the column amplifier 1021 and the multiplexer 1022 in the analog-to-digital conversion circuit 102 and then is input into the analog-to-digital converter ADC1023 to obtain the pixel value corresponding to each pixel voltage. The pixel value corresponding to each pixel voltage is input into the output register 1031-2 of the pixel scanning circuit 1031. The maximum non-saturated pixel value in the current frame is obtained through the pixel scanning circuit 1031, and the maximum non-saturated pixel value is output to the exposure time determination circuit 1032. The exposure decision circuit 1032-2 in the exposure time determination circuit 1032 receives the maximum non-saturated pixel value in the current frame input by the pixel scanning circuit 1031, and determines the adjustment multiple value for adjusting the exposure time of the next image frame according to the ratio of the highest bit of the maximum non-saturated pixel value in the current frame to the minimum exposure time. The exposure time of the current image frame is adjusted according to the adjustment multiple value to obtain the exposure time of the next image frame, and the exposure time of the next image frame is output to the exposure time feedback circuit 1033. The second operation includes that the pixel voltage after passing through the column amplifier 1021 and the multiplexer 1022 in the analog-to-digital conversion circuit 101 is output to the analog comparator 1034-2 in the pixel voltage detection circuit 1034. An identification signal for adjusting the exposure time of the next image frame is generated through the analog comparator 1034-2, and the identification signal is output to the exposure time feedback circuit 1033. In the exposure time feedback circuit 1033, if the identification signal input by the pixel voltage detection circuit 1034 is received, the exposure time of the next image frame input by the exposure time determination circuit is used as the exposure time for the pixel array to output the next image frame. Otherwise, the next image frame is output according to the exposure time of the current image frame. Finally, the image fusion circuit 104 fuses all the image frames output frame by frame by the pixel array 101 to obtain a preview RAW image frame. The image sensor 100 allows the user to adjust the exposure time of the output image frame, and by adjusting the exposure time of the output image frame, the purpose of expanding the image dynamic range is achieved.

[0057] Based on the same concept, an embodiment of the present disclosure provides an image generation method.

[0058] It is understandable that the image generation method provided in the embodiments of the present disclosure is applied to the above-mentioned image sensor. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solutions of the embodiments of the present disclosure.

[0059] Figure 10 is a flowchart of an image generation method shown according to an exemplary embodiment. Referring to Figure 10 , the image generation method, which is applied to an image sensor, includes the following steps.

[0060] In step S11, obtain the pixel values corresponding to each pixel in the currently output image frame of the pixel array of the image sensor.

[0061] In step S12, based on the pixel values corresponding to each pixel in the currently output image frame, determine the exposure time of the pixel array for outputting the next image frame.

[0062] In step S13, control the pixel array to generate the next image frame based on the exposure time of the next image frame.

[0063] In one implementation manner, determining the exposure time of the pixel array for outputting the next image frame based on the pixel values corresponding to each pixel in the currently output image frame includes: scanning out the maximum non-saturated pixel value from the pixel values corresponding to each pixel in the currently output image frame; based on the maximum non-saturated pixel value, determine the exposure time of the pixel array for outputting the next image frame.

[0064] In one implementation manner, determining the exposure time of the pixel array for outputting the next image frame based on the maximum non-saturated pixel value includes: determining the minimum exposure time in the image frames already output by the pixel array; based on the ratio of the most significant bit of the maximum non-saturated pixel value to the minimum exposure time, determine the adjustment multiple value for adjusting the exposure time of the next image frame; based on the adjustment multiple value for adjusting the exposure time of the next image frame, determine the exposure time of the pixel array for outputting the next image frame.

[0065] In one implementation manner, before controlling the pixel array to generate the next image frame based on the exposure time of the next image frame, the image generation method includes: detecting the pixel voltages generated by each pixel in the currently output image frame of the pixel array, if there are pixels whose pixel voltages are greater than the preset voltage threshold and the number of pixels whose pixel voltages are greater than the preset voltage threshold is greater than the pixel number threshold, then generate an identification signal for characterizing the adjustment of the exposure time of the next image frame; in the case of generating the identification signal, feedback the exposure time of the next image frame to the pixel array.

[0066] In one embodiment, the image generation method includes: fusing the image frames output frame by frame from the pixel array to obtain a fused preview original image frame.

[0067] Regarding the implementation process in the above embodiments, the specific implementation manners of each step have been described in detail in the embodiments related to the image sensor, and will not be elaborated here.

[0068] Figure 11 FIG. 7 is a block diagram of a device provided with an image sensor according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0069] Referring to Figure 11 , the device 200 may include one or more of the following components: a processing component 202, a memory 204, a power component 206, a multimedia component 208, an audio component 210, an input / output (I / O) interface 212, a sensor component 214, and a communication component 216.

[0070] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 202 may include one or more modules to facilitate the interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate the interaction between the multimedia component 208 and the processing component 202.

[0071] The memory 204 is configured to store various types of data to support the operation of the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0072] The power component 206 provides power to various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0073] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of a touch or swipe action, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0074] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC) that is configured to receive external audio signals when the device 200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, the audio component 210 further includes a speaker for outputting audio signals.

[0075] The I / O interface 212 provides an interface between the processing component 202 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0076] The sensor component 214 includes one or more sensors for providing a status assessment of various aspects of the device 200. For example, the sensor component 214 can detect the on / off state of the device 200, the relative positioning of components, such as the display and the keypad of the device 200. The sensor component 214 can also detect a change in the position of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200, and the temperature change of the device 200. The sensor component 214 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 214 can also include a light sensor, such as a CMOS or a CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 214 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0077] The communication component 216 is configured to facilitate communication between the device 200 and other devices in a wired or wireless manner. The device 200 can access a communication standard-based wireless network, such as WiFi, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0078] In an exemplary embodiment, the device 200 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0079] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the above instructions can be executed by a processor 220 of the device 200 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, Random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0080] It can be understood that "a plurality of" in this disclosure means two or more, and other quantifiers are similar. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after. The singular forms of "a", "the", and "said" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0081] It can be further understood that terms such as "first", "second", etc. are used to describe various information, but this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other and do not represent a specific order or degree of importance. In fact, the expressions such as "first" and "second" can be used interchangeably. For example, without departing from the scope of this disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0082] It can be further understood that, unless otherwise specified, "connection" includes both direct connection without other components between the two and indirect connection with other elements between the two.

[0083] It can be further understood that although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0084] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0085] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image sensor, characterized in that, The image sensor includes: a pixel array for outputting image frames frame by frame; an analog-to-digital conversion circuit for performing analog-to-digital conversion on the pixel voltages generated by each pixel in the currently output image frame of the pixel array and outputting pixel values; an exposure timing control circuit for obtaining the pixel values corresponding to each pixel in the currently output image frame of the pixel array and determining the exposure time for the pixel array to output the next image frame according to the pixel values corresponding to each pixel in the currently output image frame; The exposure timing control circuit includes: a pixel voltage detection circuit for detecting the pixel voltages generated by each pixel in the currently output image frame of the pixel array. If there is a pixel whose pixel voltage is greater than a preset voltage threshold and the number of pixels whose pixel voltage is greater than the preset voltage threshold is greater than a pixel number threshold, a flag signal for characterizing the adjustment of the exposure time of the next image frame is generated; an exposure time feedback circuit for feeding back the exposure time of the next image frame to the pixel array when the pixel voltage detection circuit generates the flag signal.

2. The image sensor according to claim 1, wherein The pixel voltage detection circuit includes: a threshold register for storing the preset voltage threshold and the pixel number threshold; an analog comparator for comparing the pixel voltages generated by each pixel in the currently output image frame of the pixel array with the preset voltage threshold and counting the number of pixels whose pixel voltage is greater than the preset voltage threshold. When the number of pixels whose pixel voltage is greater than the preset voltage threshold is greater than the pixel number threshold, a flag signal for characterizing the adjustment of the exposure time of the next image frame is generated.

3. The image sensor according to any one of claims 1 to 2, characterized in that The image sensor includes: an image fusion circuit for fusing the image frames output frame by frame by the pixel array to obtain a fused preview original image frame.

4. An image generation method, characterized in that, Applied to an image sensor, the image generation method includes: obtaining the pixel values corresponding to each pixel in the currently output image frame of the pixel array of the image sensor; determining the exposure time for the pixel array to output the next image frame based on the pixel values corresponding to each pixel in the currently output image frame; detecting the pixel voltages generated by each pixel in the currently output image frame of the pixel array. If there is a pixel whose pixel voltage is greater than a preset voltage threshold and the number of pixels whose pixel voltage is greater than the preset voltage threshold is greater than a pixel number threshold, a flag signal for characterizing the adjustment of the exposure time of the next image frame is generated; feeding back the exposure time of the next image frame to the pixel array when the flag signal is generated; controlling the pixel array to generate the next image frame based on the exposure time of the next image frame.

5. The image generation method according to claim 4, wherein The image generation method includes: fusing the image frames output frame by frame by the pixel array to obtain a fused preview original image frame.

6. An image sensor, characterized in that, It includes: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the image generation method according to any one of claims 4 to 5.

7. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enable the mobile terminal to execute the image generation method according to any one of claims 4 to 5.

Images