Shooting method, shooting device and readable storage medium

Abstract

The invention provides a shooting method, a shooting device and a readable storage medium, and relates to the technical field of terminals, and the shooting method comprises the steps: responding to a shooting instruction, and controlling multiple rows of photosensitive pixels to carry out light sensing; in the light sensing process of the multiple rows of light sensing pixels, the variable aperture is controlled to be in an open state in the global light sensing time period, and the variable aperture is controlled to be in a closed state in the non-global light sensing time period; the global photosensitive time period is a time period when the multiple rows of photosensitive pixels are all in a photosensitive state, and the non-global photosensitive time period is a time period when at least one row of photosensitive pixels in the multiple rows of photosensitive units are in the photosensitive state and at least one row of photosensitive pixels in the non-photosensitive state. In the shooting process, the multiple rows of photosensitive pixels in the image sensor are exposed at the same time in the global photosensitive time period by controlling the opening and closing state of the variable aperture, and inclination, deformation or distortion of the relative position of a moving object in a shooting scene in one frame of image can be avoided.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a shooting method, shooting device and readable storage medium. Background Technology

[0002] Aperture, electronic shutter, and image sensor are essential components of an imaging device. The aperture controls the amount of light entering the image sensor by adjusting the size of the opening. The electronic shutter controls the internal circuitry of the image sensor to regulate the pixel array's exposure based on either a global exposure mode or a rolling shutter mode, thereby producing an image.

[0003] In the rolling shutter exposure mode, the pixel array in the image sensor needs to be exposed line by line. When the object in the shooting area moves at high speed relative to the shooting device, the speed of the image sensor's line-by-line exposure may not be able to keep up with the speed of the object's movement. This causes the high-speed moving object to tilt, deform or distort in the captured image, resulting in the "jelly effect" and affecting image quality. Summary of the Invention

[0004] This application provides a shooting method, shooting device, and readable storage medium that can solve the technical problem of the rolling shutter effect when shooting high-speed moving objects in rolling shutter exposure mode.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides a shooting method applied to a shooting device equipped with a variable aperture and an image sensor, wherein the image sensor includes multiple rows of photosensitive pixels, and the shooting device can control the multiple rows of photosensitive pixels to perform light sensing in response to a shooting command; during the process of the multiple rows of photosensitive pixels performing light sensing, the variable aperture is controlled to be in an open state during the global light sensing period, and the variable aperture is controlled to be in a closed state during the non-global light sensing period; the global light sensing period is the period during which all rows of photosensitive pixels are in a light-sensing state, and the non-global light-sensing period is the period during which at least one row of photosensitive pixels in the multiple rows of photosensitive units is in a light-sensing state and at least one row of photosensitive pixels is in a non-light-sensing state.

[0007] Based on the shooting method provided in this application, the light-at-the-gate of the variable aperture is completely closed during the non-global light-sensing period of the image sensor, preventing light from entering the image sensor. During the global light-sensing period of the image sensor, the light-at-the-gate of the variable aperture is opened, allowing light to enter the image sensor through the variable aperture. This enables all photosensitive pixels in the image sensor to be exposed simultaneously, avoiding motion blur, distortion, and other phenomena in the captured image caused by different exposure times of the photosensitive pixels in different rows.

[0008] In one possible implementation of the first aspect, the method for controlling multiple rows of photosensitive pixels to perform light sensing includes: determining the amount of exposure to be increased by the shooting device according to the preset shooting parameters corresponding to the current shooting mode; determining the target shooting parameters of the shooting device according to the amount of exposure to be increased, the target shooting parameters including the target exposure gain, the target shutter speed, and the target aperture value of the light-gathering hole when the variable aperture is in the target open state; and controlling multiple rows of photosensitive pixels to perform light sensing according to the target shooting parameters.

[0009] Among them, the parameters in the preset shooting parameters can be parameters pre-stored in the shooting device corresponding to the current shooting mode, or parameters set by the user in the current shooting mode.

[0010] Based on this possible approach, light only enters the image sensor during the global light-sensing period. The variable aperture requires a certain amount of time to switch operating states, and the amount of light entering during this switching process is less than the amount of light entering when the variable aperture is in the target-open state within the same timeframe. Compared to existing rolling shutter exposure methods, the variable aperture switching operating states and the complete closing process during non-global light-sensing periods in this application both affect the total amount of light entering the imaging device, leading to a decrease in the exposure of the image sensor and the brightness of the image. Therefore, the amount of exposure to be increased is calculated in advance before capturing the image, and the lost light is compensated for by adjusting the shooting parameters to improve the brightness of the captured image.

[0011] In one possible implementation of the first aspect, the preset shooting parameters include a preset aperture value for the light-gathering aperture when the variable aperture is in a preset open state. The method by which the shooting device determines the amount of exposure to be increased based on the preset shooting parameters corresponding to the current shooting mode includes: determining a first exposure of the shooting device based on the preset shooting parameters; determining a second exposure of the shooting device based on the preset shooting parameters, the duration for which the variable aperture switches from a closed state to a preset open state, and the duration from the end of light sensing by the first row of photosensitive pixels to the end of light sensing by the last row of photosensitive pixels in the multi-row photosensitive pixels; and determining the amount of exposure to be increased based on the first and second exposures.

[0012] In one possible implementation of the first aspect, the preset shooting parameters further include preset exposure gain and preset shutter speed. The method for determining the target shooting parameters of the shooting device based on the amount of exposure to be increased includes: acquiring scene information of the scene to be shot; and adjusting the preset aperture value, preset exposure gain, and / or preset shutter speed based on the amount of exposure to be increased and the scene information to obtain the target shooting parameters.

[0013] Based on this possible implementation, different parameters can be adjusted in different shooting scenarios to compensate for brightness loss without affecting image quality. Scene information may include, but is not limited to, the brightness of the shooting scene, the required depth of field, and the required shutter speed. Based on the scene information, it can be determined whether the shooting scene is in low light or bright light; or whether it is a high-speed, slow-speed, or static shooting scene; or whether it is a large depth of field or a small depth of field shooting scene.

[0014] For example, the parameter adjustment strategies differ depending on the shooting scenario. In a large depth-of-field scenario, the area of ​​the light-gathering hole when the variable aperture is open can be increased to compensate for brightness loss by increasing the amount of light entering the shooting device during the global light-sensing period. In bright light scenarios and / or ultra-high-speed shooting scenarios, the exposure gain can be increased to compensate for brightness loss by improving the sensitivity of the image sensor to light. In low-light shooting scenarios, slow-motion shooting scenarios, or static shooting scenarios, the shutter speed can be reduced to increase the light-sensing time of each row of light-sensitive pixels and the duration of the global light-sensing period, thereby compensating for brightness loss by increasing the exposure time of the image sensor during the global light-sensing period.

[0015] In one possible implementation of the first aspect, the photosensitive pixels in multiple rows are in the photosensitive state for the same duration, and the time interval between the start of photosensitive states of two adjacent rows is a first duration; the global photosensitive period includes: the period from the start of photosensitive states of the last row of photosensitive pixels to the end of photosensitive states of the first row of photosensitive pixels; the non-global photosensitive period includes: the period from the start of photosensitive states of the first row of photosensitive pixels to the start of photosensitive states of the last row of photosensitive pixels, and the period from the end of photosensitive states of the first row of photosensitive pixels to the end of photosensitive states of the last row of photosensitive pixels.

[0016] In this implementation, the multiple rows of photosensitive pixels in the image sensor can be exposed row by row. That is, the photosensitive pixels in each row are in the photosensitive state for the same duration, which is equal to 1 / target shutter speed. However, the photosensitive pixels in each row start to be photosensitive at different times, and the photosensitive pixels in each row end to be photosensitive at different times.

[0017] In one possible implementation of the first aspect, the method of controlling the variable aperture to be in an open state during the global light-sensing period and to be in a closed state during the non-global light-sensing period during the light-sensing process of multiple rows of photosensitive pixels includes: controlling the variable aperture to switch from a closed state to a target open state when the last row of photosensitive pixels begins to sense light; and controlling the variable aperture to switch from a target open state to a closed state during a second duration before the first row of photosensitive pixels finishes sensing light, wherein the second duration is the duration during which the variable aperture switches from a closed state to a target open state.

[0018] Based on this possible implementation, the variable aperture switches to a closed state before the first row of photosensitive pixels begins to sense light, and gradually opens to the target open state only when the last row of photosensitive pixels begins to sense light. It also gradually closes a second time interval before the first row of photosensitive pixels finishes sensing light. This allows light to be blocked by the closed variable aperture during the periods from the start of sensing by the first row of photosensitive pixels to the start of sensing by the last row, and also during the periods from the end of sensing by the first row of photosensitive pixels to the end of sensing by the last row, preventing light from entering the image sensor. This allows light to enter the image sensor through the light-gathering aperture during the periods when all photosensitive pixels are in sensing mode, ensuring simultaneous exposure of all photosensitive pixels and avoiding the rolling shutter effect caused by inconsistent exposure times of photosensitive pixels in different rows.

[0019] In one possible implementation of the first aspect, the multiple rows of photosensitive pixels begin to sense light at the same time, and the time interval between the end of sensing light for two adjacent rows of photosensitive pixels is a first duration; the global sensing period includes the period from the start of sensing light for the multiple rows of photosensitive pixels to the end of sensing light for the first row of photosensitive pixels in the multiple rows of photosensitive pixels; the non-global sensing period includes the period from the end of sensing light for the first row of photosensitive pixels to the end of sensing light for the last row of photosensitive pixels in the multiple rows of photosensitive pixels.

[0020] In this implementation, the multiple rows of photosensitive pixels in the image sensor start sensing light at the same time, but the time when each row of photosensitive pixels ends sensing light is different, and the duration of the multiple rows of photosensitive pixels in the sensing state increases row by row.

[0021] In one possible implementation of the first aspect, the method of controlling the variable aperture to be in an open state during the global light-sensing period and to be in a closed state during the non-global light-sensing period during the light-sensing process of multiple rows of photosensitive pixels includes: controlling the variable aperture to switch to a target open state when the multiple rows of photosensitive pixels begin to sense light; and controlling the variable aperture to switch from the target open state to a closed state during a second duration before the first row of photosensitive pixels ends to sense light, wherein the second duration is the duration during which the variable aperture switches from the closed state to the target open state.

[0022] Based on this possible implementation, the variable aperture can switch to the target open state in advance before multiple rows of photosensitive pixels begin sensing light simultaneously, and gradually close a second time period before the first row of photosensitive pixels finishes sensing light. This allows light to enter the image sensor through the light entrance of the variable aperture during the period from the first row of photosensitive pixels to the last row of photosensitive pixels being in a sensing state. During the period from the end of sensing light for the first row of photosensitive pixels to the end of sensing light for the last row of photosensitive pixels, the closed variable aperture blocks the light, ensuring that all photosensitive pixels in the image sensor are exposed simultaneously during the global sensing period, thus avoiding the rolling shutter effect caused by inconsistent exposure times of photosensitive pixels in different rows.

[0023] In a second aspect, this application provides an imaging device, including a variable aperture, an image sensor, and a processor, wherein the processor is configured to run a computer program stored in a memory to control the variable aperture and the image sensor to implement the method in any possible implementation of the first aspect.

[0024] Thirdly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the method in any possible implementation of the first aspect.

[0025] Fourthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method in any possible implementation of the first aspect.

[0026] The technical effects of the second to fourth aspects provided in this application can be found in the technical effects of the various possible implementations of the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0027] Figure 1 This is a timing diagram for the control of a global exposure mode.

[0028] Figure 2 This is a control timing diagram for a roller shutter exposure mode.

[0029] Figure 3 This is a schematic diagram of the structure of a shooting device provided in an embodiment of this application.

[0030] Figure 4 This is a schematic diagram of the structure of an image sensor provided in an embodiment of this application.

[0031] Figure 5 This is a schematic diagram illustrating an application scenario of a shooting device provided in an embodiment of this application.

[0032] Figure 6 This is a timing diagram for the control of a roller shutter exposure mode provided in an embodiment of this application.

[0033] Figure 7 This is a control flowchart of a shooting device provided in an embodiment of this application.

[0034] Figure 8 This is a control timing diagram for a semi-rolling shutter exposure mode provided in an embodiment of this application.

[0035] Figure 9 A control flowchart of another imaging device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings and related embodiments. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0037] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0038] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0039] The steps involved in the methods provided in this application are merely examples, and not all steps are mandatory. They can be added or removed as needed during use. In this application, the same step or steps with the same function can be referenced and learned from each other in different embodiments.

[0040] The shutter, aperture, and image sensor are essential components of an imaging device for achieving its imaging function. The shutter speed and the size of the aperture's opening adjust the amount of light entering the image sensor. The image sensor includes a pixel array composed of multiple rows of photosensitive pixels, with each row containing a plurality of photosensitive pixels arranged in a specific order. These photosensitive pixels can be photosensitive elements such as photodiodes.

[0041] Currently, shutters include electronic shutters and mechanical shutters. Electronic shutters control the switching of internal circuitry within the image sensor to keep the photosensitive pixels in a photosensitive or non-photosensitive state. Compared to mechanical shutters, electronic shutters offer advantages such as quiet operation, high speed, and small size. Therefore, mobile electronic devices such as mobile phones, tablets, and wearable devices, as well as miniature digital cameras and surveillance cameras, all use electronic shutters to capture images.

[0042] In existing technologies, common operating modes of image sensors include global exposure mode and rolling shutter exposure mode. Figure 1 This is a timing diagram for the control of a global exposure mode. Figure 2 This is a control timing diagram for a roller shutter exposure mode.

[0043] like Figure 1As shown, in global exposure mode, the imaging device first controls the pixel array in the image sensor to reset during the time interval from t0 to t1. Then, during the time interval from t1 to t2, it controls all photosensitive pixels in the image sensor to be in a photosensitive state and controls the aperture to be open, allowing light to enter the image sensor during this period when all photosensitive pixels are in a photosensitive state, thus exposing the image sensor. Finally, the photosensitive data of each photosensitive pixel is read line by line to obtain a digital image. The duration from t1 to t2 is equal to 1 / the preset shutter speed, and the photosensitive data is the charge signal accumulated by the photosensitive pixels during the exposure period from t1 to t2. The exposure period refers to the time when light enters the image sensor and some photosensitive pixels are in a photosensitive state. In this exposure mode, the pixel array in the image sensor can be exposed simultaneously and clearly capture objects moving relative to the imaging device. However, global exposure places high demands on the hardware performance of the imaging device, resulting in relatively high hardware costs.

[0044] Currently, due to limitations in hardware cost and power consumption, mobile electronic devices such as smartphones and tablets all capture images using a rolling shutter exposure mode. In rolling shutter mode, after the imaging device controls the aperture to open to a preset open state, it can control multiple rows of photosensitive pixels in the image sensor to sense light row by row. The time interval between the start and end of light sensing for adjacent rows of photosensitive pixels is the same. Specifically, for example... Figure 2 As shown, each row of photosensitive pixels needs to be reset before it begins sensing light. At time t1, the image sensor is controlled to sense light row by row, starting from the first row of photosensitive pixels. At time t2, the image sensor is controlled to end sensing light row by row, starting from the first row of photosensitive pixels. Finally, the light-sensing data of each photosensitive pixel is read out row by row to obtain a digital image. In this exposure mode, there is a period where some photosensitive pixels have ended exposure but others are still in the exposure state. Therefore, when objects in the shooting scene move at high speed relative to the shooting device, the objects will produce motion blur or distortion in the image, resulting in a "jelly effect" and poor image quality.

[0045] To address the technical problem of the "jelly effect" that occurs when shooting in dynamic scenes, this application provides a shooting method. In this method, during the process of controlling multiple rows of photosensitive pixels in an image sensor to sense light, the shooting device can control the variable aperture to be completely closed during periods when at least one row of photosensitive pixels is in a non-sensing state and at least one row of photosensitive pixels is in a sensing state, thus preventing light from entering the image sensor. Conversely, the variable aperture can be controlled to be open during periods when all rows of photosensitive pixels are in a sensing state, allowing all photosensitive pixels in the image sensor to be exposed simultaneously within the same time period. This avoids the "jelly effect" caused by different exposure times of the photosensitive pixels in dynamic scenes and improves image quality.

[0046] The shooting method provided in this application is applied to a shooting device that captures images using an electronic shutter. In one embodiment, the shooting device can be an electronic device with shooting capabilities, such as a camera, mobile phone, tablet computer, wearable device, or surveillance camera. Figure 3 This is a schematic diagram of the structure of a shooting device provided in an embodiment of this application, such as... Figure 3 As shown, the imaging device includes a system architecture and a hardware architecture. The system architecture includes an application layer, a camera multimedia control framework layer, and a hardware driver framework layer, which can communicate with each other via a software interface. The hardware architecture includes a variable aperture, an electronic shutter, an image sensor, and a processor.

[0047] The application layer includes system applications and third-party applications (or extended applications), such as the camera application used when taking pictures. After receiving the user's shooting command, the camera application can control the shooting device to acquire images through the camera multimedia framework and hardware driver framework.

[0048] The camera multimedia control framework includes an image processing module, an exposure control module, and an aperture control module. The image processing module processes the image acquired from the image sensor, performing tasks such as noise reduction and color correction. The exposure control module controls the exposure time of the shooting device. The aperture control module controls the operating state of the variable aperture, which can be closed or open. When the variable aperture is open, the aperture control module can also control the size of the light-gathering opening within the variable aperture. The hardware driver framework provides unified peripheral access capabilities and a framework for driver development and management.

[0049] A variable aperture can control the amount of light entering the shooting device by adjusting the size of the light-gathering hole in the variable aperture through a set of rotatable or movable blades.

[0050] An image sensor is a device that uses the photoelectric conversion function of an optoelectronic device to convert an optical image on a photosensitive surface into a charge signal that is proportional to the optical image. Image sensors can be charge-coupled device (CCD) image sensors or complementary metal-oxide-semiconductor (CMOS) image sensors.

[0051] Figure 4This is a schematic diagram of the structure of an image sensor provided in an embodiment of this application. The image sensor 40 is provided with a pixel array, which includes multiple rows of photosensitive pixels arranged in a row. Each row of photosensitive pixels includes multiple photosensitive pixels 41 arranged in a row. The photosensitive pixels 41 can be photosensitive elements that convert received light into image signals, such as photodiodes.

[0052] An electronic shutter may include a control circuit and a readout circuit for reading the charge signal of each photosensitive pixel. The control circuit controls the operating state of the photosensitive pixels by controlling the power supply to and from the photosensitive pixels in the image sensor. Specifically, the operating state of the photosensitive pixels when they are powered on is the photosensitive state, and the operating state of the photosensitive pixels when they are powered off is the non-photosensitive state.

[0053] A processor may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The processor can serve as the central nervous system and command center of the imaging device. The processor can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0054] Optionally, the imaging device may further include a memory for storing instructions, computer programs, and data. The memory may be integrated with the processor or exist independently. In this embodiment, the processor can run the computer program stored in the memory to control the variable aperture and image sensor to implement the imaging method provided in this embodiment.

[0055] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the shooting device. In other embodiments of this application, the shooting device may include... Figure 3 It may contain more or fewer components, or combine some components, or separate some components, or arrange the components differently. The components shown in the diagram may be implemented in hardware, software, or a combination of software and hardware.

[0056] In another embodiment, the shooting device can be a component disposed in an electronic device for implementing the shooting function. Specifically, the shooting device may include a variable aperture, an electronic shutter, an image sensor, and a control chip. The control chip includes a memory and a processor. The processor can execute a computer program stored in the memory to control the variable aperture, the electronic shutter, and the image sensor to implement the shooting method provided in the embodiments of this application.

[0057] The shooting method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings. The shooting method provided in the embodiments of this application is applied to a shooting device that captures images by means of an electronic shutter, and the shooting device includes a variable aperture and an image sensor.

[0058] In one embodiment, see Figure 5 The diagram illustrates one application scenario. It assumes the shooting device is a mobile phone with a camera application installed. Based on this, such as... Figure 5 As shown in a and b, after the user launches the camera application, the camera application interface can be displayed on the camera device's screen. The camera application interface includes a framing area, a menu area, and a shutter area. The framing area displays images captured in real-time by the image sensor in the camera device. The menu area includes controls for multiple shooting modes and controls for setting shooting parameters for the current shooting mode. Shooting modes include large aperture, night scene, portrait, photo, video, professional, and more. Shooting parameters include, but are not limited to: focal length, ISO (International Standard Organization), aperture value, exposure gain, and / or shutter speed. The aperture value is related to the diameter of the light-gathering aperture in the variable aperture. The shutter area displays, from left to right, gallery controls, shutter controls, and camera switching controls.

[0059] After the camera application is launched, the shooting device can determine the current shooting mode based on the user's trigger operation (such as clicking) on ​​a shooting mode control in the menu area, and enter the preview state according to the current shooting mode. At the same time, it can also open the variable aperture and image sensor in the shooting device according to the preset shooting parameters corresponding to the current shooting mode, so that the shooting device can acquire images of the scene to be shot in real time and display them in the shooting area.

[0060] The preset shooting parameters corresponding to the current shooting mode include the preset aperture value of the light-gathering hole in the variable aperture, the preset exposure gain, and the preset shutter speed. The values ​​of each parameter in the preset shooting parameters can be values ​​set by the user through the controls in the menu area used to set the shooting parameters for the current shooting mode, or values ​​of the parameters preset for the current shooting mode by the camera application developer and stored in the shooting device. For example, see... Figure 5In step c, after the user triggers the night mode control in the menu area, a preview interface of the night mode can be displayed on the screen. The menu area of ​​this interface displays a first control for setting the ISO and a second control for setting the shutter speed. The user can use the first control to set the ISO in night mode to 200 and the second control to set the shutter speed in night mode to 1 / 4 second.

[0061] Furthermore, after the shooting device detects the user's trigger operation (e.g., click operation) on the shutter control in the camera application interface, it can generate a shooting command, thereby controlling the variable aperture and image sensor in the shooting device to acquire and save the image of the scene to be shot through the shooting method provided in the embodiments of this application in the current shooting mode.

[0062] Specifically, in this embodiment, the imaging device can respond to an imaging command by controlling multiple rows of photosensitive pixels in the image sensor to sense light. During the process of multiple rows of photosensitive pixels sensing light, the variable aperture is controlled to be in an open state during the global light-sensing period and in a closed state during non-global light-sensing periods. The global light-sensing period is the period during which all rows of photosensitive pixels are in a light-sensing state, and the non-global light-sensing period is the period during which at least one row of photosensitive pixels is in a light-sensing state and at least one row of photosensitive pixels is in a non-light-sensing state.

[0063] Based on the shooting method provided in the embodiments of this application, the shooting device can control the opening and closing state of the variable aperture to allow light to enter the image sensor during the global light-sensing period, while preventing light from entering the image sensor during non-global light-sensing periods. This allows multiple rows of light-sensing pixels in the image sensor to be exposed simultaneously during the same period, avoiding the rolling shutter effect caused by the difference in exposure time of each row of light-sensing pixels.

[0064] In this embodiment, the light-sensing method of the multiple rows of photosensitive pixels in the image sensor can be either a rolling shutter light-sensing method or a half-rolling shutter light-sensing method. The rolling shutter light-sensing method refers to the multiple rows of photosensitive pixels in the image sensor starting and ending light-sensing row by row, with each row of photosensitive pixels being in the sensing state for the same duration, and the time interval between the start and end of light-sensing for any two adjacent rows being the same. The half-rolling shutter light-sensing method refers to the multiple rows of photosensitive pixels in the image sensor starting and ending light-sensing row by row, with the light-sensing duration of each row increasing sequentially from the starting row (e.g., the first row of photosensitive pixels), and the time interval between the end of light-sensing for any two adjacent rows being the same.

[0065] In the rolling shutter mode, the imaging device can control the variable aperture to be open during the corresponding global light-sensing period and closed during the corresponding non-global light-sensing period using the method described in Embodiment 1 below. In the semi-rolling shutter mode, the imaging device can control the variable aperture to be open during the corresponding global light-sensing period and closed during the corresponding non-global light-sensing period using the method described in Embodiment 2 below. These two methods will be described separately below.

[0066] Example 1

[0067] Figure 6 This document provides a control timing diagram for a roller shutter photosensitive mode, as illustrated in an embodiment of this application. In the roller shutter photosensitive mode, the imaging device can control multiple rows of photosensitive pixels in the image sensor to sense light row by row. Before any row of photosensitive pixels begins sensing light, all photosensitive pixels in that row are reset. After each row of photosensitive pixels finishes sensing light, the photosensitive data of all photosensitive pixels in that row is read out. The time required to reset each row of photosensitive pixels, the time required to read out the photosensitive data of each row, the time interval between the start of sensing light from two adjacent rows, and the time interval between the end of sensing light from two adjacent rows are all defined as a first duration.

[0068] Specifically, such as Figure 6 As shown, during the period from time t0 to time t1, the first row of photosensitive pixels is reset; at time t1, the first row of photosensitive pixels begins to sense light, and the first row of photosensitive pixels remains in a photosensitive state during the period from time t1 to time t6; after a first time interval from time t1, the second row of photosensitive pixels begins to sense light; after a first time interval from the time when the second row of photosensitive pixels begins to sense light, the third row of photosensitive pixels begins to sense light; and so on, until the last row of photosensitive pixels begins to sense light at time t3. At time t6, the first row of photosensitive pixels ceases photosensitive operation, and the imaging device can sequentially read the photosensitive data of each photosensitive pixel in the first row within a first time period starting from time t6. After an interval of the first time period from time t6, the second row of photosensitive pixels ceases photosensitive operation, and the imaging device can sequentially read the photosensitive data of each photosensitive pixel in the second row within a first time period starting from the moment the second row of photosensitive pixels ceases photosensitive operation. This process continues until time t7, when the last row of photosensitive pixels ceases photosensitive operation, and the imaging device sequentially reads the photosensitive data of each photosensitive pixel in the last row within a first time period. The imaging device can obtain a frame of the scene to be captured based on the photosensitive data of each photosensitive pixel in the image sensor.

[0069] In the rolling shutter mode, the start and end times of light-sensing for each row of pixels in the image sensor differ, but the duration of light-sensing for each row is the same. During image capture in this mode, the imaging device controls the variable aperture to be closed or open based on the working state of the multiple rows of pixels. This ensures simultaneous exposure of all rows of pixels within the global light-sensing period, while preventing light from entering the image sensor during non-global light-sensing periods, thus avoiding the rolling shutter effect in moving scenes.

[0070] Specifically, such as Figure 6 As shown, in the rolling shutter photosensitive mode, the global photosensitive period of the image sensor includes the period from the start of photosensitive pixels in the last row of multiple rows to the end of photosensitive pixels in the first row. The non-global photosensitive period of the image sensor includes the period from the start of photosensitive pixels in the first row to the start of photosensitive pixels in the last row, and the period from the end of photosensitive pixels in the first row to the end of photosensitive pixels in the last row.

[0071] It should be noted that the variable aperture takes a certain amount of time to switch from any open state to a closed state and from a closed state to any open state. Therefore, in the rolling shutter photosensitive mode, in order to prevent light from entering the image sensor during non-global photosensitive periods, it is necessary to close the variable aperture in advance before the first row of photosensitive pixels begins to sense light, and open the variable aperture after the last row of photosensitive pixels begins to sense light, and close the variable aperture in advance before the first row of photosensitive pixels finishes sensing light, based on the time required for the variable aperture to switch working states.

[0072] For details, see Figure 6 The timing diagram shown and Figure 7The schematic diagram of the control flow of the shooting device shown illustrates that, in Embodiment 1 of this application, after the user activates the shooting function of the shooting device, the device is in preview mode. At this time, the variable aperture is in a preset open state, and the aperture value of the light-gathering hole in the variable aperture is a preset aperture value in the preset shooting parameters. Responding to the shooting command, the shooting device can control the first row of photosensitive pixels in the image sensor to begin sensing light at time t1. Within a third time period before the first row of photosensitive pixels begins sensing light, the device can control the variable aperture to gradually switch from the preset open state to the closed state. The third time period is the duration required for the variable aperture to switch from the preset open state to the closed state. The aperture value of the light-gathering hole in the closed state of the variable aperture is 0. The shooting device can also control the variable aperture to open at time t3 when the last row of photosensitive pixels begins sensing light, thereby switching the variable aperture from the closed state to the target open state during the time period from time t3 to time t4. The imaging device can control the first row of photosensitive pixels to stop sensing light at time t6, and within a second time period before the first row of photosensitive pixels stops sensing light (i.e., the time period from time t5 to time t6), control the variable aperture to switch from the target open state to the target closed state. Both the time required for the variable aperture to switch from the target open state to the target closed state and the time required to switch from the closed state to the target open state are considered part of the second time period.

[0073] Based on the method provided in Embodiment 1 of this application, the variable aperture can be kept closed during the period from the start of light sensing by the first row of photosensitive pixels to the start of light sensing by the last row of photosensitive pixels, and also during the period from the end of light sensing by the first row of photosensitive pixels to the end of light sensing by the last row of photosensitive pixels, thereby preventing light from entering the image sensor during these two periods. Alternatively, the variable aperture can be kept open during the period from the start of light sensing by the last row of photosensitive pixels to the end of light sensing by the first row of photosensitive pixels, allowing light to enter the image sensor and simultaneously expose all photosensitive pixels in the image sensor. Even when there are fast-moving objects in the shooting scene, or when the shooting device is used for mobile shooting, the relative positions of the objects in the shooting scene within a single frame will not change, thus avoiding the "rolling shutter effect."

[0074] It should also be noted that the amount of light entering the imaging device is related to the size of the light-receiving aperture in the variable aperture, and the exposure of the imaging device is related to the exposure time of the photosensitive pixels in the image sensor. The exposure time of any row of photosensitive pixels is the duration during which light enters that row of photosensitive pixels and that row of photosensitive pixels is in a photosensitive state. In existing technology, during the process of the imaging device responding to a shooting command to capture an image, it generally controls the light-receiving aperture in the variable aperture to open to a preset aperture value based on preset shooting parameters corresponding to the current shooting mode. During the shooting process, the state of the variable aperture remains unchanged, and the exposure time of each row of photosensitive pixels is 1 / preset shutter speed. That is to say, light enters the image sensor during the period when the pixel array in the image sensor is in a photosensitive state. However, in Embodiment 1 of this application, the variable aperture is closed during the non-global light-sensing period of the image sensor, and no light enters the image sensor during this period. Therefore, in this embodiment, if the shooting device still controls the variable aperture and the image sensor to capture images according to the preset shutter speed and preset aperture value in the preset shooting parameters corresponding to the current shooting mode, it will result in a reduction in the amount of light entering the shooting device and the exposure time of the image sensor, and the brightness of the captured image will be lower than that of the image captured based on the prior art.

[0075] Furthermore, the amount of light entering the camera during the switching of the variable aperture's working state throughout the entire light-sensing period is less than the amount of light entering the camera when the variable aperture is always open for the same period of time. Therefore, the switching of the variable aperture's working state during image capture will also affect the amount of light entering the camera and the brightness of the image.

[0076] Based on the above description, in Embodiment 1 of this application, the shooting device can adjust the preset shooting parameters in the current shooting mode through the following two steps to obtain the target shooting parameters, and then control the variable aperture and image sensor to capture images based on the target shooting parameters, so as to compensate for the loss of light intake caused by the variable aperture closing during non-global light-sensing periods and switching working states during global light-sensing periods, thereby ensuring image brightness.

[0077] Step 1: Determine the amount of exposure to be increased for the shooting device based on the preset shooting parameters corresponding to the current shooting mode.

[0078] Specifically, the initial exposure of the shooting device can be determined first based on preset shooting parameters. Then, the second exposure can be determined based on the preset shooting parameters, the duration for the variable aperture to switch from a closed state to a preset open state, and the duration from the end of light sensing by the first row of photosensitive pixels to the end of light sensing by the last row of photosensitive pixels. Finally, the additional exposure to be increased can be determined based on the first and second exposures. The aperture value of the light-gathering hole when the variable aperture is in the preset open state is the preset aperture value in the preset shooting parameters.

[0079] The preset shooting parameters corresponding to the current shooting mode may include: the preset aperture value r of the light-gathering hole in the variable aperture when the variable aperture is in the preset open state. old Initial exposure gain old and initial shutter speed t old The light-gathering aperture in a variable aperture can be circular, polygonal, or other shapes.

[0080] The first exposure indicator is based on Figure 2 The existing rolling shutter exposure method and preset shooting parameters show the amount of light entering the image sensor through the variable aperture. First exposure value (Expo) old This can be expressed as formula (1):

[0081] Expo old =S old *t old *gain old (1)

[0082] Among them, S old The aperture of the variable aperture is opened to the preset aperture value r. old The area at that time. For example, if the light-attracting aperture of the variable aperture is circular, and the aperture diameter value refers to the diameter of the light-attracting aperture, then the light-attracting aperture opens to the preset aperture diameter value r. old Area S at time old =π(r old ) 2 .

[0083] The second exposure amount refers to the amount of light entering the image sensor determined based on the shutter exposure method and preset shooting parameters provided in the embodiments of this application. The second exposure amount Expo can be expressed as formula (2):

[0084] Expo=S*(t old -t readout -2t move )*gain old +2S move *gain old (2)

[0085] Among them, t readout S represents the time interval between the end of photosensitive pixel detection in the first row and the end of photosensitive pixel detection in the image sensor. move This represents the integral area of ​​the light-gathering aperture as the variable aperture moves from a closed state to a preset open state. Where t is the time variable, a represents the linear acceleration of the blade during the blade's movement when the variable aperture switches operating states, t moveThis indicates the time required for the variable aperture to switch from a closed state to a preset open state.

[0086] It should be noted that the linear acceleration of the blades is fixed when the variable aperture switches between operating states. The time required for the variable aperture to switch from the closed state to any open state is related to the aperture value of the light inlet in the open state. That is, the aperture value of the variable aperture in an open state is equal to the product of the time required for the variable aperture to switch from the closed state to the open state and the linear acceleration.

[0087] Based on the first and second exposures, the amount of exposure to be increased (Expo) can be determined. loss =Expo old -Expo.

[0088] Step 2: Determine the target shooting parameters of the shooting device based on the amount of exposure to be increased.

[0089] In one possible implementation, the shooting device can first acquire scene information of the scene to be shot, and then adjust the preset aperture value, preset exposure gain, and / or preset shutter speed in the preset shooting parameters according to the desired increase in exposure and the scene information, thereby obtaining the target shooting parameters. The scene information may include, but is not limited to: the brightness of the scene to be shot, the type of the target in the scene, and whether there are ultra-fast moving targets in the scene. The type of target in the scene can be natural scenery (e.g., flowers, grass, trees, buildings, etc.) or people, etc.

[0090] In one embodiment, the capturing device can acquire one or more initial images of the scene to be captured in real time in a preview state before capturing the image. It can then use image recognition or image processing algorithms to detect parameters (e.g., brightness, contrast) or targets in the acquired initial images to obtain scene information. For example, if the position of the same target is detected to be offset in multiple initial images, and the offset of the target in two adjacent initial images is greater than or equal to a first preset offset, it can be determined that there is a high-speed moving target in the scene to be captured; if the offset of the target in two adjacent initial images is less than the first preset offset, it can be determined that there is a slow-moving target in the scene to be captured.

[0091] In another embodiment, the shooting device can acquire scene information of the scene to be shot based on the current shooting mode. For example, if the current shooting mode is night scene mode, the shooting device can determine that the brightness of the scene to be shot is low; if the current shooting mode is portrait mode, it can determine that the shooting target in the scene to be shot is a person; if the preset shutter speed corresponding to the current shooting mode is greater than or equal to a preset speed threshold (e.g., 1 / 300 second), it can determine that there is a shooting target moving at extremely high speed in the scene to be shot.

[0092] Furthermore, the shooting device can determine the scene type of the scene to be shot based on the acquired scene information, and then determine which one or more parameters in the preset shooting parameters to adjust based on the scene type, thereby compensating for the brightness loss caused by the variable aperture closing during non-global exposure periods and the variable aperture switching working state without affecting the image quality.

[0093] The scene type to be shot can include one or more of the following: low light scene, bright light scene, ultra-high speed shooting scene, slow speed shooting scene, still shooting scene, large depth of field shooting scene, and small depth of field shooting scene.

[0094] For example, the shooting device can determine whether the scene to be shot is a dark scene or a bright scene based on the brightness of the scene to be shot. For instance, if the brightness value of the image is greater than a first preset threshold, the scene to be shot can be determined to be a bright scene; if the brightness value of the image is less than a second preset threshold, the scene to be shot can be determined to be a dark scene. The second preset threshold is less than the first preset threshold.

[0095] The shooting device can also determine the required depth of field when shooting a scene based on the type of target in the scene. For example, if the target is a natural landscape, the scene can be determined to be a scene with a large depth of field; if the target is a person, the scene can be determined to be a scene with a small depth of field.

[0096] The shooting device can also determine whether the scene to be shot is a high-speed shooting scene, a slow-speed shooting scene, or a static shooting scene based on whether there is a moving target in the scene and the speed of the target's movement relative to the shooting device. For example, if there is a target moving at an extremely high speed in the scene to be shot, then the scene to be shot can be determined to be a high-speed shooting scene.

[0097] In this embodiment, the shooting device can determine one or more preset shooting parameters that need to be adjusted based on the scene type. For example, if the scene to be shot is a large depth-of-field shooting scene, the preset aperture value of the light-receiving aperture can be increased to compensate for brightness loss by increasing the amount of light entering the shooting device during the global light-sensing period. If the scene to be shot is a bright scene and / or an ultra-high-speed shooting scene, the preset exposure gain can be increased to compensate for brightness loss by improving the sensitivity of the image sensor to light. If the scene to be shot is a low-light scene, a slow-motion shooting scene, or a static shooting scene, the preset shutter speed can be decreased. Since shutter speed is negatively correlated with the duration of light-sensing for each row of photosensitive pixels, decreasing the preset shutter speed can increase the duration of light-sensing for each row of photosensitive pixels and the exposure time of each row of photosensitive pixels, thereby compensating for brightness loss. If the scene to be shot is both a bright scene and a large depth-of-field shooting scene, the preset aperture value of the light-receiving aperture and the preset exposure gain can be increased simultaneously to compensate for brightness loss.

[0098] It should be noted that the faster the target in the shooting scene moves, the shorter the exposure time required during the shooting process, and the faster the shutter speed required. Therefore, in ultra-high speed shooting scenes, the loss of brightness cannot be compensated by reducing the preset shutter speed.

[0099] After determining at least one parameter that needs adjustment from the preset shooting parameters using the above method, the shooting device can determine the target shooting parameters in different ways. These target shooting parameters include the target exposure gain, the target shutter speed, and the target aperture value of the light-gathering aperture when the variable aperture is in the target open state.

[0100] In one possible implementation, the shooting device may pre-store a first mapping relationship between the increase in the second exposure and any one or more of the following: the increase in the aperture value of the light-gathering aperture, the decrease in the shutter speed, and the increase in the exposure gain; and a second mapping relationship between the increase in the aperture value of the light-gathering aperture and the additional time required for the light-gathering aperture to switch from an open state to a closed state. Based on the first mapping relationship, the second mapping relationship, and at least one parameter that needs to be adjusted from the pre-determined preset shooting parameters, the shooting device can determine the exposure S to be increased. loss The target shooting parameters are determined by increasing or decreasing at least one parameter that needs to be adjusted.

[0101] In another possible implementation, after the shooting device determines the parameters that need to be adjusted in the preset shooting parameters, it can increase or decrease the parameters that need to be adjusted, and determine the second exposure of the shooting device based on the increased or decreased parameters and the above formula (2). If the second exposure is the same as or similar to the first exposure, the increased or decreased parameters corresponding to the second exposure can be used as the target shooting parameters. Here, the second exposure being similar to the first exposure means that the difference between the first exposure and the second exposure is within a preset difference range.

[0102] The shooting device adjusts the preset shooting parameters in the current shooting mode using the methods described in the two steps above. After obtaining the target shooting parameters, it can determine the target exposure duration from the start to the end of exposure for each row of photosensitive pixels in the rolling shutter mode based on the target shutter speed in the target shooting parameters. Then, it controls the pixel array in the image sensor to expose each row according to the target exposure duration and the target exposure gain. At the same time, it controls the variable aperture to switch its working state during the exposure process of the image sensor according to the preset aperture value in the preset shooting parameters and the target aperture value in the target shooting parameters, thereby compensating for the brightness loss caused by the variable aperture closing during non-global exposure periods and the variable aperture switching its working state during the global exposure period.

[0103] Optionally, see Figure 7 The diagram illustrates the control flow of the imaging device. After capturing one frame, the device controls the image sensor to switch the variable aperture from a closed state back to a preset open state. At this point, the device re-enters preview mode, preparing for the next image capture. When the device detects a new shooting command, it determines new target shooting parameters based on the preset shooting parameters corresponding to the new shooting mode, and then... Figure 6 The timing diagram and new target shooting parameters shown control the image sensor and variable aperture to capture new images.

[0104] Furthermore, after the shooting function of the shooting device is turned off, the shooting device can control the variable aperture to switch to the initial state. In the initial state, the aperture value of the light-gathering hole of the variable aperture can be any value. For example, the aperture value of the light-gathering hole in the initial state can be 0 or the maximum aperture value.

[0105] Example 2

[0106] Figure 8This is a timing diagram illustrating a control method for a semi-rolling shutter photosensitive mode, provided in an embodiment of this application. In the semi-rolling shutter photosensitive mode, the imaging device can first simultaneously reset all photosensitive pixels in the image sensor, then, after the reset, control multiple rows of photosensitive pixels to simultaneously begin exposure, and control the multiple rows of photosensitive pixels to end exposure row by row, and after each row of photosensitive pixels ends exposure, read out the photosensitive data of all photosensitive pixels in that row. The time required to reset each row of photosensitive pixels, the time required to read out the photosensitive data of each row of photosensitive pixels, and the time interval between the end of exposure for two adjacent rows of photosensitive pixels are all defined as a first duration.

[0107] Specifically, such as Figure 8 As shown, during the time interval from time t0 to time t1, all photosensitive pixels in the image sensor are reset; at time t1, all photosensitive pixels in the image sensor begin to sense light; after an interval of the target sensing time from time t1, the first row of photosensitive pixels ends sensing light at time t3; at time t4, after an interval of the first time from time t3, the second row of photosensitive pixels ends sensing light, and during the time interval from time t3 to time t4, the sensing data of all photosensitive pixels in the first row can be read sequentially; at time t5, after an interval of the first time from time t4, the third row of photosensitive pixels ends sensing light. The system can sequentially read the photosensitive data of all photosensitive pixels in the second row of photosensitive pixels during the time interval from t4 to t5. Similarly, at time t6, after a first time interval from the end of photosensitive data in the penultimate row, the last row of photosensitive pixels ends photosensitive data. During the time interval from the end of photosensitive data in the penultimate row to time t6, the photosensitive data of all photosensitive pixels in the penultimate row can be read sequentially. During the time interval from t6 to t7, the photosensitive data of all photosensitive pixels in the last row can be read sequentially.

[0108] In the semi-rolling shutter mode, each row of photosensitive pixels in the image sensor starts sensing light at the same time, but the time they end sensing light differs, and the duration of light-sensing in each row also varies. During image capture in this mode, the imaging device can control the variable aperture to be closed or open based on the working state of the multiple rows of photosensitive pixels. This ensures that all rows of photosensitive pixels are exposed simultaneously during the global sensing period, while preventing light from entering the image sensor during non-global sensing periods, thus avoiding the rolling shutter effect in moving scenes.

[0109] Among them, such as Figure 8As shown, the global light-sensing period of an image sensor is the period from the start of light sensing by multiple rows of light-sensing pixels to the end of light sensing by the first row of light-sensing pixels. The non-global light-sensing period of an image sensor is the period from the end of light sensing by the first row of light-sensing pixels to the end of light sensing by the last row of light-sensing pixels.

[0110] It should be noted that the variable aperture requires a certain amount of time to switch from any open state to a closed state, from a closed state to any open state, and from one open state to another. Therefore, in the semi-rolling shutter photosensitive mode in Embodiment 2, in order to prevent light from entering the image sensor during non-global photosensitive periods, the variable aperture needs to be closed in advance before the first row of photosensitive pixels finishes photosensitive, based on the time required for the variable aperture to switch working states.

[0111] For details, see Figure 8 The timing diagram shown and Figure 9 The control flow diagram of the shooting device shown is illustrated in Embodiment 2 of this application. After the user activates the shooting function of the shooting device, the device is in preview mode. At this time, the variable aperture can be in a preset open state, and the aperture value of the light-gathering hole in the variable aperture is a preset aperture value in the preset shooting parameters. Responding to the shooting command, the shooting device can control all photosensitive pixels in the image sensor to simultaneously begin sensing light at time t1, and within the fourth time period before each row of photosensitive pixels begins sensing light, control the variable aperture to gradually switch from the preset open state to the target open state. The shooting device can also control the first row of photosensitive pixels to end sensing light at time t3, and within the second time period before the first row of photosensitive pixels ends sensing light (i.e., the time period from time t2 to time t3), control the variable aperture to switch from the target open state to the closed state.

[0112] Based on the method provided in Embodiment 2, during the non-global photosensitive period from the end of photosensitive pixel exposure in the first row to the end of photosensitive pixel exposure in the last row, no light can enter the image sensor through the closed variable aperture. During the global photosensitive period from the start of photosensitive pixel exposure to the end of photosensitive pixel exposure in the first row, light can enter the image sensor through the variable aperture. This allows all photosensitive pixels in the image sensor to be exposed simultaneously, avoiding motion blur, distortion, and other phenomena in a single frame of an image of an object moving relative to the shooting device in motion shooting scenarios, thereby improving image quality.

[0113] As described in Embodiment 1, the amount of light entering the imaging device is related to the size of the light-receiving aperture in the variable aperture, and the exposure of the imaging device is related to the exposure time of the photosensitive pixels in the image sensor. In Embodiment 2 of this application, the variable aperture is in a closed state during the period from the end of the first row of photosensitive pixels to the end of the last row of photosensitive pixels, and no light enters the image sensor during this period. Therefore, if the imaging device still controls the variable aperture and image sensor to capture images according to the preset shutter speed and preset aperture value in the preset shooting parameters corresponding to the current shooting mode, it will result in a reduction in the amount of light entering the imaging device and the exposure time of the image sensor, and a decrease in the brightness of the image. In addition, the amount of light entering the variable aperture during the process of switching from the open state to the closed state during the global light-receiving period is less than the amount of light entering the variable aperture when it is always in the open state for the same period of time. Therefore, the process of the variable aperture closing during the image capture process will also affect the amount of light entering the imaging device and the brightness of the image.

[0114] Based on the above description, in Embodiment 2 of this application, the shooting device can adjust the preset shooting parameters in the current shooting mode through the following two steps to obtain the target shooting parameters, so as to compensate for the brightness loss caused by the variable aperture closing during the non-global light-sensing period of the image sensor and improve the image brightness.

[0115] Step 1: Determine the amount of exposure to be increased for the shooting device based on the preset shooting parameters corresponding to the current shooting mode.

[0116] Specifically, the first exposure of the shooting device can be determined based on the preset shooting parameters; then, the second exposure of the shooting device can be determined based on the preset shooting parameters, the duration of the variable aperture switching from the closed state to the preset open state, and the duration from the end of light sensing by the first row of light-sensitive pixels to the end of light sensing by the last row of light-sensitive pixels; finally, the amount of exposure to be increased can be determined based on the first and second exposures.

[0117] The first exposure amount refers to the amount of light entering the image sensor, determined based on the existing semi-rolling shutter exposure method and preset shooting parameters. It should be noted that the aperture remains open throughout the image capture process using the existing semi-rolling shutter exposure method. The method for determining the first exposure amount can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0118] The second exposure refers to the amount of light entering the image sensor through the variable aperture, determined based on the semi-rolling shutter exposure method and preset shooting parameters provided in the embodiments of this application. The second exposure, Expo, can be expressed as formula (3):

[0119] Expo=S*(t old -treadout -t move )*gain old +S move *gain old (3)

[0120] Among them, t readout S represents the time interval between the end of photosensitive pixel detection in the first row and the end of photosensitive pixel detection in the image sensor. move This represents the integral area of ​​the light-gathering aperture as the variable aperture moves from a closed state to a preset open state. Where t is the time variable, a represents the linear acceleration of the blade during the blade's movement when the variable aperture switches operating states, t move This indicates the time required for the variable aperture to switch from a closed state to a preset open state.

[0121] Based on the first and second exposures, the amount of exposure to be increased (Expo) can be determined. loss =Expo old -Expo.

[0122] Step 2: Determine the target shooting parameters of the shooting device based on the amount of exposure to be increased.

[0123] The target shooting parameters include the target exposure gain, the target shutter speed, and the target aperture value of the light-gathering hole in the variable aperture. The method for determining the target shooting parameters based on the scene information and the amount of exposure to be increased can be found in the relevant description of step two in Example 1, and will not be repeated here.

[0124] The shooting device adjusts the preset shooting parameters in the current shooting mode using the methods described in the two steps above. After obtaining the target shooting parameters, it can first determine the target exposure duration between the start and end of exposure for each row of photosensitive pixels in the half-rolling shutter mode based on the target shutter speed in the target shooting parameters. Then, it controls the pixel array in the image sensor to expose each row according to the target exposure duration and the target exposure gain. At the same time, it controls the variable aperture to switch its working state during the exposure process of the image sensor according to the preset aperture value in the preset shooting parameters and the target aperture value in the target shooting parameters, thereby compensating for the brightness loss caused by the variable aperture closing during non-global exposure periods and the variable aperture switching its working state during the global exposure period.

[0125] It should be noted that, unlike in Embodiment 1, in Embodiment 2, the target photosensitive duration determined based on the target shutter speed in the target shooting parameters can be the duration for which the starting row of photosensitive pixels in the image sensor is in a photosensitive state. The starting row of photosensitive pixels can be either the first row or the last row of photosensitive pixels in the image sensor. The shooting device can determine the photosensitive duration of each of the other rows of photosensitive pixels based on the duration of the starting row of photosensitive pixels in a photosensitive state and the time interval between the end of photosensitive states between any two rows of photosensitive pixels.

[0126] Furthermore, the preset aperture value of the variable aperture's light-gathering aperture in the preset open state can be the same as or different from the target aperture value of the variable aperture's light-gathering aperture in the target open state during image capture. When the preset aperture value and the target aperture value are different, the time required for the variable aperture to switch from the preset open state to the target open state can be a fourth time period, and the time required to switch from the target open state to the closed state can be a second time period. In this case, the aperture value of the light-gathering aperture in the closed state is 0. When the preset aperture value and the target aperture value are the same, the preset open state of the variable aperture is the same as the target open state. Based on this, the imaging device does not need to control the variable aperture's switching operating state before each row of photosensitive pixels begins to sense light.

[0127] Optionally, see Figure 9 The diagram illustrates the control flow of the imaging device. After the image sensor captures one frame, the device can control the variable aperture to switch back from a closed state to a preset open state. At this time, the imaging device re-enters preview mode, preparing for the next image capture. When the imaging device detects a new shooting command, it can determine new target shooting parameters based on the preset shooting parameters corresponding to the new shooting mode, and based on... Figure 8 The timing diagram and new target shooting parameters shown control the image sensor and variable aperture to capture new images. Furthermore, after the shooting function of the shooting device is turned off, the shooting device can control the variable aperture to switch to its initial state. In the initial state, the aperture value of the variable aperture can be any value.

[0128] Based on the methods provided in the above embodiments, this application also provides the following:

[0129] This application provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute the various steps or processes executed by the imaging device in any of the above method embodiments.

[0130] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the various steps or processes performed by the imaging device in any of the above method embodiments.

[0131] It should be understood that the processor mentioned in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0132] It should also be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. In the above embodiments, the descriptions of each embodiment have different focuses; parts not described in detail or recorded in a certain embodiment can be referred to in the relevant descriptions of other embodiments.

[0134] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0138] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a large-screen device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0139] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A shooting method, characterized in that, Applied to a shooting device, the shooting device including a variable aperture and an image sensor, the image sensor including multiple rows of photosensitive pixels, the method includes: In response to the shooting command, the multi-row photosensitive pixels are controlled to sense light; During the photosensitive process of the multiple rows of photosensitive pixels, the variable aperture is controlled to be in an open state during the global photosensitive period and to be in a closed state during the non-global photosensitive period. The global photosensitive period is the period during which all rows of photosensitive pixels are in a photosensitive state, and the non-global photosensitive period is the period during which at least one row of photosensitive pixels in the multiple rows of photosensitive units is in a photosensitive state and at least one row of photosensitive pixels is in a non-photosensitive state.

2. The method according to claim 1, characterized in that, The variable aperture includes a light inlet aperture; The control of the multi-row photosensitive pixels to perform light sensing includes: Based on the preset shooting parameters corresponding to the current shooting mode, determine the amount of exposure to be increased by the shooting device; Based on the amount of exposure to be increased, the target shooting parameters of the shooting device are determined. The target shooting parameters include the target exposure gain, the target shutter speed, and the target aperture value of the light-gathering hole when the variable aperture is in the target open state. Based on the target shooting parameters, control the multi-row photosensitive pixels to perform light sensing.

3. The method according to claim 2, characterized in that, The preset shooting parameters include the preset aperture value of the light inlet when the variable aperture is in a preset open state; The step of determining the amount of exposure to be increased by the shooting device based on the preset shooting parameters corresponding to the current shooting mode includes: The first exposure of the shooting device is determined according to the preset shooting parameters; The second exposure of the shooting device is determined based on the preset shooting parameters, the duration of the variable aperture switching from the closed state to the preset open state, and the duration from the end of light sensing by the first row of photosensitive pixels to the end of light sensing by the last row of photosensitive pixels in the multi-row photosensitive pixels. The amount of exposure to be increased is determined based on the first exposure and the second exposure.

4. The method according to claim 2 or 3, characterized in that, The preset shooting parameters also include preset exposure gain and preset shutter speed; Determining the target shooting parameters of the shooting device based on the amount of exposure to be increased includes: Obtain scene information of the scene to be photographed; Based on the amount of exposure to be increased and the scene information, the preset aperture value, the preset exposure gain, and / or the preset shutter speed are adjusted to obtain the target shooting parameters.

5. The method according to any one of claims 2 to 4, characterized in that, The duration of the multiple rows of photosensitive pixels in the photosensitive state is the same, and the time interval between the start of photosensitive states of two adjacent rows of photosensitive pixels is the first duration. The global photosensitive period includes the period from when the last row of photosensitive pixels in the multi-row photosensitive pixels begins to sense light to when the first row of photosensitive pixels in the multi-row photosensitive pixels ends to sense light. The non-global photosensitive period includes: the period from when the first row of photosensitive pixels begins to be photosensitive to when the last row of photosensitive pixels begins to be photosensitive, and the period from when the first row of photosensitive pixels ends to when the last row of photosensitive pixels ends to when the last row of photosensitive pixels ends to be photosensitive.

6. The method according to claim 5, characterized in that, During the light-sensing process of the multiple rows of photosensitive pixels, controlling the variable aperture to be in an open state during the global light-sensing period and controlling the variable aperture to be in a closed state during non-global light-sensing periods includes: When the last row of photosensitive pixels begins to sense light, the variable aperture is controlled to switch from a closed state to a target open state; During a second duration before the first row of photosensitive pixels finishes sensing light, the variable aperture is controlled to switch from the target open state to the target closed state, whereby the second duration is the duration during which the variable aperture switches from the closed state to the target open state.

7. The method according to any one of claims 2 to 4, characterized in that, The multiple rows of photosensitive pixels start sensing light at the same time, and the time interval between the end of sensing light for two adjacent rows of photosensitive pixels is the first duration. The global photosensitive period includes the period from when the multi-row photosensitive pixels begin to sense light to when the first row of photosensitive pixels in the multi-row photosensitive pixels ends to sense light. The non-global photosensitive period includes the period from the end of photosensitive period of the first row of photosensitive pixels to the end of photosensitive period of the last row of photosensitive pixels among the multiple rows of photosensitive pixels.

8. The method according to claim 7, characterized in that, During the light-sensing process of the multiple rows of photosensitive pixels, controlling the variable aperture to be in an open state during the global light-sensing period and controlling the variable aperture to be in a closed state during non-global light-sensing periods includes: When the multi-row photosensitive pixels begin to sense light, the variable aperture is controlled to switch to the target open state; During a second duration before the first row of photosensitive pixels finishes sensing light, the variable aperture is controlled to switch from the target open state to the target closed state, and the second duration is the duration during which the variable aperture switches from the target open state to the target closed state.

9. A shooting device, characterized in that, include: A variable aperture, an image sensor, and a processor, the processor being configured to run a computer program stored in a memory to control the variable aperture and the image sensor to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computer, implements the method as described in any one of claims 1 to 8.

Images