A photographing method and an electronic device

Abstract

Provided are a photographing method and an electronic device, applied to the technical field of image processing. The method comprises: starting a camera; collecting a plurality of first RAW data through a flicker sensor; determining a current brightness value based on the plurality of first RAW data; determining a target brightness value according to a first ambient illuminance and a first corresponding relationship; determining whether the difference between the current brightness value and the target brightness value is less than a difference threshold value; when the difference between the current brightness value and the target brightness value is not less than the difference threshold value, controlling the flicker sensor to sample a current shooting environment using a second gain and a second shutter speed to obtain a plurality of second RAW data; and the camera shooting using an adjusted exposure time, the adjusted exposure time being an integer multiple of a light source frequency, the light source frequency being determined according to the plurality of second RAW data. The photographing method of the application embodiment can accurately and stably identify the light source frequency, thereby effectively eliminating the banding phenomenon, and helping to improve the user's shooting experience.

Description

[0001] This application is a divisional application of Chinese patent application filed on September 23, 2022, with application number 202211167612.0 and title "A photographing method and electronic device". Technical Field

[0002] This application relates to the field of photography, and more specifically, to an image processing method and an electronic device. Background Technology

[0003] With the rapid development of smart devices, camera functionality has become an essential feature. Users' demands and experiences regarding camera (photos and / or videos) on smart devices are also constantly increasing.

[0004] When there are light sources of a certain frequency in the shooting environment, the image captured by the camera of the terminal device will exhibit banding. Banding refers to the presence of stripes of varying brightness in the captured image. Currently, banding can be eliminated by using a flicker sensor to identify the frequency of the light source in the shooting environment and adjusting the exposure time to an integer multiple of the identified light source frequency. However, this method has limitations; when the flicker sensor's data is abnormal, it cannot accurately and stably identify the light source frequency, thus failing to eliminate the banding phenomenon. Summary of the Invention

[0005] In view of this, this application provides a shooting method, electronic device, computer-readable storage medium, and computer program product that can accurately and stably identify the frequency of the light source, thereby effectively eliminating banding and helping to improve the user's shooting experience.

[0006] In a first aspect, a shooting method is provided, the method being applied to an electronic device, the electronic device including a camera and a flash sensor, the method comprising:

[0007] Turn on the camera;

[0008] The flash sensor acquires multiple first RAW data points, which are data of the current shooting scene acquired by the flash sensor using a first gain and a first shutter speed.

[0009] The current brightness value is determined based on the plurality of first RAW data;

[0010] The target brightness value is determined based on the first ambient illuminance and the first correspondence relationship. The first ambient illuminance is used to characterize the actual illuminance of the current shooting environment determined by the image sensor. The first correspondence relationship includes the brightness values ​​corresponding to different ambient illuminances.

[0011] Determine whether the difference between the current brightness value and the target brightness value is less than a difference threshold;

[0012] When the difference between the current brightness value and the target brightness value is not less than the difference threshold, the flicker sensor is controlled to sample the current shooting environment using a second gain and a second shutter speed to obtain multiple second RAW data. The second gain and the second shutter speed are determined based on the target brightness value. The second gain is different from the first gain, and the second shutter speed is different from the first shutter speed.

[0013] The camera uses an adjusted exposure time for shooting, which is an integer multiple of the light source frequency, and the light source frequency is determined based on the plurality of second RAW data.

[0014] In this embodiment, by judging the difference between the current brightness value and the target brightness value, if the difference is too large, the gain and shutter speed are adjusted to make the current brightness value reach the level of the target brightness value. The scintillator sensor collects data at an appropriate gain and shutter speed, thereby ensuring that the collected data is valid, so as to eliminate banding based on the collected data. Compared with the prior art, where the banding phenomenon cannot be eliminated due to abnormal data collected by the scintillator sensor in dark or overexposed environments, this embodiment enables the scintillator sensor to accurately and stably identify the light source frequency, thereby eliminating the banding phenomenon and improving the user's shooting experience.

[0015] Optionally, the adjusted exposure time is obtained in the following way:

[0016] The first frequency and the first amplitude are obtained by performing Fourier transform on the multiple second RAW data based on the anti-banding algorithm;

[0017] Based on the first frequency and the first amplitude, the current exposure time is adjusted to an integer multiple of the light source frequency, where the light source frequency is the first frequency.

[0018] In this embodiment, the second gain and the second shutter speed are determined based on the target brightness value.

[0019] In some possible implementations, the method further includes:

[0020] Determine the current exposure based on the current brightness value;

[0021] The target exposure is determined based on the current exposure and a first ratio, where the first ratio is the ratio between the target brightness value and the current brightness value.

[0022] When the difference between the current brightness value and the target brightness value is not less than the difference threshold, the second gain and the second shutter speed are found in the second correspondence using the target exposure. The second correspondence includes the gain and shutter speed corresponding to different exposures.

[0023] In other words, this application embodiment determines the target exposure by calculating the exposure amount, and uses the target exposure amount to match an appropriate gain and shutter speed. For example, the second correspondence is an exposure table.

[0024] In some possible implementations, controlling the scintillation sensor to sample the current shooting environment based on the second gain and the second shutter speed includes:

[0025] Adjust the first gain to the second gain;

[0026] After the first gain is adjusted to the second gain, the first shutter speed is adjusted to the second shutter speed.

[0027] In some possible implementations, determining the current luminance value based on the plurality of first RAW data includes:

[0028] The current brightness value is obtained by summing and averaging the multiple first RAW data.

[0029] In some possible implementations, the method further includes:

[0030] The first frequency and the first amplitude are obtained by performing Fourier transform on the multiple second RAW data based on the anti-striping algorithm;

[0031] Based on the first frequency and the first amplitude, the current exposure time is adjusted to an integer multiple of the light source frequency, where the light source frequency is the first frequency.

[0032] Therefore, after obtaining multiple second RAW data sets, an anti-striping algorithm can be used to process the multiple second RAW data sets in order to eliminate the banding phenomenon.

[0033] In some possible implementations, the method further includes:

[0034] When the difference between the current brightness value and the target brightness value is less than the difference threshold, the light source frequency is determined using the plurality of first RAW data.

[0035] In other words, when it is determined that the difference between the current brightness value and the target brightness value is less than the difference threshold, subsequent related operations can be performed using multiple first RAW data acquired at the first gain and the first shutter speed. For example, the light source frequency can be determined using the multiple first RAW data, and the exposure time can be adjusted to an integer multiple of the light source frequency to eliminate banding.

[0036] In some possible implementations, the method further includes, before controlling the scintillation sensor to sample the current shooting environment using a second gain and a second shutter speed:

[0037] Determine whether the second gain is the same as the first gain, and whether the second shutter speed is the same as the first shutter speed;

[0038] When the second gain is different from the first gain and the second shutter speed is different from the first shutter speed, the flash sensor is controlled to sample the current shooting environment using the second gain and the second shutter speed.

[0039] Therefore, after obtaining the second gain and the second shutter speed, it is possible to further determine whether the second gain is the same as the gain used last time, and whether the second shutter speed is the same as the first shutter speed, in order to decide whether to use the second gain and the second shutter speed for sampling.

[0040] In some possible implementations, the current shooting scene is a low-light scene, which includes scenes where the ambient light is below a first brightness threshold; or, the current shooting scene is an overexposed scene, which includes scenes where the data exceeds the upper limit of the bit width that the flicker sensor can support.

[0041] Optionally, activating the camera includes: detecting a user triggering a camera activation operation; and activating the camera in response to detecting the user triggering a camera activation operation.

[0042] Optionally, after activating the camera, a preview interface is displayed, and no banding is observed in the preview interface. When using the shooting method of this embodiment, no banding is observed in the preview interface.

[0043] Optionally, the method further includes: detecting a user's video recording operation; responding to the user's video recording operation, performing video recording, and displaying the recording screen on the interface, wherein there is no banding phenomenon in the recording screen.

[0044] In a second aspect, an electronic device is provided, including a unit for performing any of the methods in the first aspect. The electronic device may be a terminal or a chip within a terminal. The electronic device includes an input unit, a display unit, and a processing unit.

[0045] When the electronic device is a terminal, the processing unit may be a processor, the input unit may be a communication interface, and the display unit may be a graphics processing module and a screen; the terminal may also include a memory for storing computer program code, which, when the processor executes the computer program code stored in the memory, causes the terminal to perform any of the methods in the first aspect.

[0046] When the electronic device is a chip within a terminal, the processing unit can be a logic processing unit inside the chip, the input unit can be an output interface, pin, or circuit, and the display unit can be a graphics processing unit inside the chip. The chip may also include a memory, which can be memory within the chip (e.g., registers, caches, etc.) or memory located outside the chip (e.g., read-only memory, random access memory, etc.). The memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any of the methods in the first aspect.

[0047] Thirdly, a computer-readable storage medium is provided that stores computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the methods in the first aspect.

[0048] Fourthly, a computer program product is provided, the computer program product comprising: computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the methods in the first aspect. Attached Figure Description

[0049] Figure 1 This is an example diagram of an application scenario of an embodiment of this application;

[0050] Figure 2 This is an example diagram of another application scenario of the embodiments of this application;

[0051] Figure 3 This is a schematic diagram of a software architecture applied in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the structure of an electronic device applicable to this application;

[0053] Figure 5 This is a schematic flowchart illustrating one shooting method according to an embodiment of this application;

[0054] Figure 6 This is a schematic flowchart illustrating another shooting method according to an embodiment of this application;

[0055] Figure 7 This is a schematic diagram illustrating the effect of applying an embodiment of this application;

[0056] Figure 8 This is another example diagram of an interface using an embodiment of this application;

[0057] Figure 9 This is an example block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0058] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0059] In the embodiments of this application, unless otherwise stated, "multiple" can mean two or more.

[0060] The embodiments of this application are applicable to electronic devices with cameras, which can be smart terminals, mobile phones, smart screens, tablet computers, wearable electronic devices, in-vehicle electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), projectors, etc.

[0061] The electronic device in this application embodiment is equipped with an image acquisition device (such as a camera) and a flicker sensor. It should be understood that this application embodiment does not specifically limit the location of the flicker sensor. For example, the flicker sensor can be integrated into the camera. Alternatively, the flicker sensor can be set up independently.

[0062] It should also be understood that the embodiments described in this application are only exemplified by a flicker sensor, and the embodiments of this application are not limited thereto. In fact, the flicker sensor can also be replaced by other devices that have the function of a flicker sensor.

[0063] It should also be understood that the term "flicker sensor" is used to describe the device in this application, but the application is not limited to this. In fact, "flicker sensor" can also have other names, such as anti-flicker sensor.

[0064] This application does not limit the specific type of electronic device. The image processing method of this application is described below using a mobile phone as an example.

[0065] In some embodiments, the mobile phone includes a flash sensor and a camera. Key components of the camera include an optical lens and an image sensor.

[0066] This application applies to scenarios where banding occurs in the preview or captured image or video due to the presence of a certain frequency of light source during camera shooting. For example, when a certain frequency of light source exists in the shooting environment, the real-time preview displayed on the screen may exhibit uneven brightness when the phone's camera is turned on to take a photo or record video; in other words, scrolling or fixed horizontal stripes of light and dark may appear in the preview. Another example is when the phone's shutter button is triggered, banding may appear in the captured image or recorded video. Banding refers to the presence of stripes of varying brightness in the preview, captured image, or recorded video.

[0067] For example, when the camera button on a mobile phone is triggered, an image is captured using a line-by-line exposure method. If the exposure time is less than the duration of one strobe cycle, the brightness of the continuously captured preview image will be inconsistent across different stages within the strobe cycle corresponding to each exposure time, resulting in banding in the preview image displayed on the phone. Furthermore, if the exposure time is not an integer multiple of the strobe cycle duration, when the camera button is triggered and the phone captures an image of the target object, inconsistent line-by-line exposure will occur, resulting in banding in the captured image.

[0068] The following combination Figure 1 The following are examples of scenarios. Figure 1 This is an example diagram illustrating an application scenario of an embodiment of this application. For example... Figure 1 In the mobile phone interface shown in (1), the interface can display multiple applications: Application 1, Application 2, ..., Application 7 and the camera application. When the user clicks the camera application, the phone launches the camera.

[0069] After the camera is activated, the phone screen displays as follows: Figure 1 The interface shown in (2) is called the camera's shooting interface. This shooting interface may include a viewfinder 11, a zoom level of 1x (default), a shutter control, and a camera rotation control. The zoom level is used to adjust the zoom ratio used for shooting. The shutter control is used to start taking a picture. The camera rotation control can be used to switch cameras.

[0070] The viewfinder 11 is used to capture a preview image, which can be displayed in real time. As you can see, in... Figure 1 In the interface shown in (2), due to the presence of a certain frequency (e.g., 50Hz) lighting device, the preview image displayed in viewfinder 11 exhibits banding, i.e., a banding phenomenon. For example, as shown in (2), Figure 1 As shown in (2), the mobile phone's camera modes can include portrait, night scene, photo, video recording, and more. If the user selects video recording mode, the user can click on video recording to switch to video recording mode. For example, the mobile phone interface displays as follows: Figure 1 The interface shown in (3) is shown in the middle.

[0071] It is understood that the embodiments of this application do not specifically limit the subject of the photograph. The subject can be a person, an animal, a vehicle, a robot, etc. Figure 1 The example in this case is a person being photographed.

[0072] For example, Figure 1 The lighting equipment shown operates on alternating current. The operating frequency of this lighting equipment is 50Hz, meaning the duration of one flicker is 1 / 100th of a second. When taking photos with a mobile phone, the flicker of the fluorescent light can affect the image captured by the image sensor. Improper exposure time settings may cause banding in the preview image stream or the image captured by the phone.

[0073] For example, a phone generates a preview image stream and displays it on a screen to create a preview image, allowing the user to view it. If the phone's exposure time is less than the duration of one flash of a fluorescent light (e.g., capturing an image with an exposure time of 1 / 50s), the number of flashes included in the exposure of each image in the preview image stream will differ, causing banding in the preview image acquired by the phone.

[0074] like Figure 1 As shown in the interface in (3), when the user clicks the recording control 12, the phone starts recording video. For example, the interface displays as follows: Figure 1 The interface shown in (4). Figure 1 As shown in (4), in video recording mode, taking the recording time as the 2nd second as an example, there is a banding phenomenon in the recorded picture due to the presence of the lighting equipment.

[0075] Optionally, Figure 1The middle (4) includes a photo capture control 13, a video stop control 14, a video pause control 15, and a camera rotation control. If the user clicks control 13, they can manually capture a photo during the video recording. If the user clicks the video stop control 14, the video recording will be stopped. If the user clicks the video pause control 15, the video recording will be paused. The function of the camera rotation control is similar to that described above, and will not be repeated here.

[0076] For example, when the camera button on a mobile phone is triggered and the phone generates a captured image, the image sensor acquires the image by exposure line by line. Due to the flickering of fluorescent lights, if the fluorescent light flickers during the process of the image sensor acquiring the image line by line, the exposure of different lines in a frame will be different, resulting in a phenomenon of bright and dark stripes on the image generated by the mobile phone, namely the "banding" phenomenon. Figure 2 The image shown is a schematic diagram illustrating the banding phenomenon. For example... Figure 2 As shown, in Figure 1 Images captured in this scene exhibit banded stripes. This is because the brightness changes during exposure, resulting in bright and dark stripes in the image.

[0077] Optionally, Figure 2 The interface shown may also include controls for sharing, saving, editing, deleting, and more. When a user clicks the share control, they can share the image. When a user clicks the save control, they can save the image. When a user clicks the edit control, they can edit the image. When a user clicks the more control, they can view other additional functions.

[0078] Understandable. Figure 2 The interface shown is merely an exemplary description, and the embodiments of this application are not limited thereto.

[0079] It should be understood that Figure 1 The photographing scenario shown is merely illustrative of one application scenario of this application and does not constitute a limitation on the embodiments of this application. In fact, the embodiments of this application can also be applied to other scenarios using cameras, such as video recording scenarios, video call scenarios, and live video streaming scenarios.

[0080] It should also be understood that Figure 1 The description uses an example of a lighting device as the light source, but the embodiments of this application are not limited to this. For example, the light source device can be a display or other frequency-based device.

[0081] It should also be understood that Figure 1 Figure (4) shows a schematic diagram of an interface for a user to record video in portrait mode on a mobile phone, but this application is not limited to this. For example, a user can take pictures in landscape mode on a mobile phone.

[0082] It should also be understood that Figure 1 or Figure 2 The various controls shown or their positions on the interface are merely examples, and the embodiments of this application are not limited thereto.

[0083] Currently, flicker sensors can identify the frequency of light sources in the shooting environment and adjust the exposure time to an integer multiple of the identified light source frequency to eliminate banding. However, when the flicker sensor collects abnormal data, it cannot accurately and stably identify the light source frequency, thus failing to eliminate banding. This application does not specifically limit the scenarios where the flicker sensor collects abnormal data. For example, in a dark environment, the data collected by the flicker sensor may be too small. Or, in an overexposed environment, the data collected by the flicker sensor may be clamped. Data clamping can be understood as the inability to collect data when the bit width exceeds the upper limit that the flicker sensor can support.

[0084] For example, in situations where the shooting environment is dark or overexposed, the data collected by the scintillation sensor may be abnormal (e.g., the data is too small or the data is clamped), making it impossible to accurately and stably identify the light source frequency, and thus unable to eliminate banding. The shooting method of this application embodiment can be applied to shooting scenarios that are dark or overexposed.

[0085] This example illustrates the situation where data is clamped. Taking a scintillation sensor that can support a maximum of 8 bits as an example, the scintillation sensor can collect up to 256 RAW data points. These 256 RAW data points can correspond to 0-255 respectively. Data clamping means that the scintillation sensor cannot collect RAW data beyond 255, and therefore cannot accurately identify the light source frequency.

[0086] Optionally, the current shooting scene is a low-light scene, which includes scenes where the ambient illuminance is lower than a first brightness threshold. It should be understood that the above description only uses an ambient illuminance lower than the first brightness threshold as an example, and the embodiments of this application are not limited thereto.

[0087] Alternatively, the current shooting scene may be an overexposed scene, which includes situations where the data exceeds the upper limit of the bit width that the flicker sensor can support.

[0088] In view of this, embodiments of this application provide a shooting method and an electronic device. By adjusting the gain and shutter speed of the scintillation sensor, the scintillation sensor can accurately and stably identify the light source frequency in both low-light and overexposed environments, thereby eliminating banding, improving image quality, and enhancing the user's shooting experience.

[0089] To facilitate understanding, before introducing the shooting method of the embodiments of this application, some terms involved in the embodiments of this application will be explained first.

[0090] 1. Shutter speed is used to characterize the time required to capture a single pixel, or in other words, how long it takes to capture a single pixel. For consistency, shutter speed can also be referred to as exposure time. For example, when the sampling frequency of the scintillation sensor is 1kHz, the shutter speed is 1ms.

[0091] 2. Exposure value (EV) can be determined by both exposure time and gain. For example, exposure value satisfies the formula: EV = Gain * Exposure Time. Here, EV represents exposure value, and Gain represents gain.

[0092] 3. Ambient illuminance refers to the intensity of light in the shooting environment where the user is located during the shooting process. The value of ambient illuminance can be represented by the following indicators: lighting value (LV), lux, or luxindex, etc. In the embodiments of this application, the ambient illuminance is calculated by an image sensor.

[0093] LV is used to estimate ambient light, and its specific calculation formula is as follows:

[0094]

[0095] Where Exposure is the exposure time, Aperture is the aperture size, Iso is the ISO, and Luma is the average value of Y in the XYZ color space.

[0096] It should be understood that the LV mentioned above is calculated by the image sensor, and the variables in the above formula for calculating LV also come from the image sensor.

[0097] For ambient illuminance, it can be divided into low illuminance scenes (or dark light environments) and high illuminance scenes (or bright light environments) based on a first luminance threshold.

[0098] For example, if the ambient illuminance is greater than or equal to the first brightness threshold, it is a high-illuminance scene; if the ambient illuminance is less than the first brightness threshold, it is a low-illuminance scene.

[0099] It should be understood that the scenario where the ambient illuminance value equals the first luminance threshold is classified as a high-illuminance scene, but the embodiments of this application are not limited to this. For example, the scenario where the ambient illuminance equals the first luminance threshold can also be classified as a low-illuminance scene. The following description uses high-illuminance scenes and low-illuminance scenes.

[0100] As one possible implementation, ambient illuminance is represented by LV, and correspondingly, the first luminance threshold is the first LV value.

[0101] As one possible implementation, ambient illuminance is represented by the luxindex, and correspondingly, the first luminance threshold is the luxindex value. A larger luxindex value indicates lower ambient illuminance, while a smaller luxindex value indicates higher ambient illuminance.

[0102] 4. Overexposure scenes can be determined based on the upper limit of the bit width that the scintillation sensor can support. For example, when the data exceeds the upper limit of the bit width that the scintillation sensor can support, it can be considered an overexposure scene.

[0103] Alternatively, the dynamic range of an overexposed scene can be determined based on the percentage of overexposed pixels and the percentage of underexposed pixels.

[0104] 5. The flicker sensor automatic exposure (flicker sensorAE) module is used to automatically adjust the gain of the flicker sensor and the shutter speed in low-light or overexposed shooting environments.

[0105] The following combination Figure 3 and Figure 4 This application describes the software system and hardware architecture used in its embodiments.

[0106] Figure 3 This is a schematic diagram of the architecture (including software system and some hardware) used in the embodiments of this application. Figure 3 As shown, the application architecture is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), the driver layer, and the hardware layer.

[0107] like Figure 3 As shown, the application layer includes the camera and the gallery.

[0108] Understandable. Figure 3 The examples shown are only a portion of the applications; in fact, the application layer can include other applications as well, and this application does not limit this. For example, the application layer may also include applications such as messaging, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, and Alipay.

[0109] like Figure 3As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction layer and a camera algorithm library (not shown in the figure). The camera hardware abstraction layer includes multiple camera devices. The hardware abstraction layer also includes a sensor HAL, a flicker sensor automatic exposure (flicker sensor AE) module, an anti-banding algorithm module, a non-camera sensor (NCS) path module, an automatic exposure control (AEC) interface, and an AEC core.

[0110] In some embodiments, the flicker sensor AE module is used to automatically adjust the gain and shutter speed (or exposure time) of the flicker sensor when the ambient lighting conditions change.

[0111] The driver layer is used to drive hardware resources. The driver layer can include multiple driver modules. For example... Figure 3 As shown, the driver layer includes camera device drivers, digital signal processor drivers, and graphics processor drivers, etc.

[0112] The hardware layer includes sensors, image signal processors, digital signal processors, and graphics processors. The sensors include multiple sensors, a Time-of-Flight (TOF) camera, a multispectral sensor, and a scintillation sensor.

[0113] For example, a user can tap the camera application. When the user taps the camera to take a picture, the shooting command is sent to the camera hardware abstraction layer (HAL) via the camera access interface. The HAL then calls the camera device driver. The HAL sends configuration parameters to the camera device driver. The camera device driver then sends the configuration parameters sent by the HAL to the hardware layer.

[0114] The HAL sensor is used to send the gain and shutter speed (such as the adjusted gain and shutter speed obtained from the flicker sensor AE module) to the flicker sensor.

[0115] For example, the sensor HAL sends 256 acquired RAW data points to the anti-stripping algorithm module. The anti-stripping algorithm module processes the 256 RAW data points and returns the processed RAW data to the sensor HAL. The sensor HAL then sends the data obtained from the anti-stripping algorithm module to the NCS module. The NCS module serves as the data transmission path. The NCS module can send data received from the sensor HAL to the AEC interface. The AEC interface sends data to the flicker sensor AE module. Simultaneously, the AEC core also sends the ambient illumination (first ambient illumination) acquired by the image sensor to the flicker sensor AE module.

[0116] In some embodiments, the flicker sensor AE module determines the target brightness based on a first ambient illuminance and determines whether the difference between the current brightness value and the target brightness value is less than a difference threshold. When the difference between the current brightness value and the target brightness value is not less than the difference threshold, the module determines the adjusted gain and shutter speed and sends the adjusted gain and shutter speed to the sensor HAL to control the flicker sensor to take effect, so that the RAW data finally detected and output by the flicker sensor reaches the level of the target brightness value.

[0117] This application does not specifically limit the entity performing the "determining the current brightness value based on the plurality of first RAW data" in its embodiments. In other words, this application does not limit the method by which the flicker sensor AE module obtains the current brightness value.

[0118] Optionally, the anti-stripping algorithm module can determine the current brightness value based on the multiple first RAW data sets and return the current brightness value to the sensor HAL, which is then transmitted to the ficker sensor AE module. For example, after obtaining 256 RAW data sets, the anti-stripping algorithm module can sum and average the 256 RAW data sets and return the average value to the sensor HAL. That is, the ficker sensor AE module obtains the current brightness value without needing to calculate it itself.

[0119] Alternatively, the ficker sensor AE module can determine the current brightness value based on the multiple first RAW data sets. For example, after obtaining 256 RAW data sets, the anti-stripping algorithm module returns the 256 RAW data sets to the sensor HAL, which is then transmitted to the ficker sensor AE module. The ficker sensor AE module performs summation and average calculation on the 256 RAW data sets. That is, the ficker sensor AE module obtains 256 RAW data sets, and further calculations are needed to obtain the current brightness value.

[0120] Additionally, the image output from the image signal processor can be sent to the camera device driver. The camera device driver can then send the image output from the image signal processor to the camera hardware abstraction layer. The camera hardware abstraction layer can then send the image to the camera access interface. The camera access interface can then send the image returned by the camera hardware abstraction layer to the camera.

[0121] The software system used in the embodiments of this application has been described in detail above. The following section, in conjunction with... Figure 4 Describe the hardware system of electronic device 1000.

[0122] Figure 4 A schematic diagram of the structure of an electronic device 1000 applicable to this application is shown.

[0123] Electronic device 1000 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, a flash sensor 180N, etc.

[0124] It should be noted that, Figure 4 The structure shown does not constitute a specific limitation on the electronic device 1000. In other embodiments of this application, the electronic device 1000 may include... Figure 4 The components shown may include more or fewer components, or the electronic device 1000 may include... Figure 4 The components shown may be a combination of certain components, or the electronic device 1000 may include... Figure 4 The components shown are sub-components of certain components. For example, Figure 4 The proximity sensor 180G shown is optional. Figure 4 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0125] Processor 110 may include one or more processing units. For example, processor 110 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). These different processing units may be independent devices or integrated devices.

[0126] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0127] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0128] In some embodiments, processor 110 may include one or more interfaces. For example, processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0129] In some embodiments, the processor 110 is configured to: initiate camera startup; also configured to: invoke a flash sensor to acquire multiple first RAW data, wherein the multiple first RAW data are data of the current shooting scene acquired by the flash sensor using a first gain and a first shutter speed; further configured to: determine a current brightness value based on the multiple first RAW data; further configured to: determine a target brightness value according to a first ambient illuminance and a first correspondence, wherein the first ambient illuminance characterizes the actual illuminance of the current shooting environment determined by the image sensor, and the first correspondence includes brightness values ​​corresponding to different ambient illuminances; and further configured to: determine the difference between the current brightness value and the target brightness value. The system is further configured to: 1) determine if the difference between the current brightness value and the target brightness value is less than the difference threshold; 2) control the flicker sensor to sample the current shooting environment using a second gain and a second shutter speed to obtain multiple second RAW data when the difference between the current brightness value and the target brightness value is not less than the difference threshold, wherein the second gain and the second shutter speed are determined based on the target brightness value, the second gain is different from the first gain, and the second shutter speed is different from the first shutter speed; 3) invoke the camera to shoot using an adjusted exposure time, wherein the adjusted exposure time is an integer multiple of the light source frequency, and the light source frequency is determined based on the multiple second RAW data.

[0130] Figure 4 The connection relationships between the modules shown are merely illustrative and do not constitute a limitation on the connection relationships between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of various connection methods described in the above embodiments.

[0131] The charging management module 140 receives power from a charger, which can be either a wireless or wired charger. In some wired charging embodiments, the charging management module 140 receives current from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives electromagnetic waves (current path shown as dashed lines) via the wireless charging coil of the electronic device 1000. While charging the battery 142, the charging management module 140 can also supply power to the electronic device 1000 via the power management module 141.

[0132] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (e.g., leakage current, impedance). Optionally, the power management module 141 can be located within the processor 110, or the power management module 141 and the charging management module 140 can be located in the same device.

[0133] The wireless communication function of electronic device 1000 can be realized through devices such as antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0134] Electronic device 1000 can implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0135] Display screen 194 can be used to display images or videos. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 1000 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0136] Electronic device 1000 can achieve shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0137] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0138] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard red-green-blue (RGB), YUV, or other image signal format. In some embodiments, the electronic device 1000 may include one or N cameras 193, where N is a positive integer greater than 1.

[0139] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 1000 is selecting a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0140] Video codecs are used to compress or decompress digital video. Electronic device 1000 may support one or more video codecs. Thus, electronic device 1000 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

[0141] An NPU (Neural Processing Unit) is a processor that borrows from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, to rapidly process input information and continuously learn. NPUs can enable intelligent cognitive functions in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0142] Electronic device 1000 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0143] The distance sensor 180F is used to measure distance. The electronic device 1000 can measure distance via infrared or laser. In some embodiments, such as in a shooting scenario, the electronic device 1000 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0144] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 1000 can adaptively adjust the brightness of its display screen 194 based on the sensed ambient light level. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking photos. The ambient light sensor 180L can also work in conjunction with the proximity sensor 180G to detect whether the electronic device 1000 is in a pocket, preventing accidental touches.

[0145] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 1000 can use the collected fingerprint characteristics to achieve functions such as unlocking, accessing application locks, taking photos, and answering calls.

[0146] Touch sensor 180K, also known as a touch device, can be disposed on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a touch display. Touch sensor 180K is used to detect touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be disposed on the surface of electronic device 1000, and in a different location from display screen 194.

[0147] The 180N flicker sensor is used to identify the frequency of ambient light.

[0148] In some embodiments, the scintillation sensor 180N is used to acquire multiple first RAW data.

[0149] In some embodiments, the flicker sensor 180N is also used to acquire multiple second RAW data based on the adjusted gain (e.g., a second gain) and adjusted shutter speed (e.g., a second shutter speed) issued by the flicker sensor AE module.

[0150] In some embodiments, the scintillation sensor 180N is also used to eliminate banding based on a plurality of second RAW data.

[0151] Button 190 includes a power button and volume buttons. Button 190 can be a mechanical button or a touch button. Electronic device 1000 can receive button input signals and implement functions related to the button input signals.

[0152] It is understood that the shooting method of this application embodiment can be applied to Figure 4 The specific implementation steps of the electronic device shown can be referred to the description of the method embodiments above, and will not be repeated here.

[0153] The following combination Figures 5 to 8 The shooting method of the embodiments of this application is described.

[0154] Figure 5 This is a schematic flowchart illustrating the shooting method according to an embodiment of this application. Figure 5 As shown, the method 500 includes:

[0155] S510, turn on the camera.

[0156] Optionally, activating the camera includes: detecting a user triggering a camera activation operation; and activating the camera in response to detecting the user triggering a camera activation operation.

[0157] For example, such as Figure 1 As shown in (1), users can start the camera by clicking the camera application icon.

[0158] This application does not specifically limit how the camera is triggered. The above-described methods for starting the camera are merely illustrative. Users can also start the camera in other ways, such as by voice control, or by using other apps that support shooting functions (such as WeChat).

[0159] S520 acquires multiple first RAW data through a flicker sensor, wherein the multiple first RAW data are data of the current shooting scene acquired by the flicker sensor using a first gain and a first shutter speed.

[0160] RAW data refers to all the grayscale data of an image recorded by the image sensor.

[0161] For example, the multiple first RAW data sets consist of 256 RAW data sets. The scintillation sensor can read these 256 RAW values ​​using a first-in-first-out (FIFO) mechanism. FIFO is a data transmission and reading method. It should be understood that this description uses the FIFO mechanism as an example, and the embodiments of this application are not limited to this. In fact, other data reading methods can also be used to acquire RAW data.

[0162] It should be understood that the embodiments of this application do not specifically limit the number of multiple first RAW data, which may depend on the device capabilities.

[0163] It should also be understood that this description uses 256 RAW data points as an example, and the embodiments of this application are not limited to this. In fact, with the development of devices, scintillation sensors can collect other numbers of RAW data (e.g., larger numbers than 256).

[0164] The first gain is the gain currently used by the scintillation sensor. The first shutter speed is the shutter speed currently used by the scintillation sensor.

[0165] S530 determines the current brightness value based on multiple first RAW data.

[0166] The current brightness value refers to the brightness value calculated from multiple first RAW data collected by the flicker sensor. For example, the current brightness value is represented as cur_luma.

[0167] The embodiments of this application do not specifically limit the method for calculating the current brightness value.

[0168] Optionally, as a possible implementation, determining the current luminance value based on the plurality of first RAW data includes:

[0169] The summation and average of the multiple first RAW data are performed to obtain the current brightness value (cur_luma).

[0170] For example, the sum of 256 RAW data points is calculated, then divided by 256, and the average value is used as the current brightness value. "Using the average value as the current brightness value" can be understood as using the brightness level of the obtained RAW data as the current brightness value.

[0171] S540, determine the target brightness value based on the first ambient illuminance and the first correspondence relationship, wherein the first ambient illuminance is used to characterize the actual illuminance of the current shooting environment determined by the image sensor, and the first correspondence relationship includes the brightness values ​​corresponding to different ambient illuminances.

[0172] In this embodiment, a target brightness value is set, and then the gain and shutter speed are determined based on the target brightness value. By adjusting the gain and shutter speed, the current brightness value is made closer to the target brightness value.

[0173] The first ambient illuminance is used to characterize the actual illuminance (or objective ambient illuminance) of the current shooting environment calculated by the image sensor.

[0174] For example, the first ambient illuminance is represented by a luminance value LV. A suitable RAW value can be matched in the first correspondence as the target luminance value (e.g., denoted as target_luma). "Using a suitable RAW value as the target luminance value" can be understood as: using the luminance level of a suitable RAW value as the target luminance value.

[0175] The first correspondence is used to characterize the relationship between ambient illuminance and the corresponding luminance value. For example, the first correspondence includes multiple LVs and the luminance value corresponding to each LV.

[0176] For example, the first correspondence can be Table 1 below:

[0177] Table 1

[0178] LV Target brightness value LV1 target_luma1 LV2 target_luma2 LV3 target_luma3

[0179] Table 1 above shows the different target brightness values ​​corresponding to different LV values. The corresponding target brightness value can be found in Table 1 based on the LV value.

[0180] It should be understood that Table 1 above is merely an exemplary description, and the embodiments of this application are not limited thereto. For example, LV in the first row can also be replaced with LV intervals, with each LV interval corresponding to a different target brightness value (target_luma).

[0181] After obtaining the current brightness value and the target brightness value, the difference between the two can be determined, and based on the determination result, it can be decided whether to adjust the gain and shutter speed.

[0182] S550, determine whether the difference between the current brightness value and the target brightness value is less than the difference threshold.

[0183] Alternatively, step S550 can be replaced by determining whether the current brightness value is the same as the target brightness value.

[0184] Optionally, a difference threshold is introduced here to evaluate whether the current brightness value differs significantly from the target brightness value. This application embodiment does not specifically limit the value of the difference threshold; the difference threshold can tend to be infinitely small.

[0185] For example, if the absolute value of the difference between the two is less than the difference threshold, then the difference between the current brightness value and the target brightness value is considered to be small, and the two are close; if the absolute value of the difference between the two is greater than or equal to the difference threshold, then the difference between the current brightness value and the target brightness value is considered to be large.

[0186] It should be understood that this explanation uses the difference threshold as an example only, and the embodiments of this application are not limited to this. In fact, other methods can also be used to evaluate whether the current brightness value and the target brightness value differ significantly.

[0187] S560, when the difference between the current brightness value and the target brightness value is not less than (greater than or equal to) a difference threshold, the flicker sensor is controlled to sample the current shooting environment using a second gain and a second shutter speed to obtain multiple second RAW data. The second gain and the second shutter speed are determined based on the target brightness value; the second gain is different from the first gain, and the second shutter speed is different from the first shutter speed.

[0188] It should be noted that the term "multiple second RAW data" is introduced here only to distinguish it from the "multiple first RAW data" mentioned earlier, as the two were collected under different sampling conditions.

[0189] For example, the multiple second RAW data are 256 RAW data acquired at the second gain and the second shutter speed.

[0190] For example, the absolute value of the difference between the current brightness value and the target brightness value is represented as abs(target_luma-cur_luma). If abs(target_luma-cur_luma) is greater than or equal to the difference threshold, the gain of the flicker sensor is adjusted to the second gain, and the shutter speed is adjusted to the second shutter speed. That is, the flicker sensor is controlled to sample using the second gain and the second shutter speed. abs represents the absolute value operation.

[0191] In this embodiment, by employing a second gain and a second shutter speed for sampling, abnormal flicker sensor data can be avoided in scenes with excessively dark or overexposed environments. When the flicker sensor data is normal, the acquired second RAW data can be used to accurately identify the light source frequency, thereby effectively eliminating the banding phenomenon.

[0192] Furthermore, adjusting the shutter speed affects the sampling frequency of the scintillation sensor. If the sampling frequency of the scintillation sensor is too low, the frequency range (obtained by performing a Fourier transform on multiple RAW data collected by the scintillation sensor) will decrease, leading to reduced sensitivity. Therefore, to avoid the aforementioned effects caused by adjusting the shutter speed, the embodiments of this application adjust the gain and shutter speed sequentially: the gain is adjusted first, and then the shutter speed is adjusted after the gain adjustment.

[0193] Optionally, as an embodiment, S560 includes:

[0194] Adjust the first gain to the second gain; after adjusting the first gain to the second gain, adjust the first shutter speed to the second shutter speed.

[0195] For example, the gain can be adjusted to the maximum value (e.g., 512 times) to control the scintillator sensor to acquire data at the maximum gain, and then the shutter speed can be adjusted.

[0196] It should be understood that this description is only based on the example of adjusting to the maximum value, and is not intended to limit the embodiments of this application to this.

[0197] This application embodiment can determine the second gain and the second shutter speed based on the target brightness value; the specific determination method will be described later. Figure 6 The description continues. After determining the second gain and the second shutter speed, it is also possible to determine whether the second gain is the same as the previously used gain, and whether the second shutter speed is the same as the previously used shutter speed. For example, the previously used gain was the first gain. Or, for example, the previously used shutter speed could have been the first shutter speed.

[0198] Optionally, before step S560, the method 500 further includes:

[0199] Determine whether the second gain is the same as the first gain, and whether the second shutter speed is the same as the first shutter speed;

[0200] When the second gain is different from the first gain and the second shutter speed is different from the first shutter speed, the flash sensor is controlled to sample the current shooting environment using the second gain and the second shutter speed.

[0201] For example, if it is determined that the second gain is different from the first gain and the second shutter speed is different from the first shutter speed, then the gain of the flash sensor can be set to the second gain and the shutter speed of the flash sensor can be set to the second shutter speed.

[0202] It should be understood that "determining whether the second gain is the same as the first gain" and "determining whether the second shutter speed is the same as the first shutter speed" can be executed separately or simultaneously. This application embodiment does not specifically limit this.

[0203] Separate execution means that the execution of the two functions is not coupled. If it is determined that the second gain is different from the first gain, then the second gain is sent to the sensor HAL to take effect. If it is determined that the second shutter speed is different from the first shutter speed, then the second shutter speed is sent to the sensor HAL to take effect.

[0204] For example, when it is determined that the second gain is the same as the first gain and the second shutter speed is the same as the first shutter speed, then S580 can be executed, that is, the light source frequency can be determined using the plurality of first RAW data.

[0205] S570, the camera takes pictures using an adjusted exposure time, the adjusted exposure time being an integer multiple of the light source frequency, the light source frequency being determined based on the plurality of second RAW data.

[0206] Optionally, the adjusted exposure time is obtained in the following way:

[0207] The first frequency and the first amplitude are obtained by performing Fourier transform on the multiple second RAW data based on the anti-banding algorithm;

[0208] Based on the first frequency and the first amplitude, the current exposure time is adjusted to an integer multiple of the light source frequency, where the light source frequency is the first frequency.

[0209] For example, using Figure 3 The anti-banding algorithm module shown in the figure performs a fast Fourier transform (FFT) on 256 second RAW data points (these 256 data points are acquired by the scintillator at the second gain and the second shutter speed) to obtain a first frequency and a first amplitude. Then, it uses the exposure time adopted by the first frequency value and the first amplitude to adjust the current exposure time to an integer multiple of the light source frequency, so as to eliminate the banding phenomenon.

[0210] It should be understood that the above description is only an example of performing FFT transformation on multiple second RAW data, and the embodiments of this application are not limited thereto. In fact, other processing can also be performed on multiple second RAW data.

[0211] Anti-banding algorithms can identify the frequency of a light source using data collected by a flicker sensor, thereby eliminating banding. It should also be understood that the above description uses an anti-banding algorithm as an example, and the embodiments of this application are not limited thereto.

[0212] In this embodiment, by judging the difference between the current brightness value and the target brightness value, if the difference is too large, the gain and shutter speed are adjusted to make the current brightness value reach the level of the target brightness value. The flicker sensor collects data at an appropriate gain and shutter speed, thereby ensuring that the collected data is valid, so as to eliminate banding based on the collected data. Compared with the prior art, where the flicker sensor cannot eliminate banding in dark or overexposed shooting environments due to abnormal data collected by the flicker sensor, this embodiment enables the flicker sensor to accurately and stably identify the light source frequency, thereby eliminating banding and improving the user's shooting experience.

[0213] Optionally, the method 500 further includes: S580, when the difference between the current brightness value and the target brightness value is less than the difference threshold, or when it is determined that the second gain is the same as the first gain and the second shutter speed is the same as the first shutter speed, the light source frequency is determined using the plurality of first RAW data.

[0214] For example, determining the light source frequency using the plurality of first RAW data includes: performing a Fourier transform on the plurality of first RAW data based on an anti-banding algorithm to obtain a second frequency and a second amplitude;

[0215] Based on the second frequency and the second amplitude, the current exposure time is adjusted to an integer multiple of the light source frequency, where the light source frequency is the second frequency.

[0216] It should be understood that the second frequency and second amplitude are introduced here only to distinguish them from the first frequency and first amplitude mentioned earlier.

[0217] For example, when the difference between the current brightness value and the target brightness value is small, there is no need to adjust the gain and shutter speed. In this case, banding can be eliminated by using multiple first RAW data acquired at the first gain and first shutter speed. Specifically, an FFT transformation is performed on the multiple first RAW data to obtain a second frequency and a second amplitude. Using the second frequency and second amplitude, the current exposure time is adjusted to an integer multiple of the light source frequency to eliminate banding.

[0218] In this embodiment, the second gain and the second shutter speed are determined based on the target brightness value. This embodiment does not limit the specific method for determining the second gain and the second shutter speed. The following... Figure 6 Describe it.

[0219] Figure 6 A flowchart of a method 600 for determining a second gain and a second shutter speed according to an embodiment of this application is shown. It should be understood that... Figure 6 Method 600 in the middle can be used with Figure 5 The methods described in section 500 can be implemented in combination or independently; no specific limitations are imposed on this. It should also be understood that... Figure 6 For explanations of some terms or concepts involved, please refer to [link / reference]. Figure 5 The description in the text. For example... Figure 6 As shown, the method 600 includes:

[0220] S610 determines the current exposure based on the current brightness value.

[0221] The method for determining the current brightness value has been... Figure 5The details of S530 are omitted here for the sake of brevity.

[0222] For example, once the current brightness value is obtained, the current exposure can be calculated based on the current gain and shutter speed.

[0223] For example, current exposure = gain * shutter speed.

[0224] S620, determine the target exposure based on the current exposure and a first ratio, wherein the first ratio is the ratio between the target brightness value and the current brightness value.

[0225] Target exposure can be understood as the expected exposure. The purpose of introducing target exposure is to find the corresponding gain and shutter using the exposure table.

[0226] For example, the target exposure can be obtained by multiplying the current exposure by the first ratio.

[0227] S630, when the difference between the current brightness value and the target brightness value is not less than the difference threshold, the second gain and the second shutter speed are found in the second correspondence using the target exposure, the second correspondence including the gain and shutter speed corresponding to different exposures.

[0228] For example, the second shutter speed can be 1ms, 2ms, or 4ms, etc.

[0229] It should be understood that the value of the second shutter speed described here is merely an exemplary description, and the embodiments of this application are not limited thereto.

[0230] For example, the second gain can be 1x, 2x, ... or 512x.

[0231] It should be understood that the value of the second gain described here is merely an exemplary description, and the embodiments of this application are not limited thereto.

[0232] In this embodiment of the application, the corresponding gain and shutter speed can be found in the second correspondence based on the target exposure.

[0233] Optionally, the second correspondence can be in the form of an exposure table. For example, the second correspondence is shown in Table 2 below:

[0234] Table 2

[0235] index Exposure Gain shutter Index1 Exposure 1 Gain1 Shutter1 Index2 Exposure 2 Gain2 Shutter2 Index3 Exposure 3 Gain3 Shutter3

[0236] In Table 2 above, after calculating the target exposure, you can look up the corresponding index in Table 2 based on the target exposure, and then look up the corresponding gain and shutter based on the corresponding index.

[0237] It should be understood that the above description uses Table 2 as an example only, and the embodiments of this application are not limited thereto. For example, Table 2 may not include an index column, and after calculating the target exposure, the corresponding gain and shutter can be directly queried from Table 2.

[0238] In this embodiment, the trend of the target brightness value is consistent with the trend of the target brightness value of the image sensor changing with ambient illuminance (e.g., LV) in order to support the calculation of fluctuation depth.

[0239] Fluctuation depth typically refers to the difference in brightness between "bright" and "dark" flickering. For example, the greater the difference between the highest and lowest brightness, the greater the fluctuation depth. For instance, fluctuation depth can be understood as amplitude (such as the second amplitude calculated from multiple second RAW data points mentioned earlier).

[0240] The anti-banding algorithm calculates the fluctuation depth of the current frequency (such as the second amplitude), which can provide fluctuation depth for other algorithms to use in subsequent algorithms.

[0241] For example, other algorithms could refer to motion-based de-exposure algorithms. When the depth of fluctuation is small, motion-based de-exposure algorithms can reduce the exposure time to below an integer multiple of the period of the light source frequency.

[0242] For ease of understanding, the following is combined with Figure 7 and Figure 8 The interface example is described below. Figure 7 This is a schematic diagram illustrating the effect of an embodiment of this application. For example... Figure 7 As shown, compared to Figure 2 The images shown in the figure do not exhibit banding after being captured using the shooting method of the embodiments of this application.

[0243] It should be understood that Figure 7 The descriptions provided are merely illustrative, and the embodiments of this application are not limited thereto. In fact, the shooting method of the embodiments of this application can effectively eliminate banding for preview images or recorded videos.

[0244] Optionally, after activating the camera, a preview interface is displayed, and no banding is observed in the preview interface. When using the shooting method of this embodiment, no banding is observed in the preview interface.

[0245] Optionally, the 500 method further includes: detecting a user's video recording operation; responding to the user's video recording operation, performing video recording, and displaying the recording screen (or recording interface, for example) on the interface. Figure 8The interface shown in (4) indicates that there is no banding phenomenon in the recorded screen.

[0246] Figure 8 This is another schematic diagram illustrating the effect of an embodiment of this application. It should be understood that... Figure 8 The scene shown in Figure 1 The scenes shown are the same (all include lighting equipment operating at a frequency of 50Hz), and the relevant controls or interface descriptions can be found in [reference needed]. Figure 1 The description of the location is omitted here for brevity. For example, Figure 8 The description of the interface shown in (1) can be found in [reference]. Figure 1 The interface description shown in (1) is as follows. For example, Figure 8 Middle (2) to Figure 8 The descriptions of the controls included in the interface shown in (4) can be found in [reference 1]. Figure 1 Middle (2) to Figure 1 The description of the controls included in the interface shown in (4) is shown in the figure.

[0247] like Figure 8 As shown in (1), when the user clicks the camera application, the phone launches the camera. After the camera is running, the phone interface displays as follows: Figure 8 The interface shown in (2). Figure 8 As shown in (2), when the user clicks to record video 10, the recording mode is switched. For example, the mobile phone interface displays as shown in (2). Figure 8 The interface shown in (3). Figure 8 As shown in the interface in (3), when the user clicks the recording control 12, the phone starts recording video. For example, the interface displays as follows: Figure 8 The interface shown in (4) is shown in the middle.

[0248] Compared to Figure 1 The interfaces shown in the figure, after adopting the shooting method of the embodiments of this application, Figure 8 The preview interface shown in (2) and Figure 8 The preview interface shown in (3) and Figure 8 The video interface shown in (4) does not exhibit any banding phenomenon.

[0249] It should be understood that Figure 8 The interface described herein is merely an exemplary description, and the embodiments of this application are not limited thereto.

[0250] The above text combined Figures 1 to 8 This document describes in detail the imaging method provided in the embodiments of this application. The following will combine... Figure 9The apparatus embodiments of this application are described in detail below. It should be understood that the shooting apparatus of this application embodiments can execute the various shooting methods described in the foregoing embodiments of this application. That is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

[0251] Figure 9 This is a schematic block diagram of the imaging device 900 according to an embodiment of this application. It should be understood that the device 900 can perform... Figures 5 to 8 The shooting method shown.

[0252] like Figure 9 As shown, the photographing device 900 includes: a camera 910 and a data acquisition unit 920 (for example, the data acquisition unit 920 is a flash sensor), and a processing unit 930.

[0253] In some embodiments, the processing unit 930 is further configured to invoke the camera 910;

[0254] The processing unit 930 is also used to call the data acquisition unit 920 to acquire multiple first RAW data, wherein the multiple first RAW data are data of the current shooting scene acquired by the data acquisition unit 920 using a first gain and a first shutter speed;

[0255] The processing unit 930 is further configured to determine the current brightness value based on the plurality of first RAW data;

[0256] The processing unit 930 is further configured to determine a target brightness value based on a first ambient illuminance and a first correspondence relationship, wherein the first ambient illuminance is used to characterize the actual illuminance of the current shooting environment determined by the image sensor, and the first correspondence relationship includes brightness values ​​corresponding to different ambient illuminances.

[0257] The processing unit 930 is further configured to determine whether the difference between the current brightness value and the target brightness value is less than a difference threshold.

[0258] The processing unit 930 is further configured to control the data acquisition unit 920 to sample the current shooting environment using a second gain and a second shutter speed when the difference between the current brightness value and the target brightness value is not less than the difference threshold, thereby obtaining multiple second RAW data. The second gain and the second shutter speed are determined based on the target brightness value, and the second gain is different from the first gain and the second shutter speed is different from the first shutter speed.

[0259] The processing unit 930 is also configured to call the camera 910 to take pictures using an adjusted exposure time, wherein the adjusted exposure time is an integer multiple of the light source frequency, and the light source frequency is determined based on the plurality of second RAW data.

[0260] Optionally, as an embodiment, the processing unit 930 is further configured to determine the current exposure based on the current brightness value; determine the target exposure based on the current exposure and a first ratio, wherein the first ratio is the ratio between the target brightness value and the current brightness value; and when the difference between the current brightness value and the target brightness value is not less than the difference threshold, use the target exposure to look up the second gain and the second shutter speed in a second correspondence, wherein the second correspondence includes the gain and shutter speed corresponding to different exposures.

[0261] Optionally, as an embodiment, the processing unit 930 is used to control the data acquisition unit 920 to sample the current shooting environment based on the second gain and the second shutter speed, including:

[0262] Adjust the first gain to the second gain;

[0263] After the first gain is adjusted to the second gain, the first shutter speed is adjusted to the second shutter speed.

[0264] Optionally, as an embodiment, the processing unit 930 is configured to determine the current luminance value based on the plurality of first RAW data, including:

[0265] The current brightness value is obtained by summing and averaging the multiple first RAW data.

[0266] Optionally, as an embodiment, the processing unit 930 is further configured to perform Fourier transform on the plurality of second RAW data based on the anti-striping algorithm to obtain a first frequency and a first amplitude;

[0267] Based on the first frequency and the first amplitude, the current exposure time is adjusted to an integer multiple of the light source frequency, where the light source frequency is the first frequency.

[0268] Optionally, as an embodiment, the processing unit 930 is further configured to determine the light source frequency using the plurality of first RAW data when the difference between the current brightness value and the target brightness value is less than the difference threshold.

[0269] Optionally, as an embodiment, the processing unit 930 is further configured to determine whether the second gain is the same as the first gain and whether the second shutter speed is the same as the first shutter speed before controlling the data acquisition unit 920 to sample the current shooting environment using the second gain and the second shutter speed; when the second gain is different from the first gain and the second shutter speed is different from the first shutter speed, the processing unit 930 is configured to control the data acquisition unit 920 to sample the current shooting environment using the second gain and the second shutter speed.

[0270] Optionally, as an embodiment, the current shooting scene is a low-light scene, which includes scenes where the ambient illuminance is lower than a first brightness threshold; or, the current shooting scene is an overexposed scene, which includes scenes where the data exceeds the upper limit of the bit width that the data acquisition unit 920 can support.

[0271] In one possible example, processing unit 930 can be implemented by a processor or processing unit. It should be understood that the above-described device 900 is embodied in the form of a functional unit. The term "unit" here can be implemented in software and / or hardware, and this application embodiment does not specifically limit this.

[0272] For example, a "unit" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory that executes one or more software or firmware programs, integrated logic circuitry, and / or other suitable devices that can provide the above functions. In a simple embodiment, those skilled in the art will appreciate that device 900 can employ... Figure 4 As shown in the figure.

[0273] As can be seen from the above, the embodiments of this application provide a shooting method and an electronic device. By adjusting the gain and shutter speed of the scintillation sensor, the scintillation sensor can accurately and stably identify the light source frequency in both dark and overexposed environments, thereby eliminating banding, improving image quality, and enhancing the user's shooting experience.

[0274] This application also provides a computer program product that, when executed by a processor, implements the methods described in any of the method embodiments of this application.

[0275] The computer program product can be stored in memory and, after processes such as preprocessing, compilation, assembly, and linking, is finally converted into an executable object file that can be executed by a processor.

[0276] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the methods described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

[0277] The computer-readable storage medium can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0278] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and technical effects of the above-described apparatus and equipment can be referred to the corresponding processes and technical effects in the foregoing method embodiments, and will not be repeated here.

[0279] In the several embodiments provided in this application, the systems, apparatuses, and methods disclosed can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The apparatus embodiments described above are merely illustrative; the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components can be combined or integrated into another system. Furthermore, the coupling between units or components can be direct coupling or indirect coupling, including electrical, mechanical, or other forms of connection.

[0280] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0281] Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and / or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this paper generally indicates that the preceding and following related objects have an "or" relationship.

[0282] In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A shooting method, characterized in that, The method is applied to an electronic device, the electronic device including a camera and a flash sensor, the method comprising: Turn on the camera; The flash sensor acquires multiple first RAW data points, which are data of the current shooting environment acquired by the flash sensor using a first gain and a first shutter speed. The current brightness value is determined based on the plurality of first RAW data; The target brightness value is determined based on the first ambient illuminance and the first correspondence relationship. The first ambient illuminance is used to characterize the actual illuminance of the current shooting environment determined by the image sensor. The first correspondence relationship includes the brightness values ​​corresponding to different ambient illuminances. Determine whether the difference between the current brightness value and the target brightness value is less than a difference threshold; When the difference between the current brightness value and the target brightness value is not less than the difference threshold, the flicker sensor is controlled to sample the current shooting environment using a second gain and a second shutter speed to obtain multiple second RAW data, wherein the second gain is different from the first gain and the second shutter speed is different from the first shutter speed; The camera uses an adjusted exposure time for shooting, which is an integer multiple of the light source frequency, and the light source frequency is determined based on the plurality of second RAW data.

2. The method according to claim 1, characterized in that, The second gain and the second shutter speed are determined based on the target brightness value.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Determine the current exposure based on the current brightness value; The target exposure is determined based on the current exposure and a first ratio, where the first ratio is the ratio between the target brightness value and the current brightness value. When the difference between the current brightness value and the target brightness value is not less than the difference threshold, the second gain and the second shutter speed are found in the second correspondence using the target exposure. The second correspondence includes the gain and shutter speed corresponding to different exposures.

4. The method according to claim 1 or 2, characterized in that, The control of the flash sensor to sample the current shooting environment based on the second gain and the second shutter speed includes: Adjust the first gain to the second gain; After the first gain is adjusted to the second gain, the first shutter speed is adjusted to the second shutter speed.

5. The method according to claim 1 or 2, characterized in that, Determining the current brightness value based on the plurality of first RAW data includes: The current brightness value is obtained by summing and averaging the multiple first RAW data.

6. The method according to claim 1 or 2, characterized in that, The method further includes: The first frequency and the first amplitude are obtained by performing Fourier transform on the multiple second RAW data based on the anti-striping algorithm; Based on the first frequency and the first amplitude, the current exposure time is adjusted to an integer multiple of the light source frequency, where the light source frequency is the first frequency.

7. The method according to claim 1 or 2, characterized in that, The method further includes: When the difference between the current brightness value and the target brightness value is less than the difference threshold, the light source frequency is determined using the plurality of first RAW data.

8. The method according to claim 1 or 2, characterized in that, Before controlling the scintillation sensor to sample the current shooting environment using a second gain and a second shutter speed, the method further includes: Determine whether the second gain is the same as the first gain, and whether the second shutter speed is the same as the first shutter speed; When the second gain is different from the first gain and the second shutter speed is different from the first shutter speed, the flash sensor is controlled to sample the current shooting environment using the second gain and the second shutter speed.

9. The method according to claim 1 or 2, characterized in that, The current shooting environment is a low-light scene, which includes scenes where the ambient light is lower than a first brightness threshold; or, the current shooting environment is an overexposed scene, which includes scenes where the data exceeds the upper limit of the bit width that the flicker sensor can support.

10. An electronic device, characterized in that, The device includes a processor and a memory coupled together, the memory being used to store a computer program that, when executed by the processor, causes the electronic device to perform the method of any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 9.

12. A chip, characterized in that, Includes a processor, which, when executing instructions, performs the method as described in any one of claims 1 to 9.

Images