Image processing method and electronic device
By adjusting the exposure time of the mobile phone in the AC light source environment, the light strobe problem is solved, the image quality and user experience are improved, and the shooting effect is achieved without strobes.
Patent Information
- Application Number
- CN202310206843.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-23
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-02-23
AI Technical Summary
When a mobile phone takes images in an AC light source environment, light strobe problems are prone to occur, resulting in strobe-flashing of the captured images, reducing image quality and user experience.
By obtaining the shooting information of the image to be processed, including light source information and exposure time, adjusting the exposure time to eliminate strobe problems, ensuring the target exposure time of the image, and thus improving the clarity and user experience of the image.
It effectively eliminates the strobe problem in the image, improves the clarity of the image and the user's visual experience, avoids the image being overexposed or too dark, and improves the accuracy of the exposure time.
Smart Images

Figure CN117135467B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and electronic equipment. Background Art
[0002] With the development of electronic equipment (such as mobile phones) technology, the shooting function of mobile phones has also been rapidly developed, and more and more users like to use mobile phones to shoot images (such as photos or videos).
[0003] Since the energy of alternating current changes periodically over time, when the light source works under alternating current, the light source will flicker. When the mobile phone takes pictures in an AC light environment, the pictures taken will flicker due to the flicker of the light source, that is, the pictures taken will have light and dark stripes, which reduces the image quality and affects the user's shooting experience. Summary of the invention
[0004] The embodiments of the present application provide an image processing method and an electronic device for improving image quality.
[0005] In the first aspect, an image processing method is provided. In the method, an electronic device can obtain an image to be processed taken by a camera in the electronic device, and determine a target brightness value of the image to be processed, wherein the target brightness value represents the brightness that the image to be processed should reach. Then, the electronic device obtains shooting information corresponding to the image to be processed, and determines the target exposure time of the image to be processed based on the shooting information corresponding to the image to be processed and the target brightness value. The shooting information includes light source information of the shooting environment; the light source information includes the light source amplitude, and the light source amplitude is used to determine the degree of adjustment of the exposure time of the image to be processed. Finally, the electronic device can adjust the exposure time of the image to be processed to the target exposure time to obtain the target image.
[0006] In the present application, by adjusting the exposure time according to the shooting information corresponding to the image to be processed, the flicker problem in the image to be processed can be eliminated, so that the target image is an image without flicker problem, which not only improves the clarity of the captured image, but also enhances the user's visual experience.
[0007] In addition, if the user starts a camera application, the electronic device can directly display the operational interface of the camera application in response to the startup operation of the camera application, and after the shooting control of the electronic device is triggered, the electronic device can capture an image without light and dark stripes, thereby solving the problem of flicker in the image when taking subsequent images.
[0008] In a possible implementation of the first aspect, the process of determining the target brightness value of the image to be processed may specifically include: the electronic device performs feature statistics on the image to be processed to obtain a statistical result of the image to be processed, wherein the statistical result may include an AE statistical result. Then, the electronic device determines an initial brightness value and a target brightness value of the image to be processed based on the AE statistical result.
[0009] In the present application, by performing feature statistics on the image to be processed, the target brightness value of the image to be processed is determined, which can provide a basis for the subsequent adjustment of the exposure time and image brightness adaptation, thereby ensuring the image quality.
[0010] In a possible implementation manner of the first aspect, before the electronic device obtains shooting information corresponding to the image to be processed, the method further includes: the electronic device adjusts an initial brightness value of the image to be processed to a target brightness value to obtain a first image.
[0011] In the present application, the image brightness of the image to be processed is adjusted before the exposure time is adjusted, so that the problem of the image to be processed being too bright or too dark can be avoided. Afterwards, the mobile phone can eliminate the stroboscopic problem of the first image by adjusting the exposure time of the first image, and because the brightness of the first image is appropriate, it can avoid the situation where the exposure time of the image is adjusted too much due to the image being too dark, thereby causing the image to be overexposed, thereby improving the accuracy of adjusting the exposure time of the image and ensuring the image quality.
[0012] In a possible implementation of the first aspect, the light source information further includes a light source frequency, and the electronic device determines the target exposure time of the image to be processed based on the shooting information corresponding to the image to be processed in combination with the target brightness value, which may specifically include: the electronic device determines whether the light source frequency is zero. Then, when the light source frequency is zero, the electronic device uses the first image as the target image; or, when the light source frequency is not zero, the electronic device determines the target exposure time based on the light source amplitude in combination with the target brightness value.
[0013] In the present application, before adjusting the exposure time of the image to be processed, it is necessary to determine whether the light source frequency is zero. Only when the light source frequency is not zero, the exposure time is adjusted. In this way, unnecessary exposure time adjustment of the image can be avoided, reducing the waste of resources.
[0014] In a possible implementation of the first aspect, the shooting information also includes a light source energy cycle. The electronic device determines the target exposure time of the image to be processed based on the shooting information corresponding to the image to be processed and in combination with the target brightness value. Specifically, it may include: the electronic device inputs the target brightness value into the automatic exposure algorithm to obtain the original total exposure of the image to be processed. Afterwards, the electronic device determines the original exposure time of the image to be processed based on the original total exposure and in combination with a preset exposure table. Then, the electronic device determines whether the original exposure time of the image to be processed is less than the light source energy cycle. Finally, in the case that the original exposure time of the image to be processed is less than the light source energy cycle, the electronic device determines the target exposure time of the image to be processed based on the light source amplitude and the original exposure time.
[0015] In this application, different adjustment strategies are determined based on the relationship between the original exposure time and the light source energy cycle of the mobile phone camera. If the original exposure time is less than the light source energy cycle, it is necessary to adjust the original exposure time according to the light source amplitude to determine the target exposure time. In this way, the exposure time can be adjusted in a targeted manner, thereby improving the accuracy of determining the target exposure time, thereby accurately and quickly eliminating the flicker problem in the image.
[0016] In a possible implementation of the first aspect, the electronic device determines the original exposure time of the image to be processed according to the original total exposure in combination with a preset exposure table, which may specifically include: the electronic device determines a target total exposure range from the preset exposure table according to the original total exposure, wherein the original total exposure is within the target total exposure range. Then, the electronic device determines whether the original total exposure is less than the preset total exposure, wherein the preset total exposure is the product of the exposure time corresponding to the target total exposure range and the minimum gain value. Finally, when the original total exposure is less than the preset total exposure, the electronic device calculates the ratio between the original total exposure and the minimum gain value to obtain the original exposure time of the image to be processed.
[0017] In the present application, if the original total exposure is less than the exposure time corresponding to the target total exposure range, it means that the original exposure time corresponding to the original total exposure is less than the light source energy cycle, and the minimum original gain value can only be 1. Therefore, the original exposure time can be determined by dividing the original total exposure by the minimum gain value. In this way, the accuracy of the original exposure time can be improved.
[0018] In another possible implementation of the first aspect, the method further includes: when the original total exposure is greater than or equal to the preset total exposure, the electronic device determines the exposure time corresponding to the target total exposure range as the original exposure time of the image to be processed.
[0019] In the present application, if the original total exposure is greater than or equal to the exposure time corresponding to the target total exposure range, it means that the original exposure time corresponding to the original total exposure is greater than the light source energy cycle. Therefore, the original gain value of the image can be adjusted to make the original exposure time an integer multiple of the light source energy cycle, thereby eliminating the flicker phenomenon in the image and improving the image quality.
[0020] In a possible implementation of the first aspect, the electronic device determines the target total exposure range from the preset exposure table according to the original total exposure, which may specifically include: the electronic device searches the preset exposure table for the total exposure range to which the original total exposure belongs, and uses the searched total exposure range as a candidate total exposure range. Thereafter, when the number of the candidate total exposure ranges is equal to 1, the electronic device may directly determine the candidate total exposure range as the target total exposure range.
[0021] In the present application, after the electronic device finds the total exposure range to which the original total exposure belongs, it needs to determine the number of candidate total exposure ranges. If the number of candidate total exposure ranges is equal to 1, the candidate total exposure range can be directly determined as the target total exposure range. In this way, the target total exposure range can be accurately determined, providing a basis for the subsequent determination of the original exposure time.
[0022] In another possible implementation of the first aspect, the method further includes: when the number of the candidate total exposure ranges is greater than 1, the electronic device determines, from a preset exposure table, exposure times corresponding to the candidate total exposure ranges. Thereafter, the electronic device may use the candidate total exposure range corresponding to the minimum exposure time as the target total exposure range.
[0023] In the present application, if the number of candidate total exposure ranges is greater than 1, it is necessary to select the target total exposure range based on the exposure time corresponding to the candidate total exposure range, that is, to use the candidate total exposure range corresponding to the minimum exposure time as the target total exposure range. In this way, the occurrence of image overexposure can be reduced and the image quality can be improved.
[0024] In a possible implementation of the first aspect, the electronic device determines the target exposure time of the image to be processed based on the light source amplitude and the original exposure time, which may specifically include: the electronic device determines the standard exposure time of the image to be processed using the light source amplitude, wherein the standard exposure time is a critical exposure time corresponding to when there is no stroboscopic light in the image to be processed. Then, the electronic device determines the target exposure time of the image to be processed based on the standard exposure time and the original exposure time.
[0025] In the present application, since the standard exposure time is determined based on the amplitude of the light source, and the standard exposure time is the critical exposure time corresponding to the absence of flicker in the image to be processed, the target exposure time is determined by comparing the standard exposure time with the original exposure time. This can improve the accuracy of determining the target exposure time without wasting resources, thereby eliminating the flicker problem in the image.
[0026] In one embodiment, the absence of stroboscopic images in the image to be processed indicates that the stroboscopic degree of the image to be processed is lower than a preset degree. One manifestation of the stroboscopic degree of the image to be processed being lower than the preset degree is that the image to be processed actually does not have stroboscopic images, that is, the image to be processed actually does not have black and white stripes; another manifestation is that although there is stroboscopic images in the image to be processed, the user cannot see the black and white stripes on the image to be processed, that is, the black and white stripes on the image to be processed are not discovered by the user; another manifestation is that although there is stroboscopic images in the image to be processed, it does not affect the content of the image to be processed, that is, the black and white stripes on the image to be processed are within the tolerance range of the user and do not affect the user's viewing of the image to be processed.
[0027] In another embodiment, the absence of stroboscopic images in the image to be processed indicates that the quality of the image to be processed is higher than or equal to the quality of the standard image. The standard image indicates a preset image without stroboscopic images. The interpretation of the preset image without stroboscopic images can be similar to the interpretation of the above-mentioned image without stroboscopic images.
[0028] In a possible implementation of the first aspect, the electronic device determines the target exposure time of the image to be processed based on the standard exposure time and the original exposure time, which may specifically include: when the original exposure time is less than the standard exposure time, determining the target exposure time of the image to be processed as the standard exposure time; or, when the original exposure time is greater than or equal to the standard exposure time, determining the target exposure time of the image to be processed as the original exposure time.
[0029] In the present application, if the original exposure time is less than the standard exposure time, it means that there is a flicker problem in the image to be processed. Therefore, it is necessary to adjust the exposure time to the standard exposure time to eliminate the flicker problem in the image; if the original exposure time is greater than or equal to the standard exposure time, it means that there is no flicker problem in the image to be processed. Therefore, in order to avoid the problem of overexposure of the image, there is no need to adjust the exposure time of the image to be processed. The original exposure time can be directly determined as the target exposure time, thereby avoiding unnecessary waste of resources.
[0030] In another possible implementation of the first aspect, the electronic device determines the target exposure time of the image to be processed based on the standard exposure time and the original exposure time, which may specifically include: the electronic device determines the magnification of the original exposure time based on the energy cycle of the light source and the original exposure time; wherein the magnification represents the minimum increase multiple of the exposure time corresponding to the flicker of the image to be processed. Then, the electronic device determines the expected exposure time of the image to be processed based on the product of the magnification and the original exposure time; wherein the expected exposure time refers to the exposure time corresponding to the expected absence of the flicker problem in the image to be processed. Finally, the electronic device determines the target exposure time of the image to be processed from the expected exposure time, the standard exposure time and the original exposure time.
[0031] In the present application, the magnification of the original exposure time is first determined based on the energy cycle of the light source and the original exposure time, and then the product of the magnification and the original exposure time is determined as the expected exposure time of the image to be processed. Finally, the target exposure time is determined by comparing the expected exposure time, the standard exposure time and the original exposure time. In this way, not only the accuracy of determining the target exposure time can be improved and the flicker problem in the image can be eliminated, but also the overexposure problem of the adjusted image can be avoided, thereby improving the image quality.
[0032] In a possible implementation of the first aspect, the electronic device determines the magnification of the original exposure time based on the light source energy cycle and the original exposure time, which may specifically include: the electronic device determines the maximum adjustment multiple of the original exposure time based on the shooting mode corresponding to the image to be processed. Then, the electronic device determines the magnification of the original exposure time based on the light source energy cycle, the original exposure time and the maximum adjustment multiple.
[0033] In a possible implementation of the first aspect, the electronic device determines the magnification of the original exposure time based on the light source energy cycle, the original exposure time, and the maximum adjustment multiple, which may specifically include: the electronic device calculates the ratio between the light source energy cycle and the original exposure time to obtain an initial multiple. Then, the electronic device uses the minimum value between the initial multiple and the maximum adjustment multiple as the magnification of the original exposure time.
[0034] In the present application, since the maximum adjustment multiple of the original exposure time is determined according to the shooting mode corresponding to the image to be processed, and the shooting mode includes the photo mode and the video recording mode, the magnification is determined by selecting the minimum multiple from the initial multiple and the maximum adjustment multiple. This can improve the accuracy of the magnification determination and provide a basis for the subsequent determination of the expected exposure time.
[0035] In a possible implementation manner of the first aspect, the electronic device determines the target exposure time of the image to be processed from the expected exposure time, the standard exposure time, and the original exposure time, which may specifically include: the electronic device uses the minimum value of the expected exposure time and the standard exposure time as the first exposure time. Then, the electronic device uses the maximum value of the first exposure time and the original exposure time as the target exposure time of the image to be processed.
[0036] In the present application, a first exposure time with a smaller value is first selected from the expected exposure time and the standard exposure time, and the first exposure time is compared with the original exposure time to select an exposure time with a larger value as the target exposure time. In this way, not only can the flicker problem in the image be eliminated, but also the occurrence of image overexposure can be reduced, thereby reducing unnecessary waste of resources.
[0037] In another possible implementation manner of the first aspect, the method further includes: when the original exposure time of the image to be processed is greater than or equal to the energy cycle of the light source, the electronic device determines the original exposure time of the image to be processed as the target exposure time.
[0038] In the present application, if the original exposure time is greater than or equal to the light source energy cycle, it means that the image to be processed is an image without flicker problems. Therefore, the electronic device does not need to perform image flicker processing, that is, there is no need to determine the target exposure time of the image to be processed. The electronic device can directly determine the original exposure time as the target exposure time. In this way, unnecessary exposure time adjustment of the image can be avoided, reducing the occurrence of resource waste.
[0039] In a possible implementation of the first aspect, the electronic device adjusts the exposure time of the image to be processed to the target exposure time to obtain the target image, which may specifically include: the electronic device adjusts the exposure time of the first image to the target exposure time to obtain the second image. Then, the electronic device calculates the ratio between the target exposure time and the original exposure time to obtain the target gain value of the camera. Finally, the electronic device restores the brightness of the second image based on the target gain value of the camera to obtain the target image, wherein the brightness value of the target image is the target brightness value.
[0040] In the present application, after the electronic device adjusts the image according to the target exposure time, it is necessary to restore the image brightness to ensure that the brightness value of the target image is the target brightness value. In this way, the problem of overexposure or underexposure of the adjusted image can be avoided, thereby improving the image quality.
[0041] In a second aspect, the present application provides an electronic device, comprising a display screen, a memory and one or more processors; the display screen, the memory and the processor are coupled; the display screen is used to display an image generated by the processor, and the memory is used to store computer program code, wherein the computer program code comprises computer instructions; when the processor executes the computer instructions, the electronic device executes the method described above.
[0042] In a third aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the method described above.
[0043] In a fourth aspect, the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute the method described above.
[0044] In a fifth aspect, a chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method as described above.
[0045] It can be understood that the beneficial effects that can be achieved by the electronic device described in the second aspect, the computer-readable storage medium described in the third aspect, the computer program product described in the fourth aspect, and the chip described in the fifth aspect provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1A A schematic diagram of a photo-taking scene provided in an embodiment of the present application;
[0047] Figure 1B A schematic diagram of an interface for capturing an image provided by an embodiment of the present application;
[0048] Figure 1C A schematic diagram of an interface for shooting a video provided in an embodiment of the present application;
[0049] Figure 2 A schematic diagram of an interface for displaying captured images provided in an embodiment of the present application;
[0050] Figure 3A A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;
[0051] Figure 3B A schematic diagram showing a front camera of a mobile phone provided in an embodiment of the present application;
[0052] Figure 3C A schematic diagram showing a rear camera of a mobile phone provided in an embodiment of the present application;
[0053] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;
[0054] Figure 5A A flowchart of an image processing method provided in an embodiment of the present application;
[0055] Figure 5B A flowchart of another image processing method provided in an embodiment of the present application;
[0056] Figure 6 A schematic diagram of the relationship between a first light source amplitude and a first standard exposure time provided in an embodiment of the present application;
[0057] Figure 7 A schematic diagram of a relationship for adjusting image brightness provided in an embodiment of the present application;
[0058] Figure 8 A timing diagram of an image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION
[0059] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.
[0060] When an artificial light source (or simply referred to as a light source) is driven by alternating current, the energy of the light source is not evenly distributed over time, but changes in the same period as the alternating current, resulting in a light flicker problem in the light source, that is, the luminous flux, illumination or brightness of the light source will change periodically. For example, indoor places often have light sources (such as lighting equipment, (light-emitting diode, LED) billboards, etc.). Taking the light source as a fluorescent lamp as an example, the working principle of the fluorescent lamp is that under the action of high-voltage current, the inert gas in the lamp tube discharges, and the electrons generated by the discharge reach the fluorescent lamp tube to make the fluorescent lamp emit light. Assuming that the fluorescent lamp is driven by 50Hz alternating current, the 50Hz alternating current causes the fluorescent lamp to flicker 100 times in 1s. In other words, the time for a fluorescent lamp to flicker once is 0.01s.
[0061] When an electronic device captures an image under an AC light source, if the exposure time of the electronic device is improperly set, the captured image may show a stroboscopic phenomenon (or described as a banding phenomenon), that is, the image may show alternating bright and dark stripes, greatly reducing the image quality, affecting the user's visual experience, and reducing the user's shooting experience. The image here can refer to an image in a photo or video.
[0062] For example, if Figure 1A As shown, the target object 100 is dining in an indoor place, and the indoor place includes a lighting device, and the lighting device 200 is located above the target object 100. When a user uses an electronic device to take a photo including the target object 100, the user can click on the Figure 1B In response to the user's click operation on the first control 110, the electronic device can generate a corresponding photo. Alternatively, when the user uses the electronic device to shoot a video including the target object 100, the user can click on the first control 110 as shown in FIG. Figure 1C The second control 120 on the second shooting interface is shown. In response to the user's click operation on the second control 120, the electronic device can generate a corresponding video.
[0063] When the lighting device 200 is operated under AC power, there is an AC light source in the shooting environment of the electronic device. Since the lighting device 200 may have a light strobe phenomenon, the image in the photo or video generated by the electronic device may have alternating light and dark stripes (such as Figure 2 As shown), thus affecting the user's visual experience.
[0064] It is understandable that when an electronic device shoots a video under an AC light source environment, the light and dark stripes on different images in the captured video may appear at the same position or at different positions.
[0065] It should be noted that the above Figure 1A The positions of the lighting devices shown are only examples, i.e. Figure 1A The position of the AC light source shown is only an example of an AC light source existing in the shooting environment of the electronic device. When the content shot by the electronic device has an AC light source, the shooting environment of the electronic device also has an AC light source. For example, when the content shot by the electronic device has an LED billboard (such as a light box for ordering dishes), the image obtained may also have a stroboscopic problem.
[0066] In view of the above problems, an embodiment of the present application provides an image processing method. First, an image to be processed taken by a camera in an electronic device is obtained, and a target brightness value of the image to be processed is determined. Then, shooting information corresponding to the image to be processed is determined, and a target exposure time of the image to be processed is determined according to the shooting information and the target brightness value. Finally, the exposure time of the image to be processed is adjusted to the target exposure time to obtain a target image. That is, the stroboscopic problem in the image to be processed can be eliminated by adjusting the exposure time according to the shooting information corresponding to the image to be processed, so that the target image is an image without stroboscopic problem.
[0067] Afterwards, if the user starts a camera application, the electronic device can directly display the operational interface of the camera application in response to the startup operation of the camera application, and after the shooting control of the electronic device is triggered, the electronic device can capture an image without light or dark stripes, solving the problem of flicker in the image, which not only improves the clarity of the captured image, but also enhances the user's visual experience.
[0068] It should be noted that the electronic device in the embodiments of the present application may be a mobile phone, a tablet computer, a smart watch, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, as well as a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, and other devices containing a camera. The embodiments of the present application do not impose any special restrictions on the specific form of the electronic device.
[0069] For example, see Figure 3A As shown, it is a schematic diagram of the hardware structure of an electronic device 300 provided in an embodiment of the present application, such as Figure 3AAs shown, the electronic device 300 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc. Among them, the sensor module 380 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.
[0070] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0071] The processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0072] The controller may be the nerve center and command center of the electronic device 300. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.
[0073] The processor 310 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory may store instructions or data that the processor 310 has just used or cyclically used. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.
[0074] In some embodiments, the processor 310 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, etc.
[0075] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0076] The charging management module 340 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 may receive charging input from a wired charger through the USB interface 330. In some wireless charging embodiments, the charging management module 340 may receive wireless charging input through a wireless charging coil of the electronic device 300. While the charging management module 340 is charging the battery 342, it may also power the electronic device 300 through the power management module 341.
[0077] The power management module 341 is used to connect the battery 342, the charging management module 340 and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, and supplies power to the processor 310, the internal memory 321, the external memory, the display screen 394, the camera 393, etc. The power management module 341 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 341 can also be set in the processor 310. In other embodiments, the power management module 341 and the charging management module 340 can also be set in the same device.
[0078] The wireless communication function of the electronic device 300 can be implemented through the antenna 1, the antenna 2, the mobile communication module 350, the wireless communication module 360, the modem processor and the baseband processor.
[0079] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 300. The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker, a receiver, etc.), or displays an image or video through a display screen 394.
[0080] The wireless communication module 360 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 300. The wireless communication module 360 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and filters it, and sends the processed signal to the processor 310. The wireless communication module 360 can also receive the signal to be sent from the processor 310, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0081] The electronic device 300 implements the display function through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs, which execute program instructions to generate or change display information.
[0082] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 300 may include 1 or N display screens 394, where N is a positive integer greater than 1.
[0083] The electronic device 300 can realize the shooting function through ISP, camera 393, video codec, GPU, display screen 394 and application processor.
[0084] ISP is used to process the data fed back by camera 393. For example, when an electronic device takes a photo, the shutter is opened, and light is transmitted to the camera photosensitive element (or image sensor) through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 393. In some embodiments, camera 393 includes a shutter. The shutter is a device in the camera used to control the time that light irradiates the photosensitive element.
[0085] The camera 393 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 300 may include 1 or N cameras 393, where N is a positive integer greater than 1.
[0086] In some embodiments, the camera 393 may include a lens, which is an optical component for generating an image.
[0087] Exemplarily, the N cameras 393 may include: one or more front cameras and one or more rear cameras. For example, the electronic device 300 is a mobile phone. Figure 3B The illustrated mobile phone includes a front-facing camera, such as front-facing camera 10 . Figure 3C The mobile phone shown includes three rear cameras, such as rear cameras 11, 12 and 13. Of course, the number of cameras in the above mobile phone includes but is not limited to the number described in the above example.
[0088] Among them, the above-mentioned N cameras 393 may include one or more of the following cameras: a main camera, a telephoto camera, a wide-angle camera, an ultra-wide-angle camera, a macro camera, a fisheye camera, an infrared camera, a depth camera and a black and white camera.
[0089] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 300 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
[0090] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 300. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function, such as storing music, video and other files in the external memory card.
[0091] The internal memory 321 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the electronic device 300 by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 300 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
[0092] The key 390 includes a power key, a volume key, etc. The key 390 may be a mechanical key or a touch key. The electronic device 300 may receive key input and generate key signal input related to user settings and function control of the electronic device 300.
[0093] Motor 391 can generate vibration prompts. Motor 391 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback.
[0094] Indicator 392 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0095] The SIM card interface 395 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 300 by inserting it into the SIM card interface 395 or pulling it out from the SIM card interface 395. The electronic device 300 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 395 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 395 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 395 can also be compatible with different types of SIM cards. The SIM card interface 395 can also be compatible with external memory cards. The electronic device 300 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 300 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 300 and cannot be separated from the electronic device 300.
[0096] Exemplarily, the software system of the electronic device 300 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 300.
[0097] Figure 4 It is a software structure block diagram of the electronic device 300 according to an embodiment of the present application.
[0098] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer (referred to as the application layer), the application framework layer (referred to as the framework layer), the Android runtime (Android runtime) and the system library, and the kernel layer (or driver layer).
[0099] The application layer may include a series of application packages. The application layer may include multiple application packages. Figure 4 As shown, the application package can be an application such as gallery, map, phone, video, calendar, short message, camera, etc. It can be understood that the camera application can be used to trigger the electronic device to use the camera to take pictures or record videos.
[0100] In the photo mode, the image reported by the underlying layer is displayed on the viewfinder interface of the captured image, or, in the video recording mode, the video stream reported by the underlying layer is displayed on the viewfinder interface of the recorded video. The electronic device 200 may include one or more cameras, each of which may be used to capture images, and the continuous multiple frames of images captured by the camera may form a video stream. In other words, each of the above cameras may be used to capture a video stream.
[0101] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 4 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.
[0102] Among them, the window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying pictures.
[0103] The phone manager is used to provide communication functions of the electronic device 300, such as management of call status (including connecting, hanging up, etc.).
[0104] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.
[0105] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.
[0106] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.
[0107] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.
[0108] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
[0109] The system library can include multiple functional modules. For example: Media Libraries, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc. Among them, the media library supports a variety of commonly used audio and video format playback and recording, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.
[0110] The kernel layer is the layer between hardware and software. Figure 4 As shown, the core layer includes a brightness adjustment module, an automatic exposure (AE) module, a 3A statistics module, and a strobe detection module.
[0111] The brightness adjustment module is used to process the brightness of the image to reduce the brightness of the image when the image is overexposed, or to increase the brightness of the image when the image is underexposed, so as to achieve optimal adjustment of the image brightness. It can be understood that the brightness adjustment module is calculated based on the image signal processor ISP.
[0112] The automatic exposure module is used to adjust the exposure time of the image to solve the problem of stroboscopic images. In some embodiments, the AE module can be a hardware module. The 3A statistical module is used to count the characteristic information in the image. Exemplarily, the characteristic information may include the brightness value of each pixel in the image. Among them, 3A indicates automatic exposure, automatic white balance (AWB) and automatic focus (AF). Automatic exposure can automatically adjust the brightness of the image to ensure the quality of the captured image. Automatic white balance can automatically adjust and restore the color of the image under different lighting to ensure that the color of the captured image is consistent with the real color of the object. Automatic focus can automatically adjust the focal length of the image to ensure the clarity of the captured image.
[0113] It can be understood that the 3A statistical module performs feature statistics based on the processor 310. The stroboscopic detection module is used to detect light source information of the shooting environment, and the light source information may include light source amplitude and light source frequency.
[0114] In some embodiments, the electronic device can adjust the exposure time of the image taken by the mobile phone through the automatic exposure module in the core layer to remove the image flicker. Figure 4As shown, in response to the user's operation of the camera application in the electronic device, the electronic device (or alternatively described as the corresponding application) can execute step a to trigger the camera to capture an image. In addition, step b can be executed to trigger the kernel layer (such as the stroboscopic detection module in the kernel layer) to perform light source detection on the shooting environment to obtain light source information. Exemplarily, the light source information may include the light source amplitude and / or the light source frequency. The user's operation of the camera application in the electronic device is used to trigger the electronic device to capture an image, such as a click operation on the first control 110 or the second control 120 mentioned above.
[0115] Afterwards, the camera can execute step c and send the captured image to the kernel layer so that the relevant modules of the kernel layer can determine the target exposure time of the image, thereby adjusting the exposure time of the image to the target exposure time to avoid the flicker problem in the image.
[0116] The image processing method of the embodiment of the present application can be used in a scene where an electronic device captures an image. For example, the electronic device captures a photo or video through the front camera or rear camera of the electronic device. In the following embodiments, the method of the embodiment of the present application is described by taking the electronic device as a mobile phone as an example.
[0117] The embodiment of the present application provides an image processing method. After a mobile phone captures an image, the mobile phone can use the light source of the shooting environment corresponding to the image to determine the target exposure time corresponding to the image. Afterwards, the mobile phone can adjust the exposure time of the image to the target exposure time to eliminate the stroboscopic problem of the image and improve the image quality. For example, Figure 5A As shown, the image processing method may include steps S501 to S505:
[0118] S501, the mobile phone obtains an image to be processed taken by the mobile phone camera.
[0119] In one case, the image to be processed may be an image captured by the mobile phone in camera mode. In another case, the image to be processed may be an image captured by the mobile phone in video mode. Specifically, the image captured in the video mode may be an image in a video file obtained after the video shooting is completed. The image captured in the video mode may also be an image captured by the mobile phone camera during the video shooting process, so that the preview image displayed by the mobile phone is an image without light and dark stripes, so that the image in the video file obtained after the video shooting is completed does not have light and dark stripes.
[0120] Exemplarily, the camera may be a front camera or a rear camera of a mobile phone. The number of cameras used by the mobile phone to capture images may be one or more.
[0121] For example, the user can use the mobile phone camera to shoot different scenes. Taking a restaurant as an example, the shooting scene can be a scene with diners eating, a scene with diners picking up food, or a blank scene without diners; taking a gym as an example, the shooting scene can be a scene with athletes resting, a scene with athletes playing, or a blank scene without athletes.
[0122] S502: The mobile phone determines an initial brightness value and a target brightness value of the image to be processed.
[0123] Exemplarily, the mobile phone may calculate the average brightness value of all pixels in the image to be processed to obtain an initial brightness value of the image to be processed.
[0124] Exemplarily, after obtaining the image to be processed, the mobile phone may perform feature statistics on the image to be processed to obtain a statistical result of the image to be processed. The statistical result may include an AE statistical result. The AE statistical result may include a brightness histogram and / or a brightness area statistical value.
[0125] For example, the brightness histogram, also known as the gray histogram, is a statistic of the grayscale distribution of an image, representing the brightness of the image. Simply put, the grayscale histogram is to count the frequency of occurrence of all pixels in the image according to the size of the grayscale value. Therefore, the grayscale histogram is a function of the grayscale level, which represents the number of pixels with a certain grayscale level (or described as brightness) in the image, that is, it reflects the frequency of occurrence of a certain brightness in the image. Among them, the grayscale level indicates the maximum number of different grayscales in the image. The larger the grayscale level, the larger the brightness range of the image. The grayscale value refers to the brightness of a single pixel. The larger the grayscale value, the brighter the corresponding pixel.
[0126] The brightness region statistics represent the brightness values of each region in the image. The image is divided into one or more regions. A region may include one or more pixels, and each region corresponds to a brightness value. It can be understood that the brightness value corresponding to each region refers to the average brightness value of all pixels in the region. For example, if the region is a 2*2 region, the sum of the four pixels in the region can be divided by four to obtain the brightness value of the region.
[0127] Afterwards, the mobile phone can determine the initial brightness value and target brightness value of the image to be processed based on the AE statistical results. Specifically, the mobile phone can use the brightness area statistics to calculate the average brightness value of each area in the image to be processed to obtain the initial brightness value of the image to be processed. The mobile phone can determine the target brightness value of the image to be processed based on the brightness histogram. In this way, it can provide a basis for the subsequent adjustment of the image exposure time and the adaptation of the image brightness, thereby ensuring the image quality.
[0128] The initial brightness value is the current brightness value of the image to be processed. The target brightness value of the image to be processed is the brightness value that the image to be processed should reach. It should be noted that the target brightness value of the image to be processed may be consistent with the initial brightness value of the image to be processed, or may be inconsistent, and there is no specific limitation.
[0129] In one example, the target brightness value may be any brightness value in the target brightness interval, and the target brightness interval is predetermined according to actual conditions. For example, the minimum value of the target brightness interval may be the brightness value in an extremely dark environment (such as when the light intensity is less than the first set value), and the maximum value of the target brightness interval may be the brightness value in a strong light environment (such as when the light intensity is greater than the second set value). If the initial brightness value is within the target brightness interval, it means that the brightness of the image to be processed is appropriate, and therefore, the initial brightness value may be the target brightness value, which means that the brightness of the image to be processed does not need to be adjusted.
[0130] If the initial brightness value is lower than the minimum value of the target brightness range, it means that the brightness of the image to be processed is dark and the image brightness needs to be improved. Therefore, the mobile phone can select a value from the target brightness range and use it as the target brightness value. The target brightness value is higher than the initial brightness value of the image to be processed to improve the image quality.
[0131] If the initial brightness value is higher than the maximum value of the target brightness range, it means that the brightness of the image to be processed is brighter and the image brightness needs to be reduced. Therefore, the mobile phone can select a value from the target brightness range and use it as the target brightness value. The target brightness value is lower than the initial brightness value of the image to be processed to improve the image quality.
[0132] In another example, the target brightness value can be determined based on the ambient brightness of the shooting environment and the target brightness range. Among them, the target brightness range includes target brightness values corresponding to different ambient brightness levels. For example, level A is a strong light environment, and level E is an extremely dark environment. It can be understood that level A to level E are the level order corresponding to the gradual decrease in ambient brightness. If the ambient brightness corresponding to the shooting environment where the image to be processed is located is level A, the mobile phone can determine the target brightness corresponding to level A in the target brightness range as the target brightness value. It can be understood that the ambient brightness of the shooting environment can be obtained by statistically analyzing the pixel value corresponding to each pixel point in the image to be processed, that is, the ambient brightness of the shooting environment is obtained based on the brightness histogram.
[0133] In some embodiments, the above statistical results may also include statistical results of automatic white balance and statistical results of auto focus. The statistical results of the automatic white balance can be used to adjust the color of the image. The statistical results of the auto focus can be used to adjust the image focal length to ensure the clarity of the image.
[0134] S503: The mobile phone adjusts the initial brightness value of the image to be processed to the target brightness value to obtain a first image.
[0135] Specifically, after determining the initial brightness value and target brightness value of the image to be processed, it is necessary to adjust the brightness value of the image to be processed from the initial brightness value to the target brightness value, and obtain a first image with a brightness value of the target brightness value, so as to avoid the problem that the image to be processed is too bright or too dark. After that, the mobile phone can adjust the exposure time of the first image to eliminate the stroboscopic problem of the first image, and because the brightness of the first image is appropriate, it can avoid the situation that the exposure time of the image is adjusted too much due to the image being too dark, thereby causing the image to be overexposed, thereby improving the accuracy of adjusting the exposure time of the image and ensuring the image quality.
[0136] S504, the mobile phone obtains light source information of the shooting environment corresponding to the image to be processed, wherein the light source information includes light source amplitude.
[0137] The shooting environment corresponding to the above-mentioned image to be processed represents the environment in which the mobile phone is located when shooting the image to be processed. The light source information of the shooting environment corresponding to the image to be processed may represent the artificial light source in the environment in which the mobile phone is located when shooting the image to be processed.
[0138] In some embodiments, the mobile phone can detect the light source conditions of the shooting environment through the camera of the mobile phone and obtain the light source information of the shooting environment. For example, generally speaking, after the user starts the camera application, he will take photos or videos. In response to the user's startup operation of the camera application, the mobile phone can detect the light source of the shooting environment in which the mobile phone is located through the camera to obtain the light source information of the shooting environment. Afterwards, the mobile phone can obtain the above-mentioned image to be processed in response to the user's first operation, and the light source information of the shooting environment can be the light source information of the shooting environment corresponding to the image to be processed. The first operation is used to trigger the mobile phone to shoot an image. For example, the first operation can be a user's click operation on the above-mentioned first control 110 or the second control 120.
[0139] For another example, the camera of the mobile phone performs light source detection on the shooting environment during the process of shooting the above-mentioned image to be processed. For example, in response to the user's first operation, the mobile phone shoots the image to be processed and performs light source detection on the shooting environment to obtain the light source information of the shooting environment corresponding to the image to be processed.
[0140] Exemplarily, the mobile phone can obtain the light source energy in the shooting environment and sample the light source energy to obtain sampling data, perform signal conversion on the collected data through fast Fourier transform (FFT), that is, convert the time domain signal into a spectrum signal, and perform signal extraction on the converted sampling signal (or called spectrum signal) to obtain the light source information of the shooting environment.
[0141] Among them, the above-mentioned light source information may include the light source amplitude. The light source amplitude refers to the maximum absolute value that appears instantaneously in the light source within a cycle, that is, the maximum value of the luminous intensity of the light source within the same cycle. The cycle refers to the cycle corresponding to the frequency of the alternating current used by the light source. For example, the frequency of the alternating current is 50 Hz, and its corresponding cycle is 0.02 s. It should be noted that the light source amplitude corresponding to the light source in the shooting environment corresponding to the image determines the severity of the image's flicker problem. For example, the larger the light source amplitude, the more serious the image's flicker problem, that is, the more obvious the black and white stripes in the image.
[0142] In some embodiments, the above-mentioned light source information may also include the light source frequency. The light source frequency refers to the number of periodic fluctuations of the light source per second. For example, the light source frequency is 50 Hz. The light source frequency corresponding to the light source in the shooting environment can reflect whether there is an AC light source in the shooting environment, that is, it can reflect whether there is a flicker problem in the image taken under the shooting environment. If the light source frequency of the shooting environment is zero, it indicates that there is no AC light source in the shooting environment. In other words, there is no light flicker problem in the shooting environment. Correspondingly, the image captured under the shooting environment will not have a flicker problem. If the light source frequency of the shooting environment is not zero, it indicates that there is an AC light source in the shooting environment. In other words, there may be a light flicker problem in the shooting environment. Correspondingly, the image captured under the shooting environment may have a flicker problem.
[0143] Therefore, in order to avoid unnecessary exposure time adjustment of the image and eliminate the image flicker problem, the mobile phone can first use the light source frequency to determine whether the first image needs to be adjusted in exposure time, and if so, determine the target exposure time of the first image. If not, the first image can be directly used as the target image. Exemplarily, this process may include the following steps: Figure 5B S10-S20 shown.
[0144] S10. The mobile phone determines whether the frequency of the light source is zero.
[0145] In the embodiment of the present application, if the light source frequency is zero, it indicates that there may be no AC light source in the shooting environment, or that there is no flicker problem in the image to be processed taken in the shooting environment, so the mobile phone does not need to adjust the exposure time of the first image, and the mobile phone can execute S20. If the light source frequency is not zero, the image to be processed may have a flicker problem, so the mobile phone needs to determine the target exposure time of the first image so that the mobile phone can use the target exposure time to eliminate the flicker of the first image, and the mobile phone can continue to execute S505 described below.
[0146] S20. The mobile phone uses the first image as a target image.
[0147] It should be understood that the above Figure 5A The numbering of the steps shown does not represent the order in which the steps are executed. For example, the above S504 may be performed before S502, and the S504 may be performed before S503. Of course, S504 may also be performed after S503, or performed simultaneously, which is not limited in this application.
[0148] In some embodiments, the mobile phone may not adjust the brightness of the image to be processed, such as the mobile phone does not execute the above S503. Accordingly, the mobile phone can directly adjust the exposure time of the image to be processed, or when the light source frequency is zero, the mobile phone can directly use the first image as the target image.
[0149] S505: The mobile phone determines a target exposure time for the first image based on the light source amplitude and the target brightness value.
[0150] Exemplarily, the process of S505 may include, first, the mobile phone may input the target brightness value into the automatic exposure algorithm, and the automatic exposure algorithm outputs the total exposure corresponding to the target brightness value (or called the original total exposure, or called the total exposure of the first image).
[0151] It can be understood that the original total exposure is determined according to the original exposure time and the original gain value. Specifically, the original total exposure is the product of the original exposure time and the original gain value, that is, the original exposure time is determined based on the original total exposure.
[0152] Among them, the exposure time indicates the time that the shutter of the camera needs to be open when light is projected onto the photosensitive surface of the photographic photosensitive material. Correspondingly, the original exposure time can indicate the time that the shutter needs to be open when the mobile phone obtains an image with a brightness of the target brightness value. The length of the exposure time can determine the brightness of the image. The longer the exposure time, the brighter the image captured by the camera, that is, the greater the image brightness value. The shorter the exposure time, the darker the image captured by the camera, that is, the smaller the image brightness value.
[0153] Specifically, the mobile phone can determine the original exposure time according to the original total exposure and the preset exposure table. Exemplarily, the mobile phone determines the target total exposure range from the preset exposure table according to the original total exposure. The exposure time in the preset exposure table is an integer multiple of the light source energy cycle. The original total exposure is within the target total exposure range.
[0154] Exemplarily, the mobile phone can search the total exposure range to which the original total exposure belongs from the preset exposure table, and use the searched total exposure range as the candidate total exposure range. When the number of the candidate total exposure range is equal to 1, the mobile phone can directly determine the candidate total exposure range as the target total exposure range; when the number of the candidate total exposure range is greater than 1, the mobile phone can determine the target total exposure range according to the exposure time corresponding to the candidate total exposure range. The target total exposure range is the candidate total exposure range corresponding to the minimum exposure time.
[0155] Specifically, after determining the target total exposure range, the mobile phone can determine whether the original total exposure is less than a preset total exposure, where the preset total exposure is the product of the exposure time corresponding to the target total exposure range and the minimum gain value.
[0156] When the original total exposure is less than the preset total exposure, the ratio between the original total exposure and the minimum gain value is calculated to obtain the exposure time (or original exposure time) corresponding to the target brightness value; when the original total exposure is greater than or equal to the preset total exposure, the exposure time corresponding to the target total exposure range is determined as the original exposure time.
[0157] The preset exposure table includes at least one exposure time, a gain value corresponding to each exposure time in the at least one exposure time, and a corresponding total exposure range. Each exposure time is an integer multiple of the energy cycle of the light source. For example, if the energy cycle of the light source is 10ms, the exposure time can be 10ms, 20ms, 30ms, etc. Exemplarily, as shown in Table 1, the gain value corresponding to the exposure time of 10ms is 1, and accordingly, the total exposure range corresponding to the exposure time of 10ms and the gain value of 1 can be 0-10; the gain value corresponding to the exposure time of 10ms is 4, and accordingly, the total exposure range corresponding to the exposure time of 10ms and the gain value of 4 can be 10-40; the gain value corresponding to the exposure time of 20ms is 8, and accordingly, the total exposure range corresponding to the exposure time of 20ms and the gain value of 8 can be 20-160; the gain value corresponding to the exposure time of 30ms is 16, and accordingly, the total exposure range corresponding to the exposure time of 30ms and the gain value of 16 can be 30-480. That is, the total exposure range corresponding to Table 1 may include 0-10, 10-40, 20-160 and 30-480. It can be understood that the greater the total exposure, the brighter the image; the smaller the total exposure, the darker the image.
[0158] It should be noted that, in this embodiment, the minimum gain value is 1, that is, the minimum total exposure obtained by multiplying the exposure time by the gain value corresponding to the exposure time must be the product of the exposure time and the gain value of 1. Therefore, it can be determined that the minimum total exposure corresponding to the exposure time of 10ms and the gain value of 4 is 10; the minimum total exposure corresponding to the exposure time of 20ms and the gain value of 8 is 20; the minimum total exposure corresponding to the exposure time of 30ms and the gain value of 8 is 30. In addition, since the total exposure may be less than 10, the minimum total exposure corresponding to the exposure time of 10ms and the gain value of 1 can be determined as 0, so as to facilitate the subsequent determination of the original exposure time.
[0159] Table 1
[0160] Exposure time Gain value Total exposure range 10ms 1 0-10 10ms 4 10-40 20ms 8 20-160 30ms 16 30-480
[0161] In one example, taking the original total exposure as 8 as an example, according to the above Table 1, the candidate total exposure range corresponding to the original total exposure 8 is 0-10, that is, the number of candidate total exposure ranges is 1, and the candidate total exposure range 0-10 can be determined as the target total exposure range. Since the original total exposure 8 is less than the exposure time 10ms corresponding to the target total exposure range, it is necessary to determine the ratio (8 / 1) between the original total exposure 8 and the gain value 1 corresponding to the target total exposure range, and the original exposure time is 8ms. It can be understood that since the minimum gain value can be 1, that is, if the original total exposure is less than the product (10) between the exposure time 10ms and the minimum gain value 1, the original gain value of the image is 1, and therefore, the original exposure time of the image can only be 8ms.
[0162] In one example, taking the original total exposure as 45 as an example, according to the above Table 1, the candidate total exposure range corresponding to the original total exposure 45 is 20-160, that is, the number of candidate total exposure ranges is 1, and the candidate total exposure range 20-160 can be determined as the target total exposure range. Since the original total exposure 45 is greater than the exposure time 20ms corresponding to the target total exposure range, the exposure time 20ms corresponding to the target total exposure range can be determined as the original exposure time, that is, the original exposure time is 20ms.
[0163] In another example, taking the original total exposure as 30 as an example, according to the above Table 1, the candidate total exposure ranges corresponding to the original total exposure 30 are 10-40 and 20-160, that is, the number of candidate total exposure ranges is 2, and it is necessary to determine that the exposure time corresponding to the candidate total exposure range 10-40 is 10ms, and the exposure time corresponding to the candidate total exposure range 20-160 is 20ms, so the exposure time 10ms corresponding to the above candidate total exposure range can be used as the original exposure time. In this way, the occurrence of image overexposure can be reduced and the image quality can be improved.
[0164] It should be noted that the energy cycle of the light source is equivalent to the time required for the light source to strobe once. If the exposure time of the image is less than the time required for the light source to strobe once, the brightness of the image will be different, that is, the image will have fluctuations in light and dark stripes. If the exposure time of the image is an integer multiple of the energy cycle of the light source, it means that the exposure time of the image is an integer multiple of the strobe time of the light source. In this way, the brightness of the image taken by the mobile phone is the same, that is, the image will not have the strobe problem.
[0165] Specifically, when the original exposure time is greater than or equal to the above-mentioned light source energy cycle, there will be no flicker phenomenon in the above-mentioned first image; when the original exposure time is less than the above-mentioned light source energy cycle, there will be flicker phenomenon in the above-mentioned first image, then the original exposure time can be adjusted according to the light source amplitude of the above-mentioned shooting environment to eliminate the flicker phenomenon in the image and improve the image quality.
[0166] How to adjust the original exposure time will be described in detail below.
[0167] Therefore, after obtaining the original exposure time corresponding to the above-mentioned first image, the mobile phone can determine whether the original exposure time is less than the energy cycle of the light source of the mobile phone camera. Among them, the energy cycle of the light source is 0.5 times the AC cycle of the light source of the mobile phone camera. It can be understood that the AC cycle is determined according to the AC frequency. The AC frequency refers to the number of times the AC changes periodically per unit time. Exemplarily, for example, if the AC frequency is 50Hz and the AC cycle is 20ms, then the light source energy cycle is 10ms.
[0168] Exemplarily, the light source energy cycle T of the mobile phone camera can be calculated according to the following formula: T = 1 / (h*2), where h is the AC frequency of the light source of the mobile phone camera. For example, if the AC frequency is 50 Hz, the light source energy cycle T = 1 / (50*2) seconds = 1 / 100 seconds. For another example, if the AC frequency is 60 Hz, the light source energy cycle T = 1 / (60*2) seconds = 1 / 120 seconds.
[0169] When the original exposure time of the first image is less than the energy cycle of the light source, the mobile phone can first determine the standard exposure time of the first image using the amplitude of the light source in the shooting environment, so as to determine the target exposure time using the standard exposure time. The standard exposure time is the critical exposure time corresponding to the absence of flicker in the first image, that is, the image obtained based on the standard exposure time does not have a flicker problem.
[0170] In some embodiments, the standard exposure time of the first image can be determined by the mapping relationship between the first standard exposure time and the first light source amplitude. That is to say, the mobile phone can find the first standard exposure time corresponding to the light source amplitude of the shooting environment based on the mapping relationship between the first standard exposure time and the first light source amplitude, and use the first standard exposure time corresponding to the light source amplitude of the shooting environment as the standard exposure time of the first image. Figure 6As shown in the figure, if the amplitude of the first light source is 1, the standard exposure time is 4ms, that is, if the amplitude of the light source corresponding to the shooting environment is 1, the standard exposure time of the first image can be 4ms. If the amplitude of the first light source is 6, the standard exposure time is 8ms, that is, if the amplitude of the light source corresponding to the shooting environment is 6, the standard exposure time of the first image can be 8ms.
[0171] Understandably, Figure 6 The exposure time corresponding to the points on the fold line shown is the standard exposure time. Since the longer the exposure time, the brighter the image, when the exposure time of the image is the standard exposure time, the image flicker will be eliminated. Then, when the exposure time of the image is greater than the standard exposure time, the image will not have the flicker problem. When the exposure time of the image is less than the standard exposure time, the image flicker may still exist. In other words, the exposure time corresponding to the points above the fold line will eliminate the image flicker phenomenon, while the exposure time corresponding to the points below the fold line will still cause the image to flicker.
[0172] It should be noted that the above Figure 6 The mapping relationship between the first standard exposure time and the first light source amplitude is shown in FIG. 1 and is drawn based on a light source frequency of 50 Hz corresponding to the shooting environment. This is only an example. When the light source frequency is other values, the mapping relationship between the first light source amplitude and the first standard exposure time may change, and this application does not limit it.
[0173] After that, after determining the standard exposure time, the mobile phone can use the standard exposure time and the original exposure time to determine the target exposure time of the first image. For example, if the original exposure time is less than the standard exposure time, it indicates that the first image has a stroboscopic phenomenon, that is, the above-mentioned image to be processed has a stroboscopic phenomenon, and the mobile phone can adjust the exposure time of the first image from the original exposure time to the standard exposure time, where the standard exposure time is the above-mentioned target exposure time, so that when the image has a stroboscopic problem, the image exposure time can be adjusted in time to solve the image stroboscopic problem, improve the image quality, and thus improve the user's visual experience.
[0174] If the original exposure time is greater than or equal to the standard exposure time, it indicates that the first image to be processed does not have a flicker problem, that is, the first image to be processed does not have a flicker problem. Therefore, the mobile phone does not need to adjust the exposure time of the first image. The original exposure time of the first image is the target exposure time of the first image, thereby avoiding unnecessary exposure time adjustment and reducing waste of resources.
[0175] In some embodiments, when the original exposure time is less than the standard exposure time, if the mobile phone directly uses the standard exposure time as the target exposure time, that is, the mobile phone adjusts the exposure time of the first image from the original exposure time to the standard exposure time, the adjusted image may be overexposed. Therefore, in order to ensure image quality, the mobile phone can first determine the magnification of the original exposure time, so as to use the magnification to determine the target exposure time, so that when adjusting the image based on the target exposure time, the adjusted image can be prevented from being overexposed.
[0176] Exemplarily, the magnification of the image represents the minimum increase in the exposure time corresponding to the stroboscopic image to be processed. The mobile phone can determine the magnification of the original exposure time based on the energy cycle of the light source of the mobile phone camera and the original exposure time of the image (such as the first image mentioned above). The following will take the first image as an example to introduce two possible implementation methods for determining the magnification based on the energy cycle of the light source of the mobile phone camera and the original exposure time of the image.
[0177] In one implementation, the mobile phone may divide the light source energy cycle by the original exposure time to obtain an initial multiple, and directly determine the initial multiple as the magnification multiple.
[0178] In another implementation, the mobile phone can determine the maximum adjustment multiple of the original exposure time based on the shooting mode corresponding to the image to be processed, and then determine the magnification multiple of the original exposure time according to the light source energy cycle, the original exposure time and the maximum adjustment multiple.
[0179] Specifically, first, the mobile phone determines the shooting mode corresponding to the above-mentioned first image, that is, the shooting mode corresponding to the above-mentioned image to be processed. Among them, the shooting mode of the image to be processed includes a photo mode and a video recording mode (or described as a video shooting mode). For example, the image to be processed is obtained by the mobile phone in response to the user's second operation, and the second operation is used to trigger the mobile phone to take pictures. The second operation can be a user's click operation on the above-mentioned first control 110. Then the shooting mode corresponding to the image to be processed can be a photo mode. For another example, the image to be processed is obtained by the mobile phone in response to the user's third operation. The third operation is used to trigger the mobile phone to shoot a video. The third operation can be a user's click operation on the above-mentioned second control 120. Then the shooting mode corresponding to the image to be processed can be a video recording mode.
[0180] Afterwards, the mobile phone can determine the maximum adjustment amount of the preset exposure time corresponding to the shooting mode corresponding to the image to be processed, and obtain the maximum adjustment multiple of the original exposure time of the first image. Among them, the maximum adjustment amount of the preset exposure time is pre-set according to the shooting mode of the image to be processed. For example, the maximum adjustment amount of the exposure time corresponding to the video recording mode of the image to be processed is 1.5EV; the maximum adjustment amount of the exposure time corresponding to the shooting mode of the image to be processed is 1EV.
[0181] It can be understood that since the video is composed of multiple frames of images, the flicker problem of the video will be more obvious than that of a single frame of image. Therefore, the maximum adjustment amount of the exposure time of the image corresponding to the video recording mode is higher than the maximum adjustment amount of the exposure time of the image corresponding to the photo mode.
[0182] Then, after determining the maximum adjustment amount of the original exposure time, the mobile phone can determine the maximum adjustment multiple of the original exposure time according to the maximum adjustment amount. Among them, the maximum adjustment multiple refers to the maximum multiple by which the exposure time of the image can be adjusted. It can be understood that the adjustment value (or described as the degree of adjustment) of the exposure time can be limited by the maximum adjustment multiple. If the adjustment degree of the exposure time of the image is too large (such as adjusting from 10ms to 30ms), the probability of overexposure of the image will increase.
[0183] If the exposure time is adjusted too little (such as from 10ms to 12ms), the image flicker problem may still not be eliminated. Therefore, in order to eliminate the flicker problem in the image, the mobile phone needs to adjust the exposure time multiple times. The exposure time adjustment efficiency is low, which wastes adjustment time and adjustment costs, and thus causes unnecessary waste of resources. Therefore, by determining the maximum adjustment multiple, resources can be saved while ensuring image quality.
[0184] In a possible implementation manner, the calculation formula for the maximum adjustment multiple of the original exposure time is as shown in the following formula 2:
[0185] maxRatio=2 ev Formula 2
[0186] Wherein, maxRatio is the maximum adjustment multiple of the original exposure time; ev is the maximum adjustment amount of the original exposure time. For example, if the maximum adjustment amount of the original exposure time is 1, it can be determined that the maximum adjustment multiple of the original exposure time is 2.
[0187] Finally, after determining the maximum adjustment multiple, the mobile phone can determine the magnification of the original exposure time according to the light source energy cycle, the original exposure time and the maximum adjustment multiple of the above-mentioned shooting environment. Exemplarily, the mobile phone can calculate the ratio between the light source energy cycle and the original exposure time to obtain the initial multiple. After determining the initial multiple, the mobile phone can use the minimum value of the initial multiple and the maximum multiple as the magnification of the original exposure time. In this way, the accuracy of the magnification determination can be improved, which can not only reduce the occurrence of overexposure of image files due to excessive exposure time adjustment, but also reduce the occurrence of resource waste due to too small exposure time adjustment, thereby improving the picture quality of the image file.
[0188] Exemplarily, the above magnification can be calculated by the following formula 1:
[0189] ratio = min (light source energy cycle / initialExpo, maxRatio) Formula 1
[0190] Among them, ratio is the magnification; min() is the minimum function; initialExpo is the original exposure time; maxRatio is the maximum adjustment multiple of the original exposure time. For example, taking the light source energy cycle of 10ms, the original exposure time of 4ms, and the shooting mode corresponding to the image to be processed is the photo mode, and the maximum adjustment amount corresponding to the photo mode is 1, the mobile phone can determine the initial multiple as 10 / 4=2.5 based on the light source energy cycle and the original exposure time. Combined with the above formula 2, when the maximum adjustment amount is 1, the maximum adjustment multiple is 2. By comparing the initial multiple (i.e. 2.5) and the maximum adjustment multiple (i.e. 2), the mobile phone can determine that the maximum adjustment multiple is less than the initial multiple, so it can be determined that the magnification is 2.
[0191] The above describes the process of determining the magnification multiple of the original exposure time by the mobile phone. The following will continue to describe the process of determining the target exposure time of the first image using the magnification multiple of the original exposure time.
[0192] After obtaining the magnification of the original exposure time of the first image, the mobile phone can determine the expected exposure time of the first image based on the magnification and the original exposure time. For example, the mobile phone can calculate the product of the magnification and the original exposure time to obtain the expected exposure time of the first image. The expected exposure time refers to the exposure time that is expected to prevent the image from having a stroboscopic problem.
[0193] In a possible implementation, the calculation formula for the expected exposure time is as shown in the following formula 3:
[0194] preExpo=initialExpo*ratio formula three
[0195] Wherein, preExpo is the expected exposure time; initialExpo is the original exposure time; ratio is the magnification. For example, if the original exposure time is 5ms and the magnification is 2, then the expected exposure time can be determined to be 10ms.
[0196] Afterwards, the mobile phone can determine the target exposure time of the first image from the expected exposure time, the standard exposure time, and the original exposure time of the first image. Exemplarily, the mobile phone can use the minimum value of the expected exposure time and the standard exposure time as the first exposure time. It should be understood that no matter whether the exposure time of the first image is adjusted to the standard exposure time or the expected exposure time, the adjusted image does not have a flicker problem.
[0197] Afterwards, the mobile phone can use the maximum value of the first exposure time and the original exposure time as the target exposure time.
[0198] In a possible implementation, the target exposure time is calculated as shown in the following formula 4:
[0199] finalExpo=max(min(preExpo,minExpoLimit),initialExpo)Formula 4
[0200] Among them, finalExpo is the target exposure time; max() is the maximum value function; min() is the minimum value function; preExpo is the expected exposure time; minExpoLimit is the standard exposure time; initialExpo is the original exposure time.
[0201] In one example, if the expected exposure time is 10ms, the standard exposure time is 4ms, and the original exposure time is 5ms, then based on the expected exposure time and the standard exposure time, the mobile phone can determine that the first exposure time is 4ms. Afterwards, the mobile phone can determine that the target exposure time is 5ms based on the first exposure time and the original exposure time. That is, when the standard exposure time is 4ms, the image will no longer have a flicker problem. Since the longer the exposure time, the brighter the image, the image may be overexposed. Therefore, if the image does not have a flicker problem when the exposure time is 4ms, when the original exposure time of the image is 5ms, the image also does not have a flicker problem. Therefore, the mobile phone does not need to adjust the original exposure time of the image. In this way, unnecessary waste of resources can be reduced.
[0202] In another example, if the expected exposure time is 10ms, the standard exposure time is 6ms, and the original exposure time is 5ms, then based on the expected exposure time and the standard exposure time, the mobile phone can determine that the first exposure time is 6ms. Afterwards, the mobile phone can determine that the target exposure time is 6ms based on the first exposure time and the original exposure time. That is, when the standard exposure time is 6ms, the image will no longer have a flicker problem. Since the original exposure time of the image, 5ms, is lower than the standard exposure time, 6ms, it means that the image to be processed has a flicker problem. Therefore, the standard exposure time is determined as the target exposure time, that is, the target exposure time is 6ms. In this way, the flicker problem in the image can be solved.
[0203] It can be understood that when the original exposure time of the first image is greater than or equal to the energy cycle of the light source, it means that there is no flicker problem in the first image, that is, there is no flicker problem in the image to be processed. Therefore, there is no need for the mobile phone to adjust the original exposure time, that is, there is no need to perform the above process of determining the target exposure time of the first image, such as there is no need to execute the above S505 and S506 described below.
[0204] S506: The mobile phone adjusts the exposure time of the first image from the original exposure time to the target exposure time to obtain the target image.
[0205] In a possible implementation, as the exposure time increases, the brightness of the image will also increase. Therefore, after the original exposure time of the first image is adjusted to the target exposure time, the brightness value of the adjusted first image (or second image) may no longer be the target brightness value, and it may be greater than the target brightness value. For example, in a case where the original exposure time is less than the target exposure time, after the exposure time of the first image is adjusted from the original exposure time to the target exposure time, the exposure time of the first image is increased, and the brightness of the adjusted first image may be greater than the target brightness value. Therefore, in order to avoid the adjusted first image being too bright, that is, to avoid the adjusted image being overexposed, the mobile phone can adjust the brightness value of the second image to the target brightness value.
[0206] Exemplarily, the process of adjusting the brightness value of the second image may include: first, the mobile phone may divide the target exposure time by the original exposure time to obtain a target multiple, wherein the target multiple is used to indicate the degree of adjustment of the target exposure time compared to the original exposure time.
[0207] In a possible implementation, the target multiple is calculated as shown in Formula 5 below:
[0208] finalRatio=finalExpo / initialExpo formula five
[0209] Among them, finalRatio is the target multiple, finalExpo is the target exposure time, and initialExpo is the original exposure time. For example, if the target exposure time is 10ms and the original exposure time is 5ms, the target multiple can be determined to be 2.
[0210] Afterwards, the mobile phone can determine the target gain value of the camera according to the target multiple. The target gain value can be used to adjust the brightness of the image so that the image reaches a certain brightness.
[0211] In a possible implementation manner, the target gain value is calculated as shown in the following formula 6:
[0212] gain=1.0 / finalRatioFormula 6
[0213] Wherein, gain is the target gain value, and finalRatio is the target multiple. For example, if the target multiple is 1, the target gain value can be determined to be 1; if the target multiple is 2, the gain value can be determined to be 0.5.
[0214] Correspondingly, after determining the target gain value of the camera, the mobile phone can restore the brightness of the second image according to the target gain value of the camera to obtain the target image, wherein the brightness value of the target image is the target brightness value.
[0215] Specifically, Figure 7 The relationship diagram shown is used to characterize the relationship between input brightness and output brightness. The input brightness refers to the brightness of the adjusted first image (i.e., the brightness of the second image), and the output brightness refers to the brightness of the target image. It can be understood that the gain value is Figure 7 In this way, by adjusting the image brightness through the gain value, the stroboscopic problem can be solved while reducing the probability of overexposure of the target image.
[0216] It should be noted that the process of adjusting the image brightness and the process of determining the target exposure time of the image can be performed in parallel. That is to say, after the mobile phone captures the image, it can perform two tasks at the same time. One task is to adjust the brightness of the image. The other task is to determine the target exposure time of the image. After the image brightness adjustment is completed, the mobile phone can adjust the exposure time of the adjusted image (such as the first image mentioned above) to the target exposure time, thereby realizing rapid processing of the image.
[0217] In some embodiments, after determining the target brightness value of the image to be processed, the mobile phone can directly determine the target exposure time of the image to be processed based on the acquired shooting information corresponding to the image to be processed and the target brightness value. Afterwards, the mobile phone can adjust the exposure time of the image to be processed to the target exposure time.
[0218] The process in which the mobile phone determines the target exposure image of the image to be processed is similar to the process in which the mobile phone determines the target exposure time of the first image, which will not be described in detail here.
[0219] In this embodiment, optionally, after the exposure time of the image to be processed is adjusted to the target exposure time, the mobile phone can continue to adjust the brightness of the adjusted image to be processed, that is, adjust the brightness of the adjusted image to be processed to the target brightness value.
[0220] In some embodiments, after the mobile phone adjusts the exposure time of the first image from the original exposure time to the target exposure time, the mobile phone can directly determine the adjusted first image as the target image, that is, the brightness of the adjusted first image is not adjusted.
[0221] The following will combine the above Figure 4 The structure shown in FIG. 1 introduces how the relevant modules in the mobile phone adjust the exposure time of the image taken by the mobile phone camera to eliminate the stroboscopic phenomenon in the image. Figure 8 As shown, the process is as follows:
[0222] First, when it is necessary to capture an image, the camera (or image sensor) may execute S801, responding to a first operation of the user, to capture an image to be processed. And the stroboscopic detection module may execute S802, responding to the first operation of the user, to detect the light source of the shooting environment to obtain the light source information of the shooting environment. The light source information may include the light source amplitude.
[0223] After obtaining the light source information of the shooting environment, the stroboscopic detection module can also execute S803 to send the light source information of the shooting environment to the AE module. Meanwhile, the camera can execute S804 to send the image to be processed to the brightness adjustment module and the 3A statistics module.
[0224] Afterwards, the 3A statistics module can execute S805, perform feature statistics on the image to be processed, and obtain AE statistics results. Afterwards, the 3A statistics module can also execute S806, and send the AE statistics results to the AE module. Afterwards, the AE module can execute S807, and determine the initial brightness value and target brightness value of the image to be processed according to the AE statistics results. Afterwards, the AE module can also execute S808, and send the initial brightness value and target brightness value of the image to be processed to the brightness adjustment module. Then, the brightness adjustment module can execute S809, and adjust the initial brightness value of the image to be processed to the target brightness value to obtain the first image.
[0225] After obtaining the target brightness value, the AE module may further execute S810, input the target brightness value into the AE algorithm, and obtain the total exposure of the first image. Afterwards, the AE module may execute S811, determine the original exposure time according to the total exposure of the first image in combination with a preset exposure table. Afterwards, the AE module may execute S812, obtain the target exposure time of the first image according to the light source information of the shooting environment and the original exposure time. That is, the target exposure time of the image to be processed is obtained.
[0226] After obtaining the target exposure time of the first image, the AE module can execute S813 to adjust the exposure time of the first image to the target exposure time to obtain a second image (the image is free of flicker problems), but the brightness of the image may change, that is, overexposure or underexposure problems may occur. Therefore, the AE module can execute S814 to send the second image to the brightness adjustment module. Afterwards, the brightness adjustment module can execute S815 to restore the brightness of the second image to obtain the target image. In this way, the target image has neither flicker problems nor the brightness of the target image can be made to be the target brightness value, so that the target image will not be overexposed or underexposed.
[0227] In some embodiments, it is considered that the user may take multiple shots in a shooting environment. The mobile phone can also adjust the exposure time of the camera so that the images captured by the adjusted camera in the same shooting environment will not have a stroboscopic problem. Exemplarily, the AE module can execute S816 and send the above-mentioned target exposure time to the camera. Afterwards, the camera can execute S817 and adjust the exposure time of the camera to the target exposure time, so that when continuing to shoot images in the shooting environment corresponding to the above-mentioned image to be processed, the images captured by the camera (such as preview images or captured images) will not have a stroboscopic problem.
[0228] In some embodiments, by adjusting the light source brightness and light source frequency of the light source in the shooting environment, an example is given to illustrate whether the image captured in the shooting environment adopts the method disclosed in the present application. First, the user can adjust the light source frequency to 0Hz, and then continuously adjust the brightness of the light source and shoot to obtain images of different brightness. The exposure time corresponding to the image of different brightness is between 4ms and 6ms, and the original gain value is 1. It can be understood that the image brightness corresponding to the exposure time between 4ms and 6ms is more appropriate, and there will be no darker or brighter situation. After that, the light source frequency is adjusted to 50Hz (the light source energy cycle is 100Hz), take a picture and check the exposure time of the current image, determine whether the exposure time of the current image has changed, and whether the image has become brighter. It can be understood that if the exposure time becomes larger and the image brightness does not change, it can be explained that the solution not only solves the stroboscopic phenomenon in the image, but also reduces the probability of overexposure or underexposure of the image.
[0229] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step in the above-mentioned method embodiment.
[0230] The embodiment of the present application also provides a computer program product, including a computer program. When the computer program runs on an electronic device, the electronic device executes each function or step in the above method embodiment.
[0231] The embodiment of the present application provides a chip, the chip is used to execute instructions, when the chip is running, the technical solution in the above embodiment is executed. The implementation principle and technical effect are similar, and will not be repeated here.
[0232] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).
[0233] It should be understood that the "embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the various embodiments in the entire specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.
[0234] It should also be understood that in the present application, "when", "if" and "if" all mean that the UE or base station will take corresponding actions under certain objective circumstances. It is not a time limit, and it does not require the UE or base station to make judgments when implementing it, nor does it mean that there are other limitations.
[0235] A person skilled in the art may understand that the various numerical numbers such as first and second involved in the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application, and also indicate the order of precedence.
[0236] In this application, elements expressed in the singular are intended to mean "one or more" rather than "one and only one", unless otherwise specified. In this application, "at least one" is intended to mean "one or more", and "more than one" is intended to mean "two or more", unless otherwise specified.
[0237] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. A can be singular or plural, and B can be singular or plural.
[0238] In this document, the term "at least one of..." or "at least one of..." means all or any combination of the listed items. For example, "at least one of A, B, and C" may mean: A exists alone, B exists alone, C exists alone, A and B exist at the same time, B and C exist at the same time, and A, B, and C exist at the same time, where A may be singular or plural, B may be singular or plural, and C may be singular or plural.
[0239] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0240] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0241] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0242] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0243] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0244] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.
[0245] The same or similar parts between the various embodiments in this application can refer to each other. In the various embodiments in this application, and the various implementation methods / implementation methods / implementation methods in each embodiment, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment are consistent and can be referenced to each other. The technical features in different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment can be combined to form new embodiments, implementation methods, implementation methods, or implementation methods according to their inherent logical relationships. The above-described implementation methods of this application do not constitute a limitation on the scope of protection of this application.
[0246] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims. In short, the above is only a preferred embodiment of the technical solution of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. An image processing method, characterized in that: The method comprises: The electronic device obtains an image to be processed taken by a camera in the electronic device and a target brightness value of the image to be processed; wherein the target brightness value indicates the brightness that the image to be processed should achieve; The electronic device determines the target exposure time of the image to be processed based on the shooting information corresponding to the image to be processed and in combination with the target brightness value; wherein the shooting information includes light source information and light source energy cycle of the shooting environment; the light source information includes light source amplitude, and the light source amplitude is used to determine the degree of adjustment of the exposure time of the image to be processed; Wherein, the target exposure time is determined by the maximum value of the minimum exposure time and the original exposure time after the electronic device selects the minimum exposure time from the expected exposure time and the standard exposure time; the standard exposure time is the critical exposure time corresponding to the absence of stroboscopic light in the image to be processed, the original exposure time represents the exposure time of the image to be processed when the brightness value of the image to be processed is the target brightness value, the expected exposure time is determined based on the product of the magnification of the image to be processed and the original exposure time, the magnification is the minimum value selected by the electronic device from the candidate adjustment multiples and the maximum adjustment multiple of the original exposure time, the candidate adjustment multiple is the ratio between the energy cycle of the light source and the original exposure time, and the maximum adjustment multiple refers to the maximum multiple by which the exposure time of the image to be processed can be increased; The electronic device adjusts the exposure time of the image to be processed to the target exposure time to obtain a target image.
2. The method according to claim 1, characterized in that The light source information also includes a light source frequency. The determining of a target exposure time of the image to be processed based on the shooting information corresponding to the image to be processed and in combination with the target brightness value includes: When the light source frequency is not zero, the target exposure time of the image to be processed is determined based on the light source amplitude and in combination with the target brightness value.
3. The method according to claim 1, characterized in that The method further comprises: The electronic device determines the original exposure time of the image to be processed by using the original total exposure in combination with a preset exposure table; the original total exposure is obtained by inputting the target brightness value into an automatic exposure algorithm.
4. The method according to claim 1, characterized in that: The magnification of the original exposure time is determined by the following formula 1: ratio = min (light source energy cycle / initialExpo, maxRatio), Formula 1; Wherein, the ratio is the magnification multiple of the original exposure time, the initialExpo is the original exposure time, and the maxRatio is the maximum adjustment multiple of the original exposure time; The maxRatio can be determined by Formula 2: maxRatio=2 ev , Formula 2; Among them, the ev is the maximum adjustment amount of the original exposure time, and the maximum adjustment amount is determined based on the shooting mode corresponding to the image to be processed, and the shooting mode includes a photo mode and a video recording mode.
5. The method according to any one of claims 1 to 4, characterized in that The step of adjusting the exposure time of the image to be processed to the target exposure time to obtain the target image includes: The electronic device adjusts the exposure time of the first image corresponding to the image to be processed to the target exposure time to obtain a second image; the first image is an image obtained by adjusting the brightness value of the image to be processed to the target brightness value; The electronic device calculates the ratio between the target exposure time and the original exposure time to obtain a target gain value of the camera; Based on the target gain value of the camera, the brightness of the second image is restored to obtain the target image, and the brightness value of the target image is the target brightness value.
6. An electronic device, characterized in that: The electronic device includes a display screen, a camera, a memory and one or more processors; the display screen, the camera, the memory and the processor are coupled; the camera is used to capture images, the display screen is used to display images generated by the processor and images captured by the camera, and the memory is used to store computer program codes, and the computer program codes include computer instructions; when the processor executes the computer instructions, the electronic device executes the method as described in any one of claims 1 to 5.
7. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 5.
