Shooting control method and electronic equipment

Abstract

The embodiment of the invention provides a shooting control method and electronic equipment, and relates to the technical field of terminals. When a large screen with drifting light source frequency is shot, a banding phenomenon is improved. According to the specific scheme, after multiple first light source frequencies from a target large screen are detected, a first image is collected; wherein the exposure duration of the first image is a first duration; the plurality of first light source frequencies comprise a plurality of frequency multiplications of a preset frequency, and the first duration is equal to the period duration of the frequency multiplications of the preset frequency; detecting a plurality of second light source frequencies from the target large screen after the clock of the target large screen is shifted; wherein the plurality of second light source frequencies do not contain frequency multiplication of the preset frequency; determining that the plurality of second light source frequencies are light source frequencies generated after frequency drift of the target large screen occurs; acquiring a second image; wherein the exposure duration of the second image is the first duration.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a shooting control method and electronic device. Background Technology

[0002] In daily life, we often encounter scenarios where large screens or projectors are used to display important information. For example, large screens outside high-speed rail stations display scrolling train schedules. Another example is large screens embedded in the exterior walls of shopping malls displaying advertising information. Yet another example is using projectors to display online meeting footage.

[0003] When important information is displayed on a large screen, or when important information is displayed on a projector screen, users often use electronic devices (such as mobile phones) to take pictures of the large screen (or the projected image) in order to save the important information displayed on the large screen (or the projected image).

[0004] In real-world scenarios, such as when shooting a large screen (or projector) where the light source frequency has drifted, poor shooting results may occur. For example, in the preview stream displayed in the shooting preview interface or the recording stream displayed in the video recording interface, scrolling bright and dark stripes may appear on the image area of ​​the large screen (or projected image), a phenomenon known as banding. Summary of the Invention

[0005] This application provides a shooting control method and electronic device for improving banding when shooting large screens where the light source frequency has drifted.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] Firstly, embodiments of this application provide a shooting control method applicable to scenarios involving shooting a target large screen. For example, an electronic device opens an application with shooting functionality and points the camera at the target large screen. The electronic device then begins acquiring a preview stream or video stream containing the target large screen.

[0008] The aforementioned target screen corresponds to a large screen with multiple light sources, each with its own frequency. When the target screen's clock remains constant, the electronic device can detect multiple primary light source frequencies emanating from the screen, including multiple harmonics of a preset frequency. Based on the detected primary light source frequencies, the electronic device can identify the current shooting scene as a large screen capture scenario. In this scenario, the electronic device can effectively eliminate banding by employing a banding elimination strategy tailored to the large screen shooting scenario. For example, the exposure time can be configured as the period of a harmonic of the preset frequency, i.e., configured as the primary exposure time. Subsequently, with the exposure time set to the primary exposure time, the captured first image will be unaffected by the multiple primary light source frequencies, thus eliminating banding.

[0009] After the clock on the target screen shifts, the actual light source frequency generated by the target screen will also drift. In this scenario, if the electronic device continues to capture images of the target screen, it can detect multiple second light source frequencies from the target screen. These multiple second light source frequencies are different from the multiple first light source frequencies and do not contain harmonics of the preset frequency. The electronic device can determine whether the multiple second light source frequencies are light source frequencies generated after the target screen's frequency drift. If it is identified that the multiple second light source frequencies are light source frequencies generated after the target screen's frequency drift, it can be determined that the current shooting scenario is also a large screen shooting scenario, and the electronic device is triggered to follow the banding elimination strategy for large screen shooting scenarios. For example, when capturing the second image, it continues to expose for the first duration to eliminate the banding phenomenon generated by the multiple second light source frequencies. In this way, the same exposure duration will be used for image acquisition before and after the target screen's clock shift, ensuring the shooting effect.

[0010] In the above embodiments, by identifying whether the detected light source frequency is the frequency generated after the target large screen generates frequency drift, and after identifying the frequency drift, continuing to use the banding elimination strategy frequency for large screen shooting, the banding phenomenon is accurately and appropriately eliminated, avoiding the misselection of other banding elimination strategies, which would affect the banding elimination effect, and improving the shooting effect stability of electronic devices.

[0011] In some embodiments, before acquiring the second image, the electronic device can detect the frequency of a third light source present in its environment. The third light source frequency is the frequency of all light sources present in the environment of the electronic device, that is, it is not limited to light sources from the target screen. The third light source frequency includes multiple second light source frequencies. After detecting the third light source frequency, the electronic device can determine a target harmonic frequency corresponding to each third light source frequency from multiple harmonics of a preset frequency. The target harmonic frequency corresponding to the third light source frequency is the harmonic with the smallest difference from the third light source frequency among the multiple harmonics of the preset frequency. The target harmonic frequencies for different third light source frequencies may be the same or different.

[0012] The electronic device can sequentially calculate the relative offset coefficient between each third light source frequency and the target harmonic, as well as the average offset coefficient between the relative offset coefficients of all third light source frequencies.

[0013] For example, the aforementioned relative offset coefficient can represent the proportion of the deviation between the frequency of the third light source and the target harmonic. For instance, it can be based on the frequency of the third light source and the target harmonic, combined with the formula:

[0014]

[0015] Calculate the relative offset coefficient of the third light source frequency. Where a is the relative offset coefficient, λ is the frequency of the third light source, and θ is the target harmonic.

[0016] Understandably, the detected third light source frequency includes the second light source frequency. Through the above steps, the relative offset coefficient of the second light source frequency can be determined. Furthermore, the determined average offset coefficient is also related to the second light source frequency. If the difference between the relative offset coefficient and the average offset coefficient of the second light source frequency is less than a preset threshold, it can be determined that the second light source frequency is a light source frequency generated after frequency drift occurs on the target screen.

[0017] In the above embodiments, by taking advantage of the fact that the relative offset coefficients between the light source frequencies after frequency drift are relatively close, the light source frequency generated by the frequency drift of the target large screen can be quickly and accurately identified from the detected third light source frequency. In this way, it can be accurately determined whether the current shooting scene is a large screen shooting scene, so as to accurately decide on the banding removal strategy to be used and improve the banding removal effect for large screen shooting scenes.

[0018] In some embodiments, after determining that the multiple second light source frequencies are light source frequencies generated after frequency drift occurs on the target large screen, the electronic device corrects the frequency value of the second light source frequency to the corresponding target harmonic frequency value. The target harmonic frequency of the second light source frequency is the harmonic with the smallest difference from the second light source frequency among a plurality of harmonics of a preset frequency.

[0019] In the above embodiments, after identifying the light source frequency caused by frequency drift of the target large screen, the detected light source frequency is calibrated by correcting its frequency value. In this way, even without modifying the program file used to decide whether to use the banding elimination strategy for the large screen shooting scenario, the electronic device can accurately enable the banding elimination strategy for the large screen shooting scenario when the target large screen shows frequency drift, which can also improve the stability of the program file in the electronic device.

[0020] In some embodiments, the process of detecting the frequency of a third light source present in the environment is as follows: An electronic device can control a scintillation sensor to acquire raw sequence data. The raw sequence data is processed by a Fast Fourier Transform to obtain spectral data, which includes spectral information of multiple light source frequencies, such as frequency values ​​and amplitudes.

[0021] The electronic device then identifies N third light source frequencies from the spectrum data. The amplitudes of the N third light source frequencies are greater than the amplitudes of the other light source frequencies in the spectrum data, where N is a positive integer.

[0022] In the above embodiments, by analyzing the frequency of light sources with larger amplitudes, the influence of some interfering light source frequencies is avoided, thereby improving the accuracy of subsequent frequency drift determination.

[0023] Since the third light source frequency comes from all light sources in the environment, it also includes the fourth light source frequency, which is not caused by frequency drift. When processing the fourth light source frequency, if the difference between the relative offset coefficient of the fourth light source frequency and the average offset coefficient is greater than a preset threshold, the electronic device will not modify the frequency value of the fourth light source frequency; that is, it will not calibrate the fourth light source frequency.

[0024] In the above embodiments, it is possible to avoid erroneous modification of the light source frequency caused by non-frequency drift. This also avoids the problem of the inability to improve the banding phenomenon caused by the fourth light source frequency.

[0025] Additionally, when a fourth light source frequency exists within the third light source frequency, during actual banding elimination, the exposure duration can be the least common multiple between the period duration of the fourth light source frequency and the period duration of a multiple of the preset frequency. Alternatively, the period duration of a multiple of the preset frequency can be used as the exposure duration, and the frame interval of the electronic device can be adjusted according to the fourth light source frequency to fix the display position of the bright and dark stripes generated by the fourth light source frequency during image exposure in the preview or recording stream.

[0026] In some embodiments, after correcting the frequency values ​​of all the second light source frequencies to the corresponding target multiples, the electronic device can identify that the current scene is a large screen being filmed based on the third light source frequencies. For example, if the multiples of preset frequencies in the third light source frequencies are the majority, the current scene may be a large screen being filmed. Or, if all the third light source frequencies are multiples of preset frequencies, the current scene may be a large screen being filmed.

[0027] In other embodiments, the electronic device may further perform frequency doubling and / or sidelobe reduction on the calibrated third light source frequency. This reduces interference frequencies and improves the effectiveness of subsequent banding elimination.

[0028] In some embodiments, the electronic device may further sort the processed third light source frequencies in descending order of amplitude. If the third light source frequency ranked first is a multiple of a preset frequency, the frequency value of the third light source frequency ranked first is replaced with the frequency value of the preset frequency. If the third light source frequency ranked first is not a multiple of the preset frequency, the frequency value of the third light source frequency ranked second is replaced with the frequency value of the preset frequency.

[0029] In this way, during the subsequent actual elimination of banding, while retaining the priority of the light source frequency with the largest amplitude, the importance of the light source frequency with a frequency value equal to the preset frequency is also increased, thereby improving the banding phenomenon caused by multiple light source frequencies.

[0030] In some embodiments, the electronic device may acquire a third image before acquiring the first image. For example, the third image may be the image data of a previous frame acquired by the image sensor before acquiring the first image. The maximum exposure time can then be determined based on the brightness information of the third image.

[0031] As mentioned earlier, the first duration can be the period duration of a frequency multiple of a preset frequency. That is, without any restrictions, the first duration can have multiple possible values. For example, the period duration of the preset frequency, the period duration of a second harmonic of the preset frequency, the period duration of a third harmonic of the preset frequency, etc. The smaller the harmonic value, the more suitable the corresponding period duration is as the exposure duration, and the better it is at eliminating banding. In other words, the period duration of the preset frequency is the optimal choice. In this embodiment, considering that the actual configurable exposure duration will be affected by the maximum exposure duration, it is necessary to determine whether the period duration of the preset frequency is greater than the maximum exposure duration before acquiring the first image. If it is not greater than the maximum exposure duration, the period duration of the preset frequency is used as the first duration, and the exposure duration of the image sensor is configured as the first duration.

[0032] In this way, the banding phenomenon caused by the light source frequency is eliminated, and the problem of image overexposure will not occur.

[0033] If the period duration of the preset frequency is greater than the maximum exposure duration, the first duration is determined to be equal to the period duration of L times the preset frequency, where L is an integer multiple of 1. This way, in high-brightness environments, banding caused by most light source frequencies is eliminated while avoiding image overexposure.

[0034] In some embodiments, the L-fold can be an even-numbered frequency. This way, when exposing image data using an L-fold of a preset frequency, the banding phenomenon caused by even-numbered frequencies of the preset frequency can be eliminated. The effect of eliminating banding is more pronounced when even-numbered frequencies of the preset frequency predominate among a plurality of first light source frequencies.

[0035] In other embodiments, the L-frequency can be an odd-numbered frequency. This way, when exposing image data using an L-frequency of a preset frequency, the banding phenomenon caused by odd-numbered frequencies of the preset frequency can be eliminated. The effect of eliminating banding is more pronounced when odd-numbered frequencies of the preset frequency predominate among a plurality of first light source frequencies.

[0036] Furthermore, the method for determining the required exposure time after multiple second light source frequencies are calibrated is the same as the method for selecting the first duration based on multiple first light source frequencies. In this way, the exposure time of the captured second image is also the same as that of the first image.

[0037] In a second aspect, an electronic device is provided in the embodiments of this application. The electronic device includes an image sensor, a flicker sensor, one or more processors, and a memory. The image sensor is used to acquire raw image data of image frames, and the flicker sensor is used to detect the light source frequency corresponding to the light source in the environment. The memory is coupled to the processor and is used to store computer program code, which includes computer instructions. When one or more processors execute the computer instructions, the one or more processors are used to perform the methods described in the first aspect and its possible embodiments.

[0038] Thirdly, embodiments of this application provide a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the first aspect and its possible embodiments.

[0039] Fourthly, this application provides a computer program product that, when run on the aforementioned electronic device, causes the electronic device to perform the methods described in the first aspect and its possible embodiments.

[0040] Understandably, the electronic devices, computer storage media, and computer program products provided in the above aspects are all applied to the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0041] Figure 1 A schematic diagram of a large screen shooting scene provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram illustrating the principle of banding phenomenon formation in an embodiment of this application;

[0043] Figure 3 This is a schematic diagram illustrating the principle of banding elimination in an embodiment of this application;

[0044] Figure 4 Example diagrams of the hardware and software architecture of the electronic device provided in the embodiments of this application;

[0045] Figure 5 Signaling interaction diagram for an electronic device performing a shooting control method provided in an embodiment of this application;

[0046] Figure 6 Here is a flowchart of the sub-steps of S103;

[0047] Figure 7 A schematic diagram illustrating the principle of calibrating the light source frequency provided in an embodiment of this application;

[0048] Figure 8 A flowchart illustrating the generated frequency output results provided in this application embodiment;

[0049] Figure 9 This is an example diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0050] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0051] In daily life, users often use electronic devices to capture images of large screens or projectors to record the information displayed. For example... Figure 1As shown, when a user needs to photograph the large screen 101 set up outdoors, they can open an application with shooting function (such as a camera app) on the mobile phone 102. After the mobile phone 102 displays the shooting preview interface 103 provided by the camera app, the user points the camera of the mobile phone 102 at the large screen 101. In this way, the mobile phone 102 can capture a preview stream containing the large screen 101 and display the image frames in the preview stream, such as image data 104, on the shooting preview interface 103.

[0052] In possible scenarios, after opening the camera app, phone 102 can respond to user input by enabling the camera app's recording function and displaying the corresponding recording interface. Similar to displaying a shooting preview interface, the recording interface can also display the recording stream including the large screen 101.

[0053] In addition, some users also take photos with the projected image from a large screen or projector. Both large screens and projectors have at least one light source. This light source can include the device's own light source (large screen or projector) or light sources in the environment that provide illumination.

[0054] The energy of each light source varies with a fixed frequency, and correspondingly, the brightness of the light source also varies with the fixed frequency, resulting in flickering. This fixed frequency can be understood as the light source frequency. At different light source frequencies, the corresponding flicker period is different. The duration of the flicker period (also called the flicker cycle duration) is equal to the reciprocal of the light source frequency.

[0055] When electronic devices capture images of large screens or projections, the exposure duration (or exposure time) needs to be set to an integer multiple of the flicker period of the light source on the large screen (or projector). Otherwise, as... Figure 1 As shown, in image data 104, bright and dark stripes will appear in the image area 105 of the large screen 101.

[0056] To clearly illustrate the solutions mentioned in the embodiments of this application, the formation principle of light and dark stripes is described below:

[0057] Taking a large screen with a light source connected to 50Hz AC power as an example, the large screen performs signal conversion on the light source to obtain, for example... Figure 2 The optical signal waveform shown is a periodically changing envelope with a frequency of 100Hz. Let T be the duration of the flicker period of the light source, then T = 1 / 100s = 10ms. Additionally, the exposure time of the image sensor (i.e., CMOS module) of the electronic device is denoted as t, where the exposure time can refer to the time required for the CMOS module to expose each row of pixels.

[0058] The light emitted from the large screen illuminates the CMOS module, causing the corresponding pixels to generate light energy waveforms corresponding to the light signal waveform in the time direction. Pixels in the same row have the same exposure start time and exposure duration, and pixels belonging to image region 105 in the same row receive the same amount of light energy. Pixels in different rows, although having the same exposure duration, have different exposure start times; therefore, pixels belonging to image region 105 in different rows may receive different amounts of light energy.

[0059] Continue as Figure 2 As shown, the exposure time t of the CMOS module in the electronic device is not an integer multiple of the flicker period of the light source, such as t = T + t1. During the exposure of the i-th row of pixels by the CMOS module of the electronic device, the light energy received by the pixels belonging to image region 105 in the i-th row is S + S1, where S can represent the light energy received by the pixels belonging to image region 105 in the i-th row within time T, and S1 can represent the light energy received by the pixels belonging to image region 105 in the i-th row within time t1. During the exposure of the (i+1)th row of pixels in the CMOS module of the electronic device, the light energy received by the pixels belonging to image region 105 in the (i+1)th row is S2+S3+S4. Here, S2 can represent the light energy received by the pixels belonging to image region 105 in the (i+1)th row during time t4, S3 can represent the light energy received by the pixels belonging to image region 105 in the (i+1)th row during time t2, and S4 can represent the light energy received by the pixels belonging to image region 105 in the (i+1)th row during time t3.

[0060] exist Figure 2 In the given information, t1 = t2 = t3, such as... Figure 2 As shown, according to the principle of light signal integration over time, S1 = S3, S1 < S4, S2 + S3 = S, therefore, S + S1 < S2 + S3 + S4. That is, during the exposure of image data 104, the pixels in the i-th row belonging to image region 105 receive less light energy than the pixels in the (i+1)-th row belonging to image region 105. Thus, the pixels in the i-th row belonging to image region 105 are darker than those in the (i+1)-th row belonging to image region 105.

[0061] Understandably, each frame in the preview stream containing the image frame of the large screen 101 will exhibit bright and dark stripes similar to those in image data 104. Furthermore, the positions of these stripes will change depending on the exposure of the image data. Thus, during the display of the preview stream, scrolling stripes will appear in the shooting preview interface, i.e., a banding phenomenon. The recording stream behaves similarly, and will not be elaborated upon here.

[0062] In electronic devices, the exposure time t of a CMOS module is an integer multiple of the flicker period of the light source, for example, Figure 3 As shown, when the exposure time t equals T, when the CMOS module of the electronic device exposes the pixels in the i-th row, the light energy received by the pixels belonging to image region 105 in the i-th row is S. When the CMOS module of the electronic device exposes the pixels in the (i+1)-th row, the light energy received by the pixels belonging to image region 105 in the (i+1)-th row is also S. Thus, the pixels belonging to image region 105 in the i-th row have the same brightness as the pixels belonging to image region 105 in the (i+1)-th row.

[0063] In this way, no stripes will appear in the image data 104. Correspondingly, during the display of the preview stream, no scrolling stripes will appear in the shooting preview interface, thus eliminating banding.

[0064] Large-screen (or projector) devices have specific light source frequencies. For example, a large-screen device may have multiple light source frequencies, each a multiple of a preset frequency, which can be referred to as the first light source frequency. For instance, multiples of the preset frequency (e.g., 1, 2, 3 times, etc.). The preset frequency could be 60Hz. For large-screen shooting scenarios, electronic devices can employ the following strategies to eliminate banding:

[0065] (1) During the shooting process, the exposure time can be set to the period of the preset frequency.

[0066] (2) Calculate the least common multiple between the period duration of the preset frequency and the period duration of other detected frequencies (multiples of the non-preset frequency), and set the exposure duration to the above least common multiple.

[0067] The strategies described above can all be considered strategies for large-screen shooting scenarios. Their main principle is to prioritize eliminating bright and dark banding caused by harmonics of preset frequencies. That is, after determining to enable the strategy for large-screen shooting scenarios, the electronic device can prioritize selecting the period of a harmonic of the preset frequency as the actual exposure time. For example, the exposure time can be configured as a first duration, which is 1 / Z of the period of the preset frequency, where Z is a positive integer greater than zero. With the first exposure time, banding caused by multiple first light source frequencies can be eliminated. In this scenario, the captured first image is unaffected by the multiple first light source frequencies.

[0068] In summary, strategies for shooting on large screens are not limited to the two strategies listed above, and may include other strategies that conform to the same principles. Furthermore, strategies for shooting on large screens can be embedded in electronic devices in the form of program data.

[0069] Of course, electronic devices are also equipped with strategies for shooting scenarios outside of large screens. Most of these strategies operate on the principle of prioritizing the elimination of bright and dark stripes caused by the light source frequency with the largest amplitude. For example, strategies for shooting scenarios outside of large screens might include the following:

[0070] 1) During the shooting process, a light source frequency is detected, and the exposure time is configured to be an integer multiple of the period of that light source frequency.

[0071] 2) During the shooting process, if two light source frequencies are detected, the least common multiple of the period lengths of the two light source frequencies can be calculated. Then, the exposure time is configured to the calculated least common multiple.

[0072] If the calculated least common multiple is unsuitable as the exposure duration—for example, if the least common multiple is too large and its use as the exposure duration would lead to overexposure—then the light source frequency 'a' with the largest amplitude is determined from the two light source frequencies, and the exposure frequency is configured to be the period duration of light source frequency 'a'. Then, the inter-frame interval between two adjacent image frames is adjusted; that is, by using a frequency-dynamic frame rate, the positions of the bright and dark stripes generated by light source frequency 'b' are fixed in the continuously acquired image data. In this way, even if bright and dark stripes caused by light source frequency 'b' appear in the preview or recording stream acquired by the electronic device, the stripes will not scroll.

[0073] 3) During the shooting process, if multiple light source frequencies are detected, the exposure time can be configured to be the period of the light source frequency with the largest frequency amplitude.

[0074] Most current electronic devices determine whether to activate a strategy for large-screen shooting scenarios by judging whether the detected light source frequency includes multiple harmonics of a preset frequency. For example, if the detected light source frequency includes multiple harmonics of the preset frequency, a strategy for large-screen shooting scenarios is adopted to eliminate banding. For example, if the detected light source frequency does not include multiple harmonics of the preset frequency, a strategy for non-large-screen shooting scenarios is adopted to eliminate banding.

[0075] Correspondingly, most electronic devices are pre-configured with multiple preset frequencies, which can be obtained by detecting light source signals from a large number of various types of large screens or projectors.

[0076] In some embodiments, after a portion of the large screen (or projector) is put into use, clock shift may occur, causing the actual light source frequency generated by the large screen's light source to drift relative to the original light source frequency.

[0077] For example, the original light source frequency of a large screen is 60Hz, 120Hz, and 240Hz. Pre-configured frequencies in electronic devices include 60Hz.

[0078] Before the clock on the large screen shifts, the actual light source frequencies generated by the large screen are 60Hz, 120Hz, and 240Hz, etc. In this scenario, when the electronic device captures the large screen, it also detects light source frequencies of 60Hz, 120Hz, and 240Hz, meaning it detects multiple primary light source frequencies. The electronic device can identify that the detected light source frequencies include multiple harmonics of the preset frequency (60Hz), and thus determine to activate a strategy for the large screen shooting scenario. That is, the electronic device can configure the exposure time to the period length of the preset frequency (60Hz) (16.67ms). Understandably, since 16.67ms is 1 times the period length of 60Hz, 2 times the period length of 120Hz (8.33ms), and 4 times the period length of 240Hz (4.16ms), configuring the exposure time to 16.67ms can eliminate the banding phenomenon caused by light source frequencies such as 60Hz, 120Hz, and 240Hz.

[0079] However, after the clock shift of the large screen, the actual light source frequencies generated are 60.5Hz, 121Hz, and 242Hz, etc. In this scenario, when the electronic device captures the large screen, the detected light source frequencies are also 60.5Hz, 121Hz, and 242Hz, meaning multiple secondary light source frequencies are detected. The preset frequency configured in the electronic device includes 60Hz but not 60.5Hz. Thus, among 60.5Hz, 121Hz, and 242Hz, the harmonics of the preset frequency (60Hz) cannot be identified, and therefore, the strategy for non-large screen shooting scenarios is activated. If the light source frequency with the largest amplitude among 60.5Hz, 121Hz, and 242Hz is 242Hz, then after activating the strategy for non-large screen shooting scenarios, the electronic device will configure the exposure time to 4.13ms. Clearly, 4.13ms is not an integer multiple of the period length of light source frequencies such as 60.5Hz and 121Hz. Therefore, setting the exposure time to 4.13ms cannot eliminate the banding phenomenon caused by light source frequencies such as 60Hz and 120Hz. This results in a deterioration in the shooting effect when shooting large screens.

[0080] To address the aforementioned issues, this application provides a shooting control method applied to an electronic device. In a scenario where the electronic device is shooting a large screen, the frequency of light sources in the surrounding environment is detected. Then, it is determined whether the detected light source frequencies include frequencies that drift relative to a preset frequency (multiples of the preset frequency). If a light source frequency that drifts relative to the preset frequency is detected, it is calibrated to a multiple of the preset frequency. Based on the calibrated light source frequency, a multiple containing the preset frequency is determined. Subsequently, a strategy for large screen shooting scenarios is triggered to maximize the elimination of banding.

[0081] For example, electronic devices can be desktops, laptops, tablets, handheld computers, mobile phones, laptops, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), televisions, VR devices, AR devices, and other devices with cameras.

[0082] Figure 4 This is a schematic diagram of the hardware and software architecture (including software system and some hardware) of the electronic device in an embodiment of this application. Figure 4 As shown, the application architecture is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into multiple layers, from top to bottom: application layer, application framework layer, hardware abstraction layer (HAL), driver layer, and hardware layer.

[0083] The application layer can include a series of application packages. For example... Figure 4 As shown, this is an application package that includes applications such as camera apps and gallery apps.

[0084] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0085] For example, such as Figure 4 As shown, the application framework layer may include a camera access interface, which can include camera management and camera devices. The camera access interface is used to provide application programming interfaces and programming frameworks for camera applications.

[0086] The Hardware Abstraction Layer (HAL) is an interface layer located between the operating system kernel and upper-level software. Its purpose is to abstract hardware and provide a virtual hardware platform for the operating system. The HAL is an abstract interface for device kernel drivers, used to provide application programming interfaces (APIs) for accessing the underlying devices to higher-level Java API frameworks. The HAL contains multiple library modules, such as the Camera Hardware Abstraction Layer (Camera HAL). Each library module implements an interface for a specific type of hardware component. When the system framework layer API requests access to the portable device's hardware, the operating system loads the library module for that hardware component.

[0087] In this embodiment, the hardware abstraction layer may further include a camera algorithm library. The camera algorithm library includes a flicker frequency output module and an auto exposure (AE) module.

[0088] The flicker frequency output module is used to execute the shooting control method provided in this application embodiment to determine the light source frequency of the shooting object, and output the determined light source frequency to the AE module. Details regarding the determination of the light source frequency of the shooting object can be found in subsequent implementations and will not be elaborated here.

[0089] The AE module can be used to automatically adjust exposure parameters. These parameters include, but are not limited to, shutter speed (or exposure duration).

[0090] The basic principle behind automatic exposure in an After Effects (AE) module includes adjusting exposure and gain based on the current brightness of the image to stabilize the brightness within a suitable range. Specifically, the AE module improves the contrast between light and dark banding in the image by adjusting the exposure time to a target exposure frequency determined based on the maximum exposure time and the light source frequency.

[0091] Understandably, the hardware portion of the aforementioned AE module can be referred to as the AE statistics module, which can reside within the image signal processor (ISP) at the hardware layer, and is also known as ISP stat3A. The software portion of the aforementioned AE module can be referred to as the AE processing module, and is located within the camera algorithm library.

[0092] In some embodiments, the flicker frequency output module and / or the automatic exposure module may be arranged in the camera hardware abstraction layer.

[0093] The driver layer is used to drive hardware resources. The driver layer can include multiple driver modules. For example... Figure 4 As shown, the driver layer includes camera device drivers, etc. Of course, it can also include... Figure 4Digital signal processor drivers and graphics processor drivers, etc., are not shown.

[0094] The hardware layer includes sensors, ISPs, etc. Among them, sensors include image sensors, flicker sensors, etc.

[0095] The following, with reference to the accompanying drawings, uses a scenario where a user uses an electronic device to photograph a large screen (such as the target screen) as an example to illustrate the implementation details of the method provided in this application:

[0096] In some embodiments, the electronic device can respond to user input by opening a camera application. The electronic device can then display a shooting preview interface provided by the camera application, and control the image sensor to power on and acquire image frames. During this process, the electronic device also detects the frequency and amplitude information of the light source corresponding to the subject being photographed (i.e., the large screen). After detecting multiple light source frequencies, the detected light source frequencies are calibrated. Then, based on the calibrated light source frequencies, a decision is made on whether to enable a strategy for large screen shooting scenarios.

[0097] As one implementation method, electronic devices include camera applications, image sensors, afterimage (AE) modules, flicker detection modules, and image signal generators (ISPs). The hardware portion of the AE module, often called the AE statistics module, possesses powerful computing capabilities and can statistically analyze the brightness information of image data (e.g., brightness histograms, brightness region statistics, etc.). The software portion of the AE module, often called the AE processing module, integrates various processing strategies for AE. For example... Figure 5 As shown, when an electronic device opens the camera application and executes the shooting control method, the interaction between the various software and hardware modules is as follows:

[0098] S101, the camera application controls the image sensor to power on.

[0099] In some embodiments, when an electronic device detects a user instruction to open an application with shooting capabilities (e.g., a camera app), it can run the camera app in the foreground. After the camera app runs in the foreground, it can call the camera HAL through the camera access interface and create a camera control channel corresponding to the image sensor through the camera HAL.

[0100] For example, in response to a user opening the camera application or another application invoking the camera application, the camera application at the application layer can send a shooting request to the camera access interface at the application framework layer. The shooting request can include parameters such as the camera identifier (ID) corresponding to the current shooting scene, the frame rate range (including maximum and minimum frame rates), and the shooting mode. The camera ID indicates the image sensor that needs to be activated for image acquisition. The camera access interface can directly transmit the shooting request to the camera HAL in the HAL layer. After receiving the shooting request, the camera HAL can create a corresponding camera control channel for the image sensor indicated by the camera ID. Then, the camera HAL can send the camera ID and other parameters corresponding to the current shooting scene to the camera device driver through the camera node.

[0101] The camera control channel includes various software nodes. For example, the camera control channel includes a control node for controlling the image sensor, such as a camera node. The camera HAL can control the image sensor to power on, control the image sensor to start current, and acquire an image frame according to a default exposure duration via the camera node and camera device driver, then proceed to step S103. The default exposure duration can be a pre-configured fixed value or an empirical value; this embodiment does not specifically limit its value.

[0102] Optionally, the camera control channel includes processing nodes for processing image data acquired by the image sensor, such as an image front end (IFE) node. Optionally, the camera control channel may also include control nodes for related accessories of the image sensor (such as a scintillator sensor), which are not specifically limited in this embodiment.

[0103] S102, the camera application control flicker detection module is powered on.

[0104] In some embodiments, there is no necessary order between S101 and S102; for example, they can be executed simultaneously.

[0105] For example, the flicker detection module includes a hardware component (i.e., a flicker sensor) and a software component (i.e., a flicker frequency output module). After the camera application calls the camera access interface to instruct the camera HAL to create a camera control channel, the camera HAL can control the flicker sensor in the flicker detection module to power on. After the flicker sensor is powered on, it begins to detect the light source frequency corresponding to the light source appearing in the environment.

[0106] In other possible embodiments, after the camera application is opened, when a user instruction is received to enable the flicker elimination function, the flicker sensor in the flicker detection module is powered on to start detecting the frequency of light sources present in the environment until the electronic device closes the camera application.

[0107] S103, the flicker detection module detects multiple light source frequencies, calibrates the detected light source frequencies, and generates corresponding frequency detection results.

[0108] In some embodiments, the flicker sensor in the flicker detection module can scan the light source signals in the surrounding environment to obtain the raw sequence data corresponding to the light source signals. This data is then passed to the flicker frequency output module in the flicker detection module. The flicker frequency output module can determine one or more light source frequencies based on the raw sequence data and generate corresponding frequency detection results. As one implementation method, such as... Figure 6 As shown, the flashing frequency output module can perform the following steps:

[0109] S1, Obtain raw sequence data: Obtain the raw sequence data collected by the scintillation sensor.

[0110] In some embodiments, after the scintillation sensor is activated, it begins to collect light source signals within its environment, outputting the time of each sample and the corresponding electrical signal. This electrical signal represents the signal strength of the light source signal at that sampling time point, i.e., the brightness value. After multiple samplings, the output sampling times and electrical signals can form raw sequence data. It can be understood that the raw sequence data output by the scintillation sensor is the time-domain signal of each light source signal within the environment, belonging to a one-dimensional time series.

[0111] It is understood that the sampling frequency of the scintillation sensor can be set according to actual needs, and this application does not impose any restrictions on it. For example, the sampling frequency of the scintillation sensor is 2kHz, that is, the scintillation sensor samples once every 0.5 milliseconds (ms) until a specified number of data are collected, and then outputs the corresponding raw sequence data.

[0112] After outputting the raw sequence data, the flicker sensor can continue to acquire light source signals until it detects that the electronic device has exited the shooting or recording function, or detects that the flicker elimination function has been turned off. At this point, the flicker sensor is turned off and the acquisition of raw sequence data stops.

[0113] In real-world scenarios, the number of light sources providing light signals in the environment where the large screen is set up can be greater than or equal to one. These light sources include, but are not limited to, the light source of the object being photographed (the large screen) and the light sources of lighting devices located adjacent to the object being photographed. The raw sequence data output by the flicker sensor not only contains multiple light source signals from the large screen but may also include light source information from other lighting devices.

[0114] S2, Dimensional Enhancement Processing: Perform dimensional enhancement processing on the original sequence data.

[0115] Understandably, the purpose of dimensionality enhancement is to address the underfitting problem. In some possible implementations, if the amount of raw sequence data output by the scintillation sensor is insufficient, a lack of features may occur. In this scenario, the features of the dataset can be expanded. For example, if the scintillation sensor only collects 256 light source signals as the raw sequence data output, it can be padded with zeros to increase the number of data points to 1000, thus achieving dimensionality enhancement for the raw sequence data. Of course, in this embodiment, no specific limitations are made on the number of padded data points or the dimensionality enhancement method that can be used.

[0116] S3, Fast Fourier Transform (FFT): Based on the original sequence data after dimensionality enhancement, FFT is performed to obtain spectral data.

[0117] Here, FFT can be a general term for efficient and fast calculation methods that utilize computers to compute the Discrete Fourier Transform (DFT). The frequency output module can use Fast Fourier Transform to analyze the dimension-enhanced original sequence data, transforming the original sequence data from the time domain to the frequency domain, thus obtaining a spectrum of data. It can be understood that the horizontal axis of the spectrum data represents frequency, and the vertical axis represents amplitude (the intensity of the light source signal, i.e., brightness value). According to Fourier's principle, any continuously measured time series or signal can be represented as an infinite superposition of sinusoidal signals of different frequencies. In the embodiments provided in this application, after transforming the original sequence data to the frequency domain, the resulting spectrum data consists of multiple sine waves, which can be multiple detected initial light source frequencies. Each initial light source frequency corresponds to an amplitude.

[0118] S4, Frequency Search: From the light source frequencies in the spectrum data, determine N frequencies to be processed, where the amplitude of the N frequencies to be processed is greater than the amplitude of other light source frequencies in the spectrum data.

[0119] In some embodiments, such as Figure 6 As shown, after FFT transformation, the resulting spectral data contains frequency information for multiple light source frequencies (also known as initial light source frequencies). These initial light source frequencies are sorted in descending order of amplitude. The first N initial light source frequencies are determined as the frequencies to be processed (also known as the third light source frequency). Here, N is a preset positive integer.

[0120] For example, in Figure 6In the scenario shown, N can be 4. Among the detected initial light source frequencies, the amplitudes of frequencies A, B, C, and D are all greater than those of the other initial light source frequencies. Specifically, frequency C has the largest amplitude, followed by frequency D, while the amplitude of frequency A is smaller than that of frequency D, and the amplitude of frequency B is smaller than that of frequency A. Through frequency search, frequencies C, D, A, and B can be identified as the frequencies to be processed.

[0121] In a possible embodiment, the initial light source frequencies are divided into high-frequency and low-frequency categories in the spectral data, using 530Hz as the dividing line. High-frequency refers to initial light source frequencies with a frequency value greater than or equal to 530Hz, and low-frequency refers to initial light source frequencies with a frequency value less than 530Hz. Then, the high-frequency frequencies are sorted in descending order of amplitude. The initial light source frequencies ranked in the top N1 positions are determined as the frequencies to be processed. Similarly, the low-frequency frequencies are sorted in descending order of amplitude. The initial light source frequencies ranked in the top N2 positions are determined as the frequencies to be processed, where N1 and N2 are positive integers, and N1 + N2 = N.

[0122] S5, Calibrate Frequency Drift: Perform calibration processing on N frequencies to be processed.

[0123] In some embodiments, it can be determined one by one whether N frequencies to be processed are frequencies that have drifted relative to a harmonic of a preset frequency. The principle of the determination is as follows: among all harmonics of the preset frequency, the target harmonic that is closest to the frequency to be processed is determined. Then, by analyzing the difference between each frequency to be processed and the target harmonic, it is determined whether the frequency to be processed has drifted relative to the harmonic of the preset frequency. If the frequency to be processed has drifted relative to the harmonic of the preset frequency, the frequency to be processed is corrected to a harmonic of the preset frequency (e.g., the target harmonic of the frequency to be processed). If the frequency to be processed has not drifted relative to the harmonic of the preset frequency, the frequency to be processed is retained.

[0124] for example, Figure 6 As shown, if frequencies A, B, and C all drift relative to the harmonics of the preset frequency, while frequency D does not drift, then after calibration, the N frequencies to be processed include: the target harmonic corresponding to frequency A, the target harmonic corresponding to frequency B, the target harmonic corresponding to frequency C, and frequency D.

[0125] As one implementation method, the process of calibrating the frequency to be processed is as follows: Figure 7 As shown:

[0126] S5-1, among all the harmonics of the preset frequency, determine the target harmonic corresponding to each frequency to be processed.

[0127] Specifically, the absolute difference between the frequency to be processed and the target harmonic is smaller than the absolute difference between the frequency and other harmonics of the preset frequency. For example, the absolute difference between the frequency to be processed and each harmonic of the preset frequency can be calculated. Then, among the harmonics of the preset frequency, the harmonic with the smallest corresponding absolute difference is selected as the target harmonic of the frequency to be processed.

[0128] For example, the preset frequency is 60Hz, and its multipliers include 60Hz, 120Hz, 240Hz, 360Hz, etc. When the frequency to be processed is 121Hz, the target multiplier is 120Hz. When the frequency to be processed is 242Hz, the target multiplier is 240Hz. When the frequency to be processed is 100Hz, the target multiplier is 120Hz.

[0129] In summary, the target harmonics corresponding to different frequencies to be processed may be different or the same, depending on the actual situation. This application does not impose any specific limitations on this.

[0130] S5-2, calculate the relative offset coefficient between the frequency to be processed and the corresponding target harmonic.

[0131] The relative offset coefficient indicates the degree of deviation between the frequency to be processed and the corresponding target octave. For example, it can be calculated using the formula:

[0132]

[0133] Calculate the relative offset coefficients. Where 'a' is the relative offset coefficient, λ is the frequency to be processed, and θ is the target harmonic. For example, the preset frequency is 60Hz, and the frequencies to be processed include 121Hz, 242Hz, 100Hz, and 484Hz. The target harmonic corresponding to 121Hz and 100Hz is 120Hz. The target harmonic corresponding to 242Hz is 240Hz, and the target harmonic corresponding to 484Hz is 480Hz. Based on the above formula, the relative offset coefficient for 121Hz is 0.83%, the relative offset coefficient for 100Hz is 16.6%, the relative offset coefficient for 242Hz is 0.82%, and the relative offset coefficient for 484Hz is 0.83%.

[0134] S5-3, Calculate the average offset coefficient based on the relative offset coefficients of the N frequencies to be processed.

[0135] In some embodiments, the sum of the relative offset coefficients of the N frequencies to be processed can be calculated, and then the ratio between the sum and N can be calculated as the average offset coefficient. Continuing the previous example, if the frequencies to be processed include 121Hz, 242Hz, 100Hz, and 484Hz, the average offset coefficient can be determined to be (0.83% + 16.6% + 0.82% + 0.83%) / 4 = 4.77%.

[0136] S5-4, determine whether the difference between the relative offset coefficient and the average offset coefficient of the frequency to be processed is less than a preset threshold.

[0137] The aforementioned preset threshold can be an empirical value, and this embodiment does not specifically limit it. If the difference between the relative offset coefficient and the average offset coefficient of the frequency to be processed is less than the preset threshold, the frequency to be processed is corrected, and the process proceeds to S5-5. If the difference between the relative offset coefficient and the average offset coefficient of the frequency to be processed is not less than the preset threshold, the frequency to be processed is not corrected, and the process proceeds to S5-6.

[0138] Understandably, in scenes shot on large screens, the screen's light source is the primary light source and has relatively higher brightness. When actually shooting a large, illuminated screen, most of the determined frequencies to be processed are multiples of the preset frequency (or frequencies that drift relative to multiples of the preset frequency). This portion of frequencies to be processed is called the first part of the frequencies. The relative offset coefficients of the frequencies in the first part are almost identical. Of course, there may also be light emission frequencies from other light sources, or light source frequencies generated without clock offset. This portion of frequencies to be processed can be called the second part of the frequencies, for example, 100Hz. The relative offset coefficient of the frequencies in the second part differs significantly from that in the first part. Furthermore, the number of frequencies in the second part is relatively small, and the difference between the corresponding average offset coefficient and the relative offset coefficient of the frequencies in the second part is also large. Based on this difference, after comparing it with a preset threshold, correction of the frequencies in the second part can be avoided.

[0139] Of course, the difference between the average offset coefficient and the relative offset coefficient of the frequency in the first part is small. Based on this difference, a comparison with a preset threshold can trigger a correction of the frequency in the first part. This improves the accuracy of the correction and reduces the problem of false corrections.

[0140] Furthermore, in scenarios other than large-screen shooting, the relative offset coefficients between the determined processing frequency and the target frequency are relatively random. In this scenario, the relative offset coefficients of each processing frequency are more discrete than the average offset coefficient, meaning the difference between the two may be large. Therefore, it is possible to avoid triggering a correction of the processing frequency.

[0141] S5-5 corrects the frequency to be processed to the corresponding target harmonic.

[0142] S5-6, without correcting the frequency to be processed.

[0143] In some embodiments, the frequency to be processed that does not require correction may also be referred to as the fourth light source frequency.

[0144] Understandably, we can iterate through each frequency to be processed, executing S5-4 once for each frequency. Based on the actual situation of the frequency being processed, we decide whether to execute S5-5 or S5-6. After executing S5-5 or S5-6, we continue to iterate through the next frequency to be processed until all frequencies have been processed.

[0145] For example, the differences between the relative offset coefficients and the average offset coefficients corresponding to 121Hz, 242Hz, and 484Hz are all less than a preset threshold, while the difference between the relative offset coefficient and the average offset coefficient corresponding to 100Hz is greater than the preset threshold. In this scenario, 121Hz, 242Hz, and 484Hz can be corrected to 120Hz, 240Hz, and 480Hz respectively. After the above calibration, the frequencies to be processed include 100Hz, 120Hz, 240Hz, and 480Hz.

[0146] S6, Large Screen Shooting Scene Detection: Based on the calibrated N unprocessed frequencies, determine the activation of a strategy for large screen shooting scenes.

[0147] As one implementation method, if all N calibrated frequencies to be processed are multiples of the preset frequency, it can be determined that the strategy for large screen shooting scenarios is enabled; otherwise, the strategy for non-large screen shooting scenarios is enabled.

[0148] As another implementation method, among the N calibrated frequencies to be processed, determine the number of multipliers of the preset frequency (1) and the number of multipliers of the non-preset frequency (2). If the number of multipliers 1 is greater than the number of multipliers 2, it can be determined that the strategy for large screen shooting scenarios is enabled; otherwise, the strategy for non-large screen shooting scenarios is enabled.

[0149] S7, Frequency doubling and sidelobe processing: Based on the calibrated N frequencies to be processed, frequency doubling and sidelobe processing are performed to obtain at least one output light source frequency.

[0150] In some embodiments, N frequencies to be processed can be input into the frequency multiplier, and the frequency after frequency multiplication can be output.

[0151] The frequency multiplier is used to multiply the received frequencies. For example, the frequency multiplier processes frequencies as follows: Among N frequencies to be processed, multiple light source frequencies that are multiples of each other are identified. Then, among these multiple frequencies, the light source frequency 1 with the smallest frequency value and the light source frequency 2 with the largest amplitude are identified. The frequency value of light source frequency 2 is then modified to match the frequency value of light source frequency 1. Thus, among the multiple frequencies that are multiples of each other, there are two light source frequencies with the same frequency value but different amplitudes.

[0152] For example, a frequency multiplier acquires four light source frequencies, ordered from largest to smallest amplitude: light source frequency q, light source frequency w, light source frequency e, and light source frequency r. The frequency values ​​of light source frequencies q, w, e, and r are 360Hz, 120Hz, 240Hz, and 570Hz, respectively. Since 360Hz, 120Hz, and 240Hz are multiples of each other, light source frequencies q, w, and e are multiple light source frequencies in a multiple-relationship relationship. Among these frequencies, light source frequency w has a frequency value of 120Hz, which is the smallest compared to the other multiple-relationship frequencies; therefore, light source frequency w can be identified as light source frequency 1. Furthermore, the light source frequency with the largest amplitude is light source frequency q; therefore, light source frequency q can be identified as light source frequency 2. Thus, the frequency value of light source frequency q is modified to the frequency value of light source frequency w (120Hz). After frequency multiplication, the resulting light source frequencies q, w, e, and r are 120Hz, 120Hz, 240Hz, and 570Hz, respectively. The frequency multiplier can then output frequencies including: the light source frequency with the largest amplitude among multiple light source frequencies in a multiple relationship (e.g., light source frequency q with a frequency value of 120Hz), and other light source frequencies besides those in a multiple relationship (e.g., light source frequency r with a frequency value of 570Hz).

[0153] For details on frequency multiplication processing, please refer to relevant technologies; they will not be elaborated upon here.

[0154] In some embodiments, sidelobe processing can also be performed based on the frequency output after frequency doubling.

[0155] The principle of sidelobe processing is as follows: After receiving multiple light source frequencies requiring sidelobe processing, the frequency with the largest amplitude is determined. Then, among the received light source frequencies, frequencies whose absolute difference from the frequency with the largest amplitude is less than a specified value are eliminated. For example, if the specified value is 15Hz, and the frequency with the largest amplitude among the multiple light source frequencies requiring sidelobe processing is 360Hz, in this scenario, light source frequencies with values ​​between 345Hz and 375Hz are eliminated from the multiple light source frequencies requiring sidelobe processing, and the remaining light source frequencies are output. Other implementation details can be found in related technologies and will not be elaborated here.

[0156] S8 generates the corresponding frequency detection results.

[0157] In some embodiments, the output light source frequencies, after frequency doubling and / or sidelobe processing, can be arranged in descending order of amplitude, and corresponding frequency detection results can be generated.

[0158] In a possible implementation, the frequency doubling and sidelobe processing can be omitted. In this case, the calibrated frequencies to be processed can be arranged in descending order of amplitude, and the corresponding frequency detection results can be generated.

[0159] In the two possible embodiments described above, the light source frequency used to generate the frequency detection result (e.g., the output light source frequency after frequency doubling and / or sidelobe processing; or, for example, the calibrated frequency to be processed) can be collectively referred to as the output frequency. After obtaining M output frequencies through the above S1 to S7 steps, the output frequencies can be processed as follows: Figure 8 The preprocessing shown:

[0160] (1) Sort the M frequencies to be output in ascending order of amplitude. Where M is a positive integer greater than 2.

[0161] (2) Determine if the first output frequency is a multiple of the preset frequency. If it is a multiple of the preset frequency, proceed to (3). If it is not a multiple of the preset frequency, proceed to (4).

[0162] (3) Modify the frequency value of the first frequency to be output to the frequency value corresponding to the preset frequency.

[0163] For example, the output frequencies, arranged from largest to smallest amplitude, are: light source frequencies of 240Hz, 100Hz, 120Hz, and 300Hz. The preset frequency is 60Hz. In this example, the first output frequency (240Hz) is a multiple of the preset frequency. Therefore, the frequency value of this output frequency is changed from 240Hz to 60Hz. After the modification, the output frequencies, arranged from largest to smallest amplitude, are: 60Hz, 100Hz, 120Hz, and 300Hz.

[0164] After step (3), the process proceeds to (5).

[0165] (4) Modify the frequency value of the second frequency to be output to the frequency value corresponding to the preset frequency.

[0166] For example, the output frequencies, arranged from largest to smallest amplitude, are: light source frequencies of 100Hz, 240Hz, 120Hz, and 300Hz. The preset frequency is 60Hz. In this example, the first output frequency (100Hz) is not a multiple of the preset frequency. Accordingly, the second output frequency is changed from 240Hz to 60Hz. After this modification, the output frequencies, arranged from largest to smallest amplitude, are: 100Hz, 60Hz, 120Hz, and 300Hz.

[0167] After step (4), the process also proceeds to (5).

[0168] (5) Generate frequency output results based on the modified output frequency and the unmodified output frequency.

[0169] In some embodiments, if the first output frequency is modified, a frequency output result is generated based on the modified output frequency (the first output frequency) and the second output frequency. Similarly, if the second output frequency is modified, a frequency output result is generated based on the modified output frequency (the second output frequency) and the first output frequency.

[0170] In other embodiments, if the first output frequency is modified, a frequency output result is generated based on the modified output frequency (the first output frequency) and all other output frequencies. If the second output frequency is modified, a frequency output result is generated based on the modified output frequency (the second output frequency) and all other output frequencies.

[0171] In a possible embodiment, an identifier for the large-screen shooting scene can also be added to the frequency output result.

[0172] S104, the flicker detection module sends the frequency detection result to the AE module.

[0173] S105, the AE module determines the appropriate target exposure time based on the frequency detection results.

[0174] In some embodiments, the AE module can determine whether the frequency detection result includes a preset frequency or an identifier for a large-screen shooting scene. If the preset frequency is included, the period of the preset frequency is determined as the target exposure duration.

[0175] In a possible embodiment, if the period length of the preset frequency is greater than the maximum exposure time, the period length of a multiple of L of the preset frequency can be selected as the target exposure time. For example, the multiple of L can be an even multiple of the preset frequency, and the multiple of L of the preset frequency is the multiple of L with the smallest difference from the preset frequency among all even multiples of the preset frequency whose period length is less than the current maximum exposure time. For example, if the preset frequency is 60Hz, and the period length corresponding to 60Hz is greater than the maximum exposure time, it can be determined whether the period length of twice the preset frequency (120Hz) is greater than the maximum exposure time. If the period length of 120Hz is not greater than the maximum exposure time, the period length of 120Hz is used as the target exposure time. If the period length of 120Hz is greater than the maximum exposure time, it is further determined whether the period length of 240Hz is greater than the maximum exposure time, and so on. In summary, the target exposure time can be determined as a first duration, which can be the period length of the preset frequency, or the period length of a multiple of L of the preset frequency.

[0176] For example, the L-multiplier can be an odd multiplier. The L-multiplier of the preset frequency is the multiplier with the smallest difference from the preset frequency among all odd multipliers with a period duration shorter than the current maximum exposure duration. In this scenario, L is a positive odd number greater than 1.

[0177] In a possible embodiment, in the frequency detection results, if the number of even-numbered harmonics of the preset frequency is greater than the number of odd-numbered harmonics, the aforementioned L harmonic is an even-numbered harmonic. If the number of even-numbered harmonics of the preset frequency is less than the number of odd-numbered harmonics, the aforementioned L harmonic is an odd-numbered harmonic. Alternatively, in the frequency detection results, if the light source frequency with the largest amplitude is an even-numbered harmonic of the preset frequency, the aforementioned L harmonic is an even-numbered harmonic. If the light source frequency with the largest amplitude is an odd-numbered harmonic of the preset frequency, the aforementioned L harmonic is an odd-numbered harmonic.

[0178] In some embodiments, the AE module can acquire all light source frequencies recorded in the frequency detection results. If the frequency detection results include a preset frequency or an identifier for a large-screen shooting scene, the least common multiple (LCM) between the period duration of the preset frequency and the period durations of other light source frequencies in the frequency detection results is calculated. If the LCM is less than the current maximum exposure time, the LCM is used as the target exposure time. If the LCM is greater than the current maximum exposure time, the period duration of the preset frequency is used as the target exposure time. Simultaneously, by adjusting the frame interval, the bright and dark stripes caused by other light source frequencies in the frequency detection results are fixed.

[0179] Furthermore, the brightness information of the image data that can be acquired during the current maximum exposure time is determined. For example, after the image sensor is powered on, it can be controlled to acquire image data 1 (referred to as the third image). The exposure time corresponding to image data 1 is the default exposure time. The default exposure time can be a fixed value or the exposure time when acquiring the adjacent previous frame image data. After acquiring image data 1, the image sensor sends image data 1 to the AE statistics module. The AE statistics module can calculate various brightness information corresponding to image data 1, such as brightness histogram and brightness area statistics. The brightness area statistics can be the average values ​​of the four components R / Gr / Gb / B of the entire image. The brightness area statistics can also be the average values ​​of the four components R / Gr / Gb / B of a specified image area (e.g., the image area displayed on a large screen). The AE statistics module sends the brightness information corresponding to image data 1 to the AE processing module. Based on the brightness information of image data 1, the AE processing module determines the maximum exposure time.

[0180] In some embodiments, the AE processing module can determine the exposure level for shooting in the current shooting scene based on the brightness information of image data 1. Details of the exposure determination process can be found in related art on automatic exposure techniques, and will not be elaborated upon here.

[0181] The AE processing module determines the maximum exposure time based on the determined exposure level. It can be understood that the exposure level is equal to the product of the exposure time and the exposure setpoint. If the exposure time needs to be increased when the exposure level is fixed, the exposure setpoint needs to be decreased. The exposure setpoint can refer to the illuminance received by the image sensor's sensing surface. That is, the exposure setpoint can be adjusted by changing the aperture and shutter speed; of course, the adjusted exposure setpoint should not be less than 1. In other words, when the exposure level is fixed, an exposure setpoint of 1 corresponds to the maximum exposure time.

[0182] S106, the AE module sends the exposure parameters to be configured to the image sensor, including the target exposure duration.

[0183] S107, the image sensor acquires image data according to the target exposure time.

[0184] S108, the image sensor sends the acquired image data to the ISP.

[0185] The image sensor can send the acquired image data (i.e., the second image) to the ISP TONE in the ISP. The ISP TONE is responsible for performing brightness adaptation processing on the acquired image data after the AE module performs automatic exposure, ensuring the image data brightness is within a suitable range. After receiving the image data, the ISP TONE performs brightness adaptation processing. Then, the ISP TONE sends the image data to other processing modules within the ISP. These modules process the image data, transforming it into an image visible to the user. Specific details can be found in relevant technologies and will not be elaborated upon here.

[0186] S109, the ISP sends processed image data to the camera application.

[0187] In some embodiments, the ISP can feed image data back to the camera application through the camera device driver and camera control channel.

[0188] After the camera application receives the image data, it can control the display screen to show the image data.

[0189] Some embodiments of this application also provide an electronic device, which may include a memory and one or more processors. The memory and processors are coupled. The memory is used to store computer program code, which includes computer instructions. When the processor executes the computer instructions, the electronic device can perform various functions or steps performed by the electronic device in the above method embodiments.

[0190] like Figure 9 As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0191] The aforementioned sensor module 180 may include sensors such as pressure sensors, gyroscope sensors (also known as gyroscopes), barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, and bone conduction sensors.

[0192] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0193] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), ISP, controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0194] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0195] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0196] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0197] It is understood that the interface connection relationships between the modules illustrated in this embodiment are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0198] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0199] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a touch layer and a display panel. The touch layer is used to sense user interaction with display screen 194. The display panel can be a liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode (AMOLED), flexible light-emitting diode (FLED), minimized, microLED, micro-OLED, quantum dot light-emitting diode (QLED), etc. Electronic device 100 can realize shooting functions through ISP, camera 193, video codec, GPU, display screen 194, and application processor.

[0200] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element (image sensor). The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0201] The aforementioned camera 193 can be a camera module with OIS capability, supporting the enabling of OIS function. The camera 193 is used to capture still images or videos; an object is projected onto a photosensitive element through the lens, generating an optical image. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor, such as a camera sensor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include N cameras 193, where N is a positive integer greater than 1.

[0202] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform various functions or steps performed by the mobile phone in the above method embodiments.

[0203] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform various functions or steps performed by the mobile phone in the above method embodiments.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0205] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0206] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0208] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0209] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A shooting control method, characterized in that, include: After detecting multiple first light source frequencies from the target large screen, a first image is acquired; wherein, the exposure duration of the first image is a first duration; the multiple first light source frequencies include multiple harmonics of a preset frequency, and the first duration is equal to the period duration of the harmonics of the preset frequency; After the clock of the target large screen deviates, multiple second light source frequencies from the target large screen are detected; wherein, the multiple second light source frequencies do not contain a harmonic of the preset frequency; The frequencies of the plurality of second light sources are determined to be the light source frequencies generated after the target screen experiences frequency drift; A second image is acquired; wherein the exposure time of the second image is the first exposure time, and both the first image and the second image include the target screen.

2. The method according to claim 1, characterized in that, The detection of multiple second light source frequencies from the target screen includes: detecting a third light source frequency present in the environment, wherein the third light source frequency includes the multiple second light source frequencies; Determining that the plurality of second light source frequencies are light source frequencies generated after the target large screen exhibits frequency drift includes: Among the multiple harmonics of the preset frequency, a target harmonic corresponding to each of the third light source frequencies is determined. The target harmonic corresponding to the third light source frequency is the harmonic with the smallest difference from the third light source frequency among the multiple harmonics of the preset frequency. Determine the relative offset coefficient between each of the third light source frequencies and the target harmonic, and the average offset coefficient among all the relative offset coefficients of the third light source frequencies; wherein the relative offset coefficient can represent the deviation ratio of the third light source frequency relative to the target harmonic. If the difference between the relative offset coefficient of the second light source frequency and the average offset coefficient is less than a preset threshold, the second light source frequency is determined to be the light source frequency generated after the target screen experiences frequency drift.

3. The method according to claim 1 or 2, characterized in that, After determining that the plurality of second light source frequencies are light source frequencies generated after the target large screen exhibits frequency drift, the method further includes: The frequency value of the second light source frequency is corrected to the frequency value of the corresponding target harmonic; wherein, the target harmonic of the second light source frequency is the harmonic with the smallest difference between the second light source frequency and the harmonic of the preset frequency.

4. The method according to claim 2, characterized in that, Applied to an electronic device, the electronic device including a flicker sensor, the detection of the frequency of a third light source present in the environment includes: The scintillation sensor is controlled to acquire raw sequence data; The original sequence data is processed by Fast Fourier Transform to obtain spectral data, which includes multiple light source frequencies. N third light source frequencies are determined from the spectrum data; the amplitudes of the N third light source frequencies are greater than the amplitudes of the other light source frequencies in the spectrum data, and N is a positive integer.

5. The method according to claim 2, characterized in that, The third light source frequency also includes a fourth light source frequency; If the difference between the relative offset coefficient of the fourth light source frequency and the average offset coefficient is greater than a preset threshold, it is determined that the fourth light source frequency is not the light source frequency generated after the target screen experiences frequency drift, and the frequency value of the fourth light source frequency is not modified.

6. The method according to claim 3 or 5, characterized in that, After correcting the frequency values ​​of all the second light source frequencies to the corresponding target harmonic frequencies, the method further includes: The scene of shooting a large screen is identified based on the frequency of the third light source; The frequency of the third light source is subjected to frequency doubling and / or sidelobe processing; The frequencies of the processed third light source are sorted in descending order of amplitude. If the frequency of the third light source ranked first is a multiple of the preset frequency, the frequency value of the third light source ranked first is replaced with the frequency value of the preset frequency. If the frequency of the third light source ranked first is not a multiple of the preset frequency, the frequency value of the third light source ranked second is replaced with the frequency value of the preset frequency.

7. The method according to any one of claims 1-6, characterized in that, Applied to electronic devices, the electronic devices including image sensors for acquiring image data; Before acquiring the first image, the method further includes: Acquire a third image; The maximum exposure time is determined based on the brightness information of the third image; If the period duration of the preset frequency is less than the maximum exposure duration, then the first duration is determined to be equal to the period duration of the preset frequency. The exposure time of the image sensor is configured to the first duration.

8. The method according to claim 7, characterized in that, If the period duration of the preset frequency is greater than the maximum exposure duration, the first duration is determined to be the period duration of L times the preset frequency, where L is an integer multiple of 1.

9. The method according to claim 8, characterized in that, If, among the plurality of first light source frequencies, the number of even-numbered harmonics of the preset frequency is greater than the number of odd-numbered harmonics of the preset frequency, or, among the plurality of first light source frequencies, the first light source frequency with the largest amplitude is an even-numbered harmonic of the preset frequency, then the L harmonic is an even-numbered harmonic. If, among the plurality of first light source frequencies, the number of even-numbered harmonics of the preset frequency is less than the number of odd-numbered harmonics of the preset frequency, or, among the plurality of first light source frequencies, the first light source frequency with the largest amplitude is an odd-numbered harmonic of the preset frequency, then the L harmonic is an odd-numbered harmonic.

10. The method according to any one of claims 1-9, characterized in that, The preset frequency is 60Hz.

11. An electronic device, characterized in that, The electronic device includes: an image sensor, a flicker sensor, a processor, and a memory. The image sensor is used to acquire raw image data of an image frame, and the flicker sensor is used to detect the light source frequency corresponding to a light source present in the environment. The memory is used to store computer instructions and the raw image data of the image frame. When the processor executes the computer instructions, it causes the electronic device to perform the method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program or instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-10.

13. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1-10.

Images