Photographing method and device, electronic equipment and storage medium

Abstract

The invention relates to a photographing method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting an image in response to a detected photographing instruction, and determining a current photographing scene; determining a target frequency of processing the image by a processor based on the photographing scene; wherein the target frequency is greater than the initial frequency of the processor; and processing the image based on the target frequency by using the processor, and outputting a processed target image. Through the scheme, the photographing performance can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of photographing technology, and in particular to a photographing method and device, electronic equipment, and storage medium. Background Art

[0002] Camera-enabled electronic devices, such as mobile phones, cameras, and wearables, have undergone multiple generations of development, but significant improvements in hardware performance are becoming increasingly difficult. To improve camera performance and enhance the user experience, many manufacturers have begun focusing on software improvements. These efforts, including further standardizing the camera process, reducing unnecessary logic paths, and refining image input and output formats, can only improve camera performance to a limited extent. Further improving camera performance remains an urgent issue. Summary of the Invention

[0003] The present disclosure provides a photographing method and device, an electronic device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a photographing method, comprising:

[0005] In response to detecting a photographing instruction, capturing an image and determining a current photographing scene;

[0006] Determining a target frequency for a processor to process the image based on the photographing scene; wherein the target frequency is greater than an initial frequency of the processor;

[0007] The processor is used to process the image based on the target frequency, and a processed target image is output.

[0008] In some embodiments, the method further comprises:

[0009] Obtaining the temperature and / or occupancy rate of the processor;

[0010] The determining, based on the photographing scene, a target frequency at which the processor processes the image includes:

[0011] When the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, a target frequency for the processor to process the image in association with the photographing scene is determined based on the photographing scene.

[0012] In some embodiments, the image captured in the photographing scene includes multiple images, and the method further includes:

[0013] pre-processing each image based on the initial frequency;

[0014] The determining, when the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, based on the photographing scene, a target frequency at which the processor processes the image in association with the photographing scene includes:

[0015] When the temperature is greater than the preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, the target frequency at which the processor performs image fusion on the preprocessed multiple images is determined based on the photographing scene.

[0016] In some embodiments, determining a target frequency for a processor to process the image based on the photographing scene includes:

[0017] determining, based on the photographing scene, a frequency threshold at which the processor processes the image;

[0018] A frequency greater than the frequency threshold is determined as the target frequency for the processor to process the image.

[0019] In some embodiments, the method further comprises:

[0020] Based on the photographing scene, determining a target duration for processing the image based on the target frequency;

[0021] The processing of the image based on the target frequency by the processor and outputting the processed target image includes:

[0022] Within the target duration, the processor is used to process the image based on the target frequency, and the processed target image is output.

[0023] In some embodiments, determining a target frequency for a processor to process the image based on the photographing scene includes:

[0024] In response to the photographing scene being a preset target photographing scene, a target frequency for the processor to process the image is determined based on the photographing scene.

[0025] In some embodiments, the method further comprises:

[0026] In response to the completion of processing the image based on the target frequency by the processor, the frequency of the processor is restored to the initial frequency.

[0027] In some embodiments, the processor comprises a graphics processor.

[0028] According to a second aspect of an embodiment of the present disclosure, there is provided a photographing device, including:

[0029] An acquisition module, configured to acquire an image and determine a current photographing scene in response to detecting a photographing instruction;

[0030] A first determining module is configured to determine a target frequency for a processor to process the image based on the photographing scene; wherein the target frequency is greater than an initial frequency of the processor;

[0031] A processing module is used to process the image based on the target frequency using the processor and output the processed target image.

[0032] In some embodiments, the apparatus further comprises:

[0033] an acquisition module, configured to acquire the temperature and / or occupancy rate of the processor;

[0034] The first determination module is configured to determine, based on the photographing scene, a target frequency at which the processor processes the image in association with the photographing scene when the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold.

[0035] In some embodiments, the image collected in the photographing scene includes multiple images, and the apparatus further includes:

[0036] a preprocessing module, configured to preprocess each image based on the initial frequency;

[0037] The first determination module is configured to determine, based on the photographing scene, the target frequency at which the processor performs image fusion on the preprocessed multiple images when the temperature is greater than the preset temperature threshold and / or the occupancy rate is greater than the preset occupancy rate threshold.

[0038] In some embodiments, the first determination module is used to determine a frequency threshold for the processor to process the image based on the photographing scene; and determine a frequency greater than the frequency threshold as the target frequency for the processor to process the image.

[0039] In some embodiments, the apparatus further comprises:

[0040] A second determining module is configured to determine, based on the photographing scene, a target duration for processing the image based on the target frequency;

[0041] The processing module is configured to process the image based on the target frequency using the processor within the target duration, and output the processed target image.

[0042] In some embodiments, the first determining module is configured to determine a target frequency for the processor to process the image based on the photographing scene in response to the photographing scene being a preset target photographing scene.

[0043] In some embodiments, the apparatus further comprises:

[0044] A recovery module is configured to, in response to the completion of the image processing by the processor based on the target frequency, restore the frequency of the processor to the initial frequency.

[0045] In some embodiments, the processor comprises a graphics processor.

[0046] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0047] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method as described in the first aspect above.

[0048] According to a fourth aspect of an embodiment of the present disclosure, there is provided a storage medium, including:

[0049] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method as described in the first aspect above.

[0050] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0051] Compared to a scheme that schedules the processor frequency according to relevant processor frequency scheduling rules, which results in the processor taking a longer time to process an image at a lower frequency, such as the initial frequency, the embodiment of the present disclosure responds to detecting a photo instruction, captures an image, determines the current photo scene, and determines the target frequency of the processor for image processing after the photo scene is increased based on the photo scene. Thus, by using the processor to process the image based on the higher target frequency, the image processing time can be reduced, the efficiency of outputting the target image can be improved, and the photo performance can be improved. In addition, the embodiment of the present disclosure determines the increased target frequency based on the photo scene, so that the target frequency is adapted to the photo scene. For example, for a photo scene corresponding to a more complex image processing process, image processing is performed based on the target frequency of the processor after a larger increase; for a photo scene corresponding to a less complex image processing process, image processing is performed based on the target frequency of the processor after a smaller increase. This improves the intelligence of regulating the processor frequency on the basis of improving photo performance, thereby improving the overall photo experience.

[0052] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0054] Figure 1 This is an example of a photographing method process shown in the embodiment of the present disclosure. Figure 1 .

[0055] Figure 2 This is an example diagram of a mobile phone photo-taking scene provided by an embodiment of the present disclosure.

[0056] Figure 3 This is an example diagram comparing the photography efficiency before and after processor frequency increase in a super-resolution photography scenario, shown in an embodiment of the present disclosure.

[0057] Figure 4 This is an example of a photographing method process shown in the embodiment of the present disclosure. Figure 2 .

[0058] Figure 5 2 is a diagram of a photographing device according to an embodiment of the present disclosure.

[0059] Figure 6 It is a block diagram of an electronic device shown in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0061] In related art, electronic devices operate processors at a scheduled frequency based on processor frequency scheduling rules. The frequency scheduled by the processor frequency scheduling rules can be an assigned preset frequency, or it can be adjusted based on the processor temperature, processor occupancy, and other factors. For example, when the processor temperature or processor occupancy is high, the processor frequency is reduced based on the preset frequency. Taking mobile phone photography as an example, the process includes the following steps: In response to the mobile phone detecting an instruction to launch a camera application, the camera is launched; upon receiving the photo request, the framework layer of the mobile phone system obtains an identifier for the current photo scene and creates a photo process that matches the photo scene. During the photo process, the mobile phone captures an image using an image sensor, uses a processor to convert the captured image format, and performs image processing on the converted image to obtain a target image. The target image is then saved to an album for the user to view. The frequency of the processor performing the format conversion and subsequent processing of the captured image is scheduled by the mobile phone according to the default processor frequency scheduling rules. For example, it can be an assigned preset frequency or a frequency reduced based on the processor temperature and / or occupancy.

[0062] However, the frequency scheduled based on the aforementioned processor frequency scheduling rules is usually not too high. For example, in a mobile phone camera scenario, the image captured by the image sensor is processed at the allocated lower processor frequency based on the processor frequency scheduling rules, so that the processor performance cannot be released to a greater extent; in addition, when the processor temperature is high and / or the processor occupancy rate is high, the processor frequency will be limited to reduce the device temperature rise and the processor occupancy rate, which will also prevent the processor performance from being released to a greater extent, resulting in longer image processing time and unsatisfactory camera performance.

[0063] The above-mentioned wide range of control schemes for the processor means that there is still room for improvement in the photographing performance of electronic devices. An embodiment of the present disclosure provides a photographing method, the execution subject of which may be a photographing device including a processor. For example, the photographing method may be executed by a terminal device or other electronic device including an image acquisition component, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a mobile phone, a tablet computer, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the photographing method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0064] Figure 1 This is an example of a photographing method process shown in the embodiment of the present disclosure. Figure 1 ,Depend on Figure 1 It can be seen that the following steps are included:

[0065] S11, in response to detecting a photographing instruction, capturing an image and determining a current photographing scene;

[0066] S12. Determining a target frequency for a processor to process the image based on the photographing scene; wherein the target frequency is greater than an initial frequency of the processor;

[0067] S13: Utilize the processor to process the image based on the target frequency, and output the processed target image.

[0068] In step S11, the photographing device detects a photographing instruction, such as detecting a user operation instruction to take a photograph based on a display component (such as a display screen) of the photographing device, or detecting a user voice instruction to take a photograph based on a sound receiving component of the photographing device, etc. The embodiments of the present disclosure do not limit this.

[0069] Figure 2 This is an example diagram of a mobile phone photography scene provided by an embodiment of the present disclosure. The photography device is a mobile phone. The mobile phone responds to detecting the user's operation instruction on the control marked L1, that is, detecting the aforementioned photography instruction, and performs image acquisition.

[0070] In the embodiment of the present disclosure, when a photographing device detects a photographing instruction, the photographing device determines the current photographing scene. For example, the photographing device may use a preset default photographing scene as the current photographing scene. The photographing scenes include normal photographing scenes, super-resolution photographing scenes (SR), high dynamic range photographing scenes (HDR), night scene photographing scenes (SN), etc. Night scenes include super night and night owl.

[0071] In some embodiments, based on a button component representing a photographing scene displayed on a display screen of a photographing device, after a user clicks the button component, the photographing scene corresponding to the button component may be determined as the current photographing scene.

[0072] In some embodiments, the camera device can automatically detect the current shooting scene after detecting a shooting instruction based on a preset scene automatic detection algorithm. For example, if the current amount of light entering the lens is less than a preset light threshold, the current shooting scene is determined to be a night scene; if the subject's motion is detected based on the preview image before shooting, the current shooting scene is determined to be a high dynamic range shooting scene; if the camera device's built-in motion sensor detects the camera device's motion, the current shooting scene is determined to be a super-resolution shooting scene; wherein the motion sensor may be a gyroscope, an angular velocity sensor, a displacement sensor, etc.

[0073] In the disclosed embodiment, the initial frequency of the processor is determined based on the aforementioned processor frequency scheduling rules, such as an assigned preset frequency; or the frequency of the processor is adjusted based on the assigned frequency and / or the processor's occupancy rate. After determining the current photography scene, the photographing device of the disclosed embodiment determines a target frequency for the processor to process the captured image based on the photography scene; wherein the target frequency is greater than the initial frequency of the processor; the processor may be, for example, a graphics processing unit (GPU), a central processing unit (CPU), or a microprogrammed control unit (MCU).

[0074] In some embodiments, based on a preset mapping relationship between photographic scenes and frequencies, the frequency corresponding to the current photographic scene is determined as the target frequency. The preset mapping relationship between photographic scenes and frequencies is derived from multiple experiments, and in the preset mapping relationship between photographic scenes and frequencies, the frequency corresponding to the photographic scene is greater than the initial frequency of the processor. The initial frequency of the processor may be the same or different in different photographic scenes, and this is not limited in the presently disclosed embodiments.

[0075] In some embodiments, based on the correspondence between preset photographic scenes and frequency increments, the frequency increment corresponding to the current photographic scene is determined, and the weighted sum of the initial frequency of the processor and the frequency increment under the photographic scene is determined as the target frequency; wherein the correspondence between the preset photographic scenes and the frequency increment is obtained based on multiple experiments.

[0076] In some embodiments, the photographing device can also determine the target frequency of the processor for processing the image based on the historical image processing time in the photographing scene; wherein the target frequency is greater than the initial frequency of the processor, and the target frequency corresponding to different photographing scenes can be positively correlated with the historical image processing time in the photographing scene.

[0077] It should be noted that the target frequency of the processor for processing the image based on the photo-taking scene must not exceed the maximum frequency supported by the processor hardware, so that the processor can operate normally at the target frequency.

[0078] In step S13, after determining the target frequency, the photographing device increases the processor frequency to the target frequency, uses the processor to process the image based on the target frequency, and outputs the processed target image. Exemplarily, the photographing scene is a super-resolution photographing scene, and the captured images include multiple images. The photographing device determines the image with the greatest clarity and / or the least noise among the multiple images as a reference image, and uses the processor to superimpose images other than the reference image on the reference image based on the target frequency to obtain the target image in the super-resolution photographing scene. The target image can be obtained by fusing the other images onto the reference image based on a fusion algorithm; or, for each pixel in the reference image, comparing the pixel value of a pixel with the same coordinate position in the other images with the pixel value of the pixel at that coordinate position in the reference image. If the difference in pixel value is less than a preset difference threshold, the average of the pixel values ​​of the pixels at that coordinate position with a difference less than the preset difference threshold is used as the pixel value of the pixel at that coordinate position in the imaging result to obtain the target image.

[0079] Exemplarily, the photographic scene is a high dynamic range photographic scene, and the captured images include images captured under short exposure, images captured under long exposure, and images captured under normal exposure. The processor uses the aforementioned target frequency to superimpose the images captured under short exposure and the images captured under long exposure on the images captured under normal exposure to obtain the target image in the high dynamic range photographic scene.

[0080] Exemplarily, the photographic scene is a night scene, and the captured images include images captured when the flash is on and images captured when the flash is off. The processor uses the image captured when the flash is off based on the aforementioned target frequency to superimpose the image captured when the flash is on to obtain the target image in the night scene photographic scene.

[0081] Among them, superimposing the image captured under short exposure and the image captured under long exposure on the image captured under normal exposure, and superimposing the image captured under flash-off state on the image captured under flash-on state are similar to the aforementioned method of superimposing images other than the baseline image among multiple images captured on the baseline image, and are not repeated here.

[0082] It should be noted that, after capturing an image, the embodiment of the present disclosure may also perform pre-processing on the captured image, such as performing white balance processing, noise reduction processing, format conversion, etc. on the image, and use the processor to process the pre-processed image based on the target frequency to obtain the target image; wherein, the image may be pre-processed based on the target frequency by the processor, or may be pre-processed based on the initial frequency by the processor, and the embodiment of the present disclosure does not limit this.

[0083] Figure 3 This is an example diagram comparing the efficiency of a super-resolution photography scene before and after the processor frequency is increased in an embodiment of the present disclosure. In the area marked by L2, each row represents the time required to process the image for super-resolution photography scene association based on the target frequency to obtain the target image after the processor frequency is increased in the super-resolution photography scene; each row in the area marked by L3 represents the time required to process the image for super-resolution photography scene association based on the initial frequency to obtain the target frequency before the processor frequency is increased in the super-resolution photography scene. Figure 3 It can be seen that the photography efficiency after the processor frequency is increased in the super-resolution photography scene is higher than the photography efficiency before the processor frequency is increased in the super-resolution photography scene.

[0084] After a photography test in an ultra-night photography scenario, the time required to associate the image with the ultra-night photography scenario based on the target frequency and obtain the target image after the processor frequency is increased was 1166.98 milliseconds (ms) and 1081.96ms respectively; before the processor frequency is increased, the time required to associate the image with the ultra-night photography scenario based on the initial frequency and obtain the target image was 1296.22ms and 1150ms respectively. It can be seen that the photography efficiency of the processor after the frequency increase in the ultra-night photography scenario is higher than the photography efficiency before the frequency increase.

[0085] In a photography test in a night owl photography scenario, the time required to process the image based on the target frequency to obtain the target image after the processor frequency is increased is 942ms; before the processor frequency is increased, the time required to process the image based on the initial frequency to obtain the target image is 963ms. It can also be seen that the photography efficiency in the night owl photography scenario is higher after the processor frequency is increased than before the processor frequency is increased.

[0086] After a photography test in a high dynamic range photography scenario, the time required to process the image for high dynamic range photography scenario association based on the target frequency to obtain the target image after the processor frequency was increased was 867.47ms and 830.02ms respectively; before the processor frequency was increased, the time required to process the image for high dynamic range photography scenario association based on the initial frequency to obtain the target image was 929.87ms and 926.93ms respectively. It can be seen that the photography efficiency of the processor after the frequency increase in the high dynamic range photography scenario is higher than the photography efficiency before the frequency increase.

[0087] It is understandable that, compared to the scheme of scheduling the processor frequency according to the relevant processor frequency scheduling rules, which causes the processor to spend a longer time processing the image at a lower frequency, such as the initial frequency, the embodiment of the present disclosure responds to detecting a photo instruction, captures an image and determines the current photo scene, and determines the target frequency of the processor for image processing after the increase based on the photo scene, thereby using the processor to process the image based on the higher target frequency, which can reduce the image processing time, improve the efficiency of outputting the target image, and thus improve the photo performance. In addition, the embodiment of the present disclosure determines the increased target frequency based on the photo scene, so that the target frequency is adapted to the photo scene, so that, for example, for a photo scene including a more complex image processing process, image processing is performed based on the target frequency of the processor after a larger increase; for a photo scene including a less complex image processing process, image processing is performed based on the target frequency of the processor after a smaller increase, thereby improving the intelligence of regulating the processor frequency on the basis of improving the photo performance, thereby improving the overall photo experience.

[0088] In some embodiments, the method further comprises:

[0089] Obtaining the temperature and / or occupancy of the processor;

[0090] The determining, based on the photographing scene, a target frequency at which the processor processes the image includes:

[0091] When the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, a target frequency for the processor to process the image in association with the photographing scene is determined based on the photographing scene.

[0092] As can be seen from the foregoing, based on the processor frequency scheduling rules, when the processor temperature is high and / or the processor occupancy is high, the processor frequency is reduced, resulting in the processor taking a longer time to process the image at a lower frequency, such as the initial frequency, during the photo shooting process. In response to this, the photo shooting device of the disclosed embodiment obtains the processor temperature based on, for example, a temperature sensor and / or obtains the processor occupancy based on a preset command (such as the top command), and when the processor temperature is greater than a preset temperature threshold and / or the occupancy is greater than a preset occupancy threshold, determines a target frequency for the processor to process the image associated with the photo shooting scene based on the photo shooting scene, so that the processor performs image processing based on the higher target frequency to obtain the target image.

[0093] In the disclosed embodiments, the processing associated with the photographic scene refers to image processing specific to the photographic scene, rather than general image processing, such as the aforementioned preprocessing. For example, if the photographic scene is a super-resolution photographic scene, the processing associated with the photographic scene may be superimposing an image other than the baseline image among the multiple images captured on the baseline image; if the photographic scene is a high dynamic range photographic scene, the processing associated with the photographic scene may be superimposing an image captured under short exposure and an image captured under long exposure on an image captured under normal exposure; if the photographic scene is a night scene, the processing associated with the photographic scene may be superimposing an image captured under flash-off state and an image captured under flash-on state.

[0094] It can be understood that the embodiment of the present disclosure increases the frequency of the processor's processing of images associated with photographing scenes when the processor temperature is greater than a preset temperature threshold and / or the processor occupancy is greater than a preset occupancy threshold. On the one hand, it can effectively reduce the restrictions on the processor frequency by the processor frequency scheduling rules, so that the processor can still perform image processing at a higher target frequency in scenarios with high temperature and high occupancy, thereby improving photographing performance; on the other hand, since the processor operating at a higher target frequency will lead to a further increase in temperature rise and occupancy, the processor of the embodiment of the present disclosure only performs the processing process of the image associated with the photographing scene, rather than the overall photographing process, thereby taking into account the temperature rise, occupancy and photographing performance of the processor.

[0095] In some embodiments, the image captured in the photographing scene includes multiple images, and the method further includes:

[0096] pre-processing each image based on the initial frequency;

[0097] The determining, when the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, based on the photographing scene, a target frequency at which the processor processes the image in association with the photographing scene includes:

[0098] When the temperature is greater than the preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, the target frequency at which the processor performs image fusion on the preprocessed multiple images is determined based on the photographing scene.

[0099] In the embodiment of the present disclosure, since the image format captured by the image sensor is usually in raw format and / or the captured image quality is not high, the photographing device pre-processes each of the multiple captured images based on the initial frequency of the processor before performing a more complex image processing process such as processing associated photographing scenes on the image; wherein the pre-processing includes the aforementioned white balance processing, noise reduction processing, format conversion, etc.; if the photographing scene is a super-resolution photographing scene, the process of selecting a reference image from multiple images can also be attributed to the pre-processing process of the embodiment of the present disclosure.

[0100] After pre-processing the image based on the initial frequency of the processor, in response to the temperature of the processor being greater than a preset temperature threshold and / or the occupancy rate being greater than a preset occupancy rate threshold, the target frequency at which the processor performs image fusion on the pre-processed multiple images is determined based on the photographing scene, so that the photographing device performs image fusion on the pre-processed multiple images based on the higher target frequency of the processor to obtain a target image.

[0101] It can be understood that, on the one hand, the photographing device pre-processes each image based on a lower initial frequency, which can reduce the processor temperature rise and processor occupancy rate. At the same time, since the pre-processed image quality is higher and / or the image format is more conducive to subsequent processing, it helps to improve the accuracy of the subsequent multi-image fusion to obtain the target image; on the other hand, due to the high complexity of the multi-image fusion algorithm, the photographing device of the embodiment of the present disclosure performs image fusion on the pre-processed multiple images based on the higher target frequency of the processor, thereby improving the quality of the fused target image and improving the photographing performance.

[0102] In some embodiments, determining a target frequency for a processor to process the image based on the photographing scene includes:

[0103] determining, based on the photographing scene, a frequency threshold at which the processor processes the image;

[0104] A frequency greater than the frequency threshold is determined as the target frequency for the processor to process the image.

[0105] In an embodiment of the present disclosure, the photographing device determines the frequency threshold for the processor to process the image based on the photographing scene; in some embodiments, based on the correspondence between the preset photographing scene and the frequency threshold, the frequency threshold corresponding to the current photographing scene is determined as the frequency threshold for the processor to process the image; wherein, the correspondence between the preset photographing scene and the frequency threshold is obtained based on multiple experiments, and in the correspondence between the preset photographing scene and the frequency threshold, the frequency threshold corresponding to the photographing scene is greater than the initial frequency of the processor.

[0106] In some embodiments, based on the correspondence between preset photographing scenes and frequency levels, the frequency level corresponding to the current photographing scene is determined, and the frequency indicated by the frequency level corresponding to the current photographing scene is determined as the frequency threshold; wherein the frequency level and the frequency have a correspondence.

[0107] After determining the frequency threshold for image processing by the processor, the camera device of the disclosed embodiment determines a frequency greater than the frequency threshold as the target frequency for image processing by the processor. The processor then processes the image at any frequency within the frequency range greater than the frequency threshold as the target frequency, generating the target image. Compared to a single target frequency, the disclosed embodiment improves camera performance while also increasing the flexibility of target frequency control.

[0108] In some embodiments, the method further comprises:

[0109] Based on the photographing scene, determining a target duration for processing the image based on the target frequency;

[0110] The processing of the image based on the target frequency by the processor and outputting the processed target image includes:

[0111] Within the target duration, the processor is used to process the image based on the target frequency, and the processed target image is output.

[0112] In an embodiment of the present disclosure, a photographing device determines a target duration for processing an image at a target frequency based on a photographing scenario, i.e., the duration for which the processor operates at the target frequency. For example, based on a preset correspondence between a photographing scenario and duration, the duration corresponding to the photographing scenario is used as the target duration for processing the image at the target frequency. For another example, based on the historical image processing time consumed in a photographing scenario, the target duration is determined as the weighted sum of the historical image processing time consumed in the photographing scenario and a preset duration increment. Thus, after determining the target duration, the photographing device uses the processor to process the image at the target frequency within the target duration to obtain a processed target image.

[0113] It should be noted that the target duration corresponding to the photographing scene in the embodiment of the present disclosure should be greater than the time consumed for image processing in the photographing scene; for example, the target duration corresponding to the super-resolution photographing scene is 5000 milliseconds.

[0114] It can be understood that the embodiment of the present disclosure limits the time the processor operates at a higher target frequency based on the target duration, taking into account the processor performing image processing at a higher target frequency within the target duration, improving image processing efficiency, and reducing problems such as temperature rise caused by the processor operating at a higher frequency for a long time.

[0115] In some embodiments, determining a target frequency for a processor to process the image based on the photographing scene includes:

[0116] In response to the photographing scene being a preset target photographing scene, a target frequency for the processor to process the image is determined based on the photographing scene.

[0117] In the embodiment of the present disclosure, the preset target scene is a photographing scene that requires relatively complex image processing, such as the aforementioned super-resolution photographing scene, high dynamic range photographing scene, and night scene photographing scene.

[0118] When the photographing scene is a preset target photographing scene, the photographing device of the embodiment of the present disclosure determines the target frequency of the improved processor for processing the image based on the photographing scene, and processes the image at the target frequency based on the processor to obtain the target image.

[0119] It is understandable that since ordinary photo-taking scenarios only require format conversion and direct image output, or low-complexity pre-processing is performed to obtain the target image, the embodiments of the present disclosure only increase the processor frequency to the target frequency for preset target scenarios with higher image processing complexity, and perform image processing based on the target frequency to improve image processing efficiency, thereby increasing the necessity of frequency increase.

[0120] In some embodiments, the method further comprises:

[0121] In response to the completion of processing the image based on the target frequency by the processor, the frequency of the processor is restored to the initial frequency.

[0122] In an embodiment of the present disclosure, after image processing is completed using the processor based on a higher target frequency, the processor frequency is restored to a lower initial frequency, and subsequent processes are executed at the initial frequency, such as saving the image to an album, etc. Through this solution, unnecessary frequency increases that increase the processor load can be reduced.

[0123] In some embodiments, the processor comprises a graphics processor.

[0124] In an embodiment of the present disclosure, the processor includes a graphics processor for image processing; the photographing device of the embodiment of the present disclosure determines the target frequency of the graphics processor for image processing based on the photographing scene, and uses the graphics processor to process the image based on the target frequency to output a target image.

[0125] It is understandable that processors such as CPU and MCU not only perform image processing, but may also execute other application-related processes, such as video playback processes, call processes, etc.; in this regard, the embodiment of the present disclosure only increases the frequency of GPU for image processing, which can reduce the impact of increasing the frequency of other processors on other application processes in the electronic device.

[0126] Figure 4 This is an example of a photographing method process shown in the embodiment of the present disclosure. Figure 2 ,Depend on Figure 4 It can be seen that the following steps are included:

[0127] S401, the electronic device receives a photo taking request;

[0128] In an embodiment of the present disclosure, a processor in an electronic device may receive a photo request based on a user operation. For example, in a smartphone, the processor may receive a photo request in response to detecting a voice command to take a photo or an operation command on a displayed photo control. The photo request may include a photo scene identifier, such as sr for a super-resolution photo scene, hdr for a high dynamic range photo scene, or sn for a night scene photo scene.

[0129] S402, the electronic device creates a corresponding photo-taking process according to scene detection;

[0130] In an embodiment of the present disclosure, a processor in an electronic device determines a current photographing scene according to an identifier of the photographing scene carried in a photographing request, and creates a photographing process corresponding to the photographing scene including the following steps.

[0131] S403, an image sensor in the electronic device captures an image;

[0132] In an embodiment of the present disclosure, a processor in an electronic device controls an image sensor (sensor node) in the electronic device to capture images based on a capture process corresponding to a capture scenario created by a capture request. For example, the image sensor is controlled to capture eight images in a super-resolution capture scenario; to capture images at short exposure, long exposure, and normal exposure in a high dynamic range capture scenario; and to capture images with the flash on and with the flash off in a night scene capture scenario.

[0133] S404, the electronic device obtains multiple original images output by the image sensor;

[0134] In an embodiment of the present disclosure, the image sensor captures multiple original images, so that the processor of the electronic device obtains the multiple original images, wherein the format of the original images is, for example, a raw format.

[0135] S405, the electronic device optimizes the image quality;

[0136] In an embodiment of the present disclosure, an electronic device performs image quality optimization processing, such as noise reduction and white balancing, on each image based on a processor at an initial frequency. The processor's initial frequency is determined based on the aforementioned processor frequency scheduling rules, and may be, for example, a preset frequency assigned; or, alternatively, a frequency adjusted based on the assigned frequency and / or processor occupancy. The processor in this embodiment of the disclosure is, for example, a graphics processor.

[0137] S406, the electronic device selects a reference image;

[0138] In an embodiment of the present disclosure, a processor in an electronic device selects a reference image from multiple images; for example, based on the processor using an initial frequency, the image with the highest clarity among the eight images corresponding to the super-resolution photography scene is used as the reference image; the image corresponding to normal exposure in the high dynamic range photography scene is used as the reference image; and the image corresponding to the flash-off state in the night scene photography scene is used as the reference image.

[0139] S407, the electronic device converts the format of the image;

[0140] In an embodiment of the present disclosure, a processor in an electronic device performs image format conversion on each image, for example, converting the image format from RAW to YUV format based on the processor at an initial frequency.

[0141] S408, the electronic device enables a process of associating a photographing scene;

[0142] In an embodiment of the present disclosure, a processor in an electronic device enables processing associated with a photographing scene; taking a super-resolution photographing scene as an example, in response to the photographing scene being identified as sr, the processor enables a plug-in algorithm associated with a super-resolution photographing scene (SRPlugin enable), and after successful enabling, the SRenable variable is assigned a value of 1.

[0143] S409, the electronic device calls a preset interface to transfer parameters;

[0144] In the disclosed embodiment, the processor in the electronic device determines whether SRenable is 1. If SRenable == 1, indicating that the processing associated with the super-resolution photography scene is successfully enabled, a preset interface is called to pass parameters. Exemplarily, the external interface PerLockMananger() of the BoostGPU software framework is called, and the passed parameters include the identification of the photography scene sr and a duration, such as 5000 milliseconds; the duration corresponds to the target duration for processing the image based on the target frequency.

[0145] S410, the electronic device determines a photographing scene;

[0146] In the embodiment of the present disclosure, the processor in the electronic device determines the photo-taking scene based on the parameters passed by PerLockMananger().

[0147] S411: The electronic device determines that the photographing scene is a super-resolution photographing scene;

[0148] In an embodiment of the present disclosure, a processor in an electronic device determines that a photographic scene is a super-resolution photographic scene, and determines a target frequency corresponding to the super-resolution photographic scene that is greater than the aforementioned initial frequency. The target frequency corresponding to the super-resolution photographic scene can be determined as the target frequency based on the aforementioned preset correspondence between the photographic scene and the frequency; or, based on the preset correspondence between the photographic scene and the frequency level, the frequency level corresponding to the current photographic scene (such as MPCTLV3_GPU_MIN_POWER_LEVEL) is determined, and the frequency indicated by the frequency level corresponding to the current photographic scene is determined as the frequency threshold, wherein the target frequency is greater than the frequency threshold. Alternatively, the maximum frequency supported by the processor hardware can be used as the target frequency, etc.

[0149] S412. The electronic device determines a frequency-increasing parameter of the processor;

[0150] In an embodiment of the present disclosure, the processor in the electronic device is expanded in the originally configured processor parameter array, and the aforementioned target frequency (or frequency threshold), target duration, and identification of the photo-taking scene are added to the processor parameter array.

[0151] S413: The processor in the electronic device completes frequency increase.

[0152] In an embodiment of the present disclosure, a processor in an electronic device increases a frequency to a target frequency based on a parameter array.

[0153] S414. During the processing time associated with the super-resolution photography scene, the electronic device controls the processor to maintain a maximum frequency.

[0154] In an embodiment of the present disclosure, a processor in an electronic device performs super-resolution photography scene association processing on an image within a target time period based on the maximum frequency supported by the processor hardware (i.e., the target frequency). For example, seven images are superimposed on a reference image to obtain a target image.

[0155] S415: The electronic device completes the process of associating the super-resolution photography scene.

[0156] In an embodiment of the present disclosure, when the processing associated with the super-resolution photography scene is completed at the target frequency, the processor in the electronic device deconstructs the current thread and restores the frequency of the processor to the initial frequency.

[0157] S416: The electronic device saves the processed image to an album.

[0158] In the embodiment of the present disclosure, the processor in the electronic device saves the target image obtained by taking the photo to an album for the user to view.

[0159] It can be understood that in order to better improve the photo-taking performance of electronic devices, the electronic devices in the embodiments of the present disclosure add optimizations such as the BoostGPU software framework design in the photo-taking process path. By calling the BoostGPU interface before algorithm processing of different photo-taking scenes, the processing process associated with the photo-taking scene can reduce factors such as the temperature control limit on the processor and the frequency limit on the processor occupancy rate. In this way, the processor frequency within the estimated photo-taking time (such as 5000ms) is increased to the maximum value supported by the hardware without restricting the normal processing state of other processes. In this way, the electronic device can more reasonably utilize the resources of the existing processor to optimize the photo-taking performance, thereby significantly improving the photo-taking performance and greatly improving the user's photo-taking experience.

[0160] Figure 5 is a diagram of a photographing device shown in an embodiment of the present disclosure, Figure 5 It is known that:

[0161] The acquisition module 501 is configured to acquire an image and determine a current photographing scene in response to detecting a photographing instruction;

[0162] A first determining module 502 is configured to determine a target frequency for a processor to process the image based on the photographing scene; wherein the target frequency is greater than an initial frequency of the processor;

[0163] The processing module 503 is configured to process the image based on the target frequency using the processor and output a processed target image.

[0164] In some embodiments, the apparatus further comprises:

[0165] an acquisition module, configured to acquire the temperature and / or occupancy rate of the processor;

[0166] The first determining module 502 is configured to determine, based on the photographing scene, a target frequency at which the processor processes the image in association with the photographing scene when the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold.

[0167] In some embodiments, the image collected in the photographing scene includes multiple images, and the apparatus further includes:

[0168] a preprocessing module, configured to preprocess each image based on the initial frequency;

[0169] The first determination module 502 is configured to determine, based on the photographing scene, the target frequency at which the processor performs image fusion on the preprocessed multiple images when the temperature is greater than the preset temperature threshold and / or the occupancy rate is greater than the preset occupancy rate threshold.

[0170] In some embodiments, the first determination module 502 is configured to determine a frequency threshold for the processor to process the image based on the photographing scene; and determine a frequency greater than the frequency threshold as the target frequency for the processor to process the image.

[0171] In some embodiments, the apparatus further comprises:

[0172] A second determining module is configured to determine, based on the photographing scene, a target duration for processing the image based on the target frequency;

[0173] The processing module 503 is configured to process the image based on the target frequency using the processor within the target duration, and output the processed target image.

[0174] In some embodiments, the first determining module 502 is configured to determine a target frequency for the processor to process the image based on the photographing scene in response to the photographing scene being a preset target photographing scene.

[0175] In some embodiments, the apparatus further comprises:

[0176] A recovery module is configured to, in response to the completion of the image processing by the processor based on the target frequency, restore the frequency of the processor to the initial frequency.

[0177] In some embodiments, the processor comprises a graphics processor.

[0178] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0179] Figure 6 FIG. 6 is a block diagram of an electronic device 600 according to an embodiment of the present disclosure. For example, the electronic device 600 may be a mobile phone, a mobile computer, or the like.

[0180] Reference Figure 6, the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0181] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 602 may include one or more modules to facilitate interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate interaction between the multimedia component 608 and the processing component 602.

[0182] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0183] The power supply assembly 606 provides power to the various components of the electronic device 600. The power supply assembly 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0184] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0185] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0186] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0187] The sensor assembly 614 includes one or more sensors for providing various aspects of the status assessment of the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of components, such as the display and keypad of the electronic device 600. The sensor assembly 614 can also detect changes in the position of the electronic device 600 or a component of the electronic device 600, the presence or absence of user contact with the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and changes in the temperature of the electronic device 600. The sensor assembly 614 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 can also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 can also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0188] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as Wi-Fi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wide band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0189] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0190] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is further provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of an electronic device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0191] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the aforementioned photographing method.

[0192] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0193] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A photographing method, characterized in that: The method comprises: In response to detecting a photographing instruction, capturing an image and determining a current photographing scene; Determining a target frequency for a processor to process the image based on the photographing scene; wherein the target frequency is greater than an initial frequency of the processor; The processor is used to process the image based on the target frequency, and a processed target image is output.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining the temperature and / or occupancy rate of the processor; The determining, based on the photographing scene, a target frequency at which the processor processes the image includes: When the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, a target frequency for the processor to process the image in association with the photographing scene is determined based on the photographing scene.

3. The method according to claim 2, characterized in that The images collected in the photographing scene include a plurality of images, and the method further includes: pre-processing each image based on the initial frequency; The determining, when the temperature is greater than a preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, based on the photographing scene, a target frequency at which the processor processes the image in association with the photographing scene includes: When the temperature is greater than the preset temperature threshold and / or the occupancy rate is greater than a preset occupancy rate threshold, the target frequency at which the processor performs image fusion on the preprocessed multiple images is determined based on the photographing scene.

4. The method according to claim 1, wherein The determining, based on the photographing scene, a target frequency at which the processor processes the image includes: determining, based on the photographing scene, a frequency threshold at which the processor processes the image; A frequency greater than the frequency threshold is determined as the target frequency for the processor to process the image.

5. The method according to claim 1, wherein The method further comprises: Based on the photographing scene, determining a target duration for processing the image based on the target frequency; The processing of the image based on the target frequency by the processor and outputting the processed target image includes: Within the target duration, the processor is used to process the image based on the target frequency, and the processed target image is output.

6. The method according to any one of claims 1 to 5, characterized in that The determining, based on the photographing scene, a target frequency at which the processor processes the image includes: In response to the photographing scene being a preset target photographing scene, a target frequency for the processor to process the image is determined based on the photographing scene.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: In response to the completion of processing the image based on the target frequency by the processor, the frequency of the processor is restored to the initial frequency.

8. The method according to any one of claims 1 to 5, characterized in that The processor includes a graphics processor.

9. A photographing device, characterized in that: The device comprises: An acquisition module, configured to acquire an image and determine a current photographing scene in response to detecting a photographing instruction; A first determining module is configured to determine a target frequency for a processor to process the image based on the photographing scene; wherein the target frequency is greater than an initial frequency of the processor; A processing module is used to process the image based on the target frequency using the processor and output the processed target image.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 8.

Images