Photographing method and related equipment thereof

Abstract

The invention provides a photographing method and related equipment thereof, and relates to the field of image processing, and the photographing method comprises the steps: determining the memory occupation amount for the x-th photographing times; when the first memory threshold is smaller than the first memory threshold, determining the x-level processing algorithm as a first photographing algorithm, and determining the x-level processing frame number as a first frame number; when the x-level processing algorithm is greater than or equal to the first memory threshold value and less than the second memory threshold value, determining the x-level processing algorithm as a second photographing algorithm, and determining the x-level processing frame number as a second frame number; when the first memory threshold value is greater than the second memory threshold value, determining the x-level processing algorithm as a third photographing algorithm, and determining the x-level processing frame number as a third frame number; processing the x-level processing frame number in the original image by using an x-level processing algorithm to obtain a corresponding shot image; and storing the shot image. According to the method, the memory occupancy is distinguished, and processing algorithms with different time lengths and / or different processing frame numbers are adaptively selected for processing, so that the pressure of memory increment can be reduced, and continuous and rapid photographing is realized.

Description

Photographing method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 25, 2022, with application number 202211026658.0 and application name “Photographing method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image technology, and in particular, to a photographing method and related equipment. Background Art

[0003] With the popularity of electronic devices with camera functions in our lives, taking photos with electronic devices has become a daily behavior.

[0004] Currently, most electronic devices use multi-frame photo processing algorithms to achieve better image quality when processing photos. However, the more frames the algorithm processes, the longer the processing time. In this case, if the user triggers multiple photos in a row, the background will appear to be waiting for photo requests. The accumulated images waiting to be processed will cause the memory usage to continue to expand. When the remaining memory is insufficient to support photo processing, the background will no longer respond to the user's photo command, and the electronic device will be unable to continue taking photos. A new solution is urgently needed to solve this problem.

[0005] Summary of the Invention

[0006] The present application provides a photographing method and related equipment, which can reduce the pressure of memory increment and achieve continuous and rapid photographing by distinguishing the memory usage and adaptively selecting processing algorithms with different processing times and / or processing frame numbers.

[0007] In a first aspect, a photographing method is provided, which is applied to an electronic device including a camera, the method comprising:

[0008] Displaying a first interface, wherein the first interface includes a first control;

[0009] When a first operation on the first control is detected, capturing an original image in response to the first operation;

[0010] Determine the memory usage for the xth number of photos;

[0011] When the memory usage is less than the first memory threshold, determining the x-level processing algorithm as the first photographing algorithm, and determining the x-level processing frame number as the first frame number, where x is an integer greater than 1;

[0012] When the memory usage is greater than or equal to the first memory threshold but less than a second memory threshold, determining that the x-level processing algorithm is a second photographing algorithm, and determining that the x-level processing frame number is a second frame number;

[0013] When the memory usage is greater than the second memory threshold, determining that the x-level processing algorithm is a third photographing algorithm, and determining that the x-level processing frame number is a third frame number, and the first memory threshold is less than the second memory threshold;

[0014] Processing the x-level processing frames in the original image using the x-level processing algorithm to obtain a corresponding captured image;

[0015] Saving the captured image;

[0016] Wherein, when the first photographing algorithm, the second photographing algorithm and the third photographing algorithm are the same, the first frame number, the second frame number and the third frame number gradually decrease;

[0017] When the processing times of the first photographing algorithm, the second photographing algorithm, and the third photographing algorithm gradually decrease, the first number of frames, the second number of frames, and the third number of frames are the same or gradually decrease.

[0018] Optionally, the first operation is a click operation. The first operation may also include a voice instruction operation or other operations that instruct the electronic device to take a photo.

[0019] An embodiment of the present application provides a photographing method. In response to a user's first operation on a first control, the electronic device uses a camera to capture an original image. Then, by determining the size of the memory usage and adaptively selecting algorithms with different processing times for processing based on the size of the memory usage, for example, when the memory usage is small, an algorithm with a longer processing time is selected for processing, and when the memory usage gradually increases, an algorithm with a shorter processing time is selected for processing. This can reduce the pressure of memory increment in the electronic device, improve the efficiency of photographing processing, respond to the user's photographing needs in a timely manner, and achieve continuous and rapid photographing.

[0020] In a possible implementation of the first aspect, the method further includes:

[0021] For the x-th number of photographs, when the device temperature is greater than or equal to the temperature threshold, processing one frame of the original image using a single-frame algorithm to obtain a corresponding photographed image;

[0022] When the first photographing algorithm, the second photographing algorithm, and the third photographing algorithm are the same, the processing time of the single-frame algorithm is shorter than the processing time of the third photographing algorithm.

[0023] When the processing time of the first photographing algorithm, the second photographing algorithm and the third photographing algorithm gradually decreases, the processing time of the single-frame algorithm is the same as the processing time of the third photographing algorithm, or the processing time of the single-frame algorithm is shorter than the processing time of the third photographing algorithm.

[0024] The device temperature refers to the temperature inside the electronic device.

[0025] In this implementation, when the device temperature is higher than the temperature threshold, it can switch to a single-frame algorithm to process one frame of the original image, thereby minimizing the workload of the electronic device, slowing down the growth of heat, and reducing the incremental heat pressure, so that the electronic device can take more photos before it stops working, thereby meeting the user's photography needs.

[0026] It should be understood that since the single-frame algorithm can be processed only in the image signal processor, the processing speed of the single-frame algorithm is relatively fast and the processing time is relatively short.

[0027] In a possible implementation of the first aspect, the method further includes:

[0028] When the first operation corresponds to the first photo-taking time, performing environmental detection for the scene to be photographed;

[0029] Determine the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first number of photographs according to the environmental detection result;

[0030] The first-level processing algorithm is used to process the first-level processing frame number in the original image collected corresponding to the first number of photographs to obtain a corresponding photographed image.

[0031] In this implementation, the present application can determine different scenes by performing environmental detection on the shooting scene, and then, based on the different scenes that have been segmented, select different algorithms, and select different numbers of frames from the corresponding collected original images for processing, so as to adaptively improve the quality and effect of the captured images corresponding to the first number of photos in each scene.

[0032] In a possible implementation of the first aspect, the environment detection includes at least illumination detection and dynamic range detection, and the environment detection result includes at least illumination and dynamic range values;

[0033] Determining the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first number of photographs according to the environmental detection result includes:

[0034] When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, determining the number of level 1 processing frames to be a1 frames, and determining the level 1 processing algorithm to be the A1 algorithm;

[0035] When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, determining the number of level 1 processing frames to be b1 frames, and determining the level 1 processing algorithm to be the B1 algorithm;

[0036] When the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, determining the number of level 1 processing frames to be c1 frames and determining the level 1 processing algorithm to be the C1 algorithm;

[0037] When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, determining the number of level 1 processing frames to be d1 frames, and determining the level 1 processing algorithm to be the D1 algorithm;

[0038] Wherein, a1, b1, c1 and d1 are all integers greater than or equal to 1.

[0039] Optionally, the a1 frame, b1 frame, c1 frame or d1 frame may be selected from all frames of the original image in the acquisition order, or the a1 frame, b1 frame, c1 frame or d1 frame may be arbitrarily extracted from all frames of the original image.

[0040] In a possible implementation of the first aspect, using the level 1 processing algorithm, processing the level 1 processed frames in the original image corresponding to the first number of photographs to obtain the corresponding captured image includes:

[0041] When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the A1 algorithm is used to process the a1 frame in the original image corresponding to the first photographing time to obtain a corresponding photographed image;

[0042] When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, the B1 algorithm is used to process the b1 frame in the original image corresponding to the first photographing time to obtain a corresponding photographed image;

[0043] When the illumination is greater than or equal to the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the C1 algorithm is used to process the C1 frame of the original image collected corresponding to the first shooting time to obtain a corresponding captured image;

[0044] When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, the D1 algorithm is used to process the d1 frame in the original image collected corresponding to the first shooting time to obtain a corresponding captured image.

[0045] Optionally, when the dynamic range value is greater than or equal to the dynamic range threshold, the first photo-taking number corresponds to the a1 frame in the collected original image, and / or the first photo-taking number corresponds to the c1 frame in the collected original image, including at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image.

[0046] In this implementation, when the processed original images include long-exposure, normal-exposure, and short-exposure original images, the dynamic range and image details of the image can be adjusted so that the content presented by the obtained captured image is more realistic and the image quality is relatively better.

[0047] In a possible implementation of the first aspect, after determining that the x-level processing algorithm is a third photographing algorithm and determining that the x-level processing frame number is a third frame number, the method further includes:

[0048] Determining a frequency of a first operation on the first control, and a difference between a memory watermark and the memory usage;

[0049] When the frequency meets the preset frequency condition or the difference is less than the preset difference, determining the number of photos to be taken in the background waiting;

[0050] The x-level processing algorithms corresponding to the number of times of photos waiting in the background are respectively used as the initial processing algorithms;

[0051] The x-level processing algorithm corresponding to the most recently determined number of photos is used as the target processing algorithm;

[0052] Using the target processing algorithm to replace the initial processing algorithm of each of the number of photos waiting in the background;

[0053] When any one of the photographing times waiting in the background is called for processing, the target processing algorithm is used to process the x-level processing frames in the original image corresponding to the any one of the photographing times to obtain a corresponding photographed image.

[0054] In this implementation, since the processing algorithm corresponding to the most recently determined number of photos is determined based on the current memory situation, the processing algorithm for the waiting number of photos is adjusted accordingly, which can control the degree of memory accumulation, reduce memory usage, and achieve continuous and rapid photography.

[0055] In a possible implementation of the first aspect, the first interface refers to a photographing interface, and the first control refers to a control for instructing photographing.

[0056] In a possible implementation of the first aspect, the first interface refers to a video recording interface, and the first control refers to a control for instructing snapshot.

[0057] In a second aspect, an image processing apparatus is provided, which includes a unit for executing each step in the above first aspect or any possible implementation of the first aspect.

[0058] According to a third aspect, an electronic device is provided, comprising one or more processors and a memory;

[0059] The memory is coupled to the one or more processors, and the memory is used to store computer program code, wherein the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method provided in the above first aspect or any possible implementation of the first aspect.

[0060] In a fourth aspect, a chip is provided, which is applied to an electronic device, and the chip includes one or more processors, and the processors are used to call computer instructions to enable the electronic device to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0061] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes the method provided in the first aspect or any possible implementation of the first aspect.

[0062] In a sixth aspect, a computer program product is provided, comprising: a computer program code, which, when executed by an electronic device, enables the electronic device to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0063] The embodiment of the present application provides a photographing method and related devices. In response to a first operation of a user on a first control, the electronic device uses a camera to capture an original image. When the first operation corresponds to the first photographing number, an environmental detection is performed on the scene to be photographed, and then, based on the environmental detection result, the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first photographing number are determined. When the first operation corresponds to the second photographing number and subsequent photographing numbers, the memory usage is first detected, the determined memory usage size is compared with a preset memory threshold, and then, based on different memory usage sizes, different processing frames and processing algorithms are selected.

[0064] When processing the first number of photos, since the scene is subdivided according to the environmental detection results, different algorithms and frame rates are selected according to different scenes, so that the quality and effect of the captured images corresponding to the first number of photos in each scene can be adaptively improved.

[0065] When processing the second and subsequent photos, the memory usage is subdivided and different algorithms and frame rates are selected according to different memory usage. When the memory usage is larger, the algorithm with shorter processing time and / or the number of processing frames is selected, thereby reducing the pressure of memory increment, improving processing efficiency, and meeting the needs of fast photo taking. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] FIG1 is a schematic diagram of a scenario applicable to an embodiment of the present application;

[0067] FIG2 is a schematic diagram of another scenario applicable to the embodiment of the present application;

[0068] FIG3 is a flow chart of a photographing method provided in an embodiment of the present application;

[0069] FIG4 is a schematic diagram of an interface for a click operation provided in an embodiment of the present application;

[0070] FIG5 is a schematic diagram of another interface for click operation provided in an embodiment of the present application;

[0071] FIG6 is a schematic diagram of an image processing process provided by an embodiment of the present application;

[0072] FIG7 is a schematic diagram of a process of environmental detection provided by an embodiment of the present application;

[0073] FIG8 is a table provided in an embodiment of the present application;

[0074] FIG9 is a flow chart of an algorithm for determining the corresponding number of frames according to memory usage according to an embodiment of the present application;

[0075] FIG10 is a schematic diagram of the growth of memory usage provided by an embodiment of the present application;

[0076] FIG11 is a flow chart of another photographing method provided in an embodiment of the present application;

[0077] FIG12 is a flow chart of another photographing method provided in an embodiment of the present application;

[0078] FIG13 is a flow chart of another photographing method provided in an embodiment of the present application;

[0079] FIG14 is another table provided in an embodiment of the present application;

[0080] FIG15 is another table provided in an embodiment of the present application;

[0081] FIG16 is another table provided in an embodiment of the present application;

[0082] FIG17 is a schematic diagram of the processing process corresponding to FIG16;

[0083] FIG18 is a schematic diagram of a display interface of an electronic device provided in an embodiment of the present application;

[0084] FIG19 is a schematic diagram of a display interface of another electronic device provided in an embodiment of the present application;

[0085] FIG20 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0086] FIG21 is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0087] FIG22 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0088] FIG23 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] The technical solution in this application will be described below with reference to the accompanying drawings.

[0090] In the description of the embodiments of this application, unless otherwise specified, " / " represents or. For example, A / B can represent A or B. "And / or" in this article is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" means two or more than two.

[0091] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0092] First, some of the terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0093] 1. The RGB (red, green, blue) color space refers to a color model related to the structure of the human visual system. Based on the structure of the human eye, all colors are considered to be different combinations of red, green, and blue.

[0094] 2. YUV color space refers to a color encoding method, where Y represents brightness, and U and V represent chrominance. While the RGB color space focuses on the human eye's perception of color, the YUV color space emphasizes the visual sensitivity to brightness. RGB and YUV color spaces are convertible.

[0095] 3. Pixel value refers to the set of color components corresponding to each pixel in a color image in the RGB color space. For example, each pixel corresponds to a set of three primary color components, where the three primary color components are red (R), green (G), and blue (B).

[0096] 4. Bayer pattern color filter array (CFA). When an image is converted from an actual scene into image data, the image sensor usually receives the red channel signal, the green channel signal, and the blue channel signal respectively, and then synthesizes the information of the three channel signals into a color image. However, this solution requires three filters at each pixel position, which is expensive and difficult to manufacture. Therefore, a layer of color filter array can be covered on the surface of the image sensor to obtain information of the three channel signals. The Bayer pattern color filter array refers to the arrangement of filters in a checkerboard pattern. For example, the minimum repeating unit in the Bayer pattern color filter array is: a filter for obtaining a red channel signal, two filters for obtaining a green channel signal, and a filter for obtaining a blue channel signal arranged in a 2×2 manner.

[0097] 5. Bayer format image, that is, the image output by an image sensor based on a Bayer format color filter array. The pixels of multiple colors in the image are arranged in a Bayer format. Among them, each pixel in the Bayer format image corresponds to a channel signal of only one color. For example, since human vision is more sensitive to green, it can be set that green pixels (pixels corresponding to green channel signals) account for 50% of all pixels, blue pixels (pixels corresponding to blue channel signals) and red pixels (pixels corresponding to red channel signals) each account for 25% of all pixels. Among them, the minimum repeating unit of the Bayer format image is: one red pixel, two green pixels and one blue pixel arranged in a 2×2 manner. The Bayer format image is an image in the RAW domain.

[0098] 6. Shooting Parameters

[0099] Shooting parameters include shutter speed, exposure time, aperture value (AV), exposure value (EV), and ISO. These are described below.

[0100] The shutter controls the amount of time light enters the camera, thereby determining the image exposure time. The longer the shutter remains open, the more light enters the camera, and the corresponding image exposure time becomes longer. Conversely, the shorter the shutter remains open, the less light enters the camera, and the corresponding image exposure time becomes shorter.

[0101] Exposure time refers to the amount of time the shutter remains open to allow light to reach the camera's photosensitive surface. Exposure time is determined by the sensitivity of the photosensitive material and the illumination on the photosensitive surface. Longer exposure times allow more light to enter the camera, while shorter exposure times reduce the amount of light entering. Therefore, long exposure times are required in low-light scenes, while short exposure times are required in backlit scenes.

[0102] The aperture value (f-number) is the ratio of the focal length of the camera lens to the diameter of the lens' aperture. The larger the aperture value, the more light enters the camera. The smaller the aperture value, the less light enters the camera.

[0103] The exposure value is a combination of exposure time and aperture value to represent the light transmission capacity of the camera lens. The exposure value can be defined as:

[0104] Where N is the aperture value; t is the exposure time in seconds.

[0105] ISO is a measure of a film's sensitivity to light, also known as its speed or gain. Insensitive film requires a longer exposure time to achieve an image with the same brightness as a more sensitive film. Sensitive film requires a shorter exposure time to achieve an image with the same brightness as an insensitive film.

[0106] Among the shooting parameters, shutter, exposure time, aperture value, exposure value and ISO, electronic devices can use algorithms to implement at least one of auto focus (AF), automatic exposure (AE), and auto white balance (AWB) to achieve automatic adjustment of these shooting parameters.

[0107] Autofocus involves electronic devices adjusting the focus lens position to obtain the highest image frequency components, thereby achieving higher image contrast. Focusing is a cumulative process, where the electronic device compares the contrast of images captured with the lens in different positions to determine the lens position that maximizes image contrast, and thus determines the focal length for focus.

[0108] Automatic exposure refers to electronic devices automatically setting the exposure value based on available lighting conditions. Electronic devices can automatically set the shutter speed and aperture value based on the exposure value of the currently captured image to achieve automatic exposure setting.

[0109] The color of objects changes depending on the color of the light projected onto them. Images captured by electronic devices under different lighting colors will have different color temperatures. White balance is closely related to ambient light. Regardless of ambient lighting conditions, electronic device cameras can detect white and use it as a reference to reproduce other colors. Automatic white balance allows electronic devices to adjust the color fidelity of images based on lighting conditions. The 3A functions are autofocus, auto exposure, and auto white balance.

[0110] Exemplarily, the exposure value may be any one of -24, -4, -3, -2, -1, 0, 1, 2, 3, 4, and 24.

[0111] The exposure image corresponding to EV0 is used to indicate the exposure image captured by the electronic device using an algorithm to achieve exposure using an exposure value of 0. The exposure image corresponding to EV-2 is used to indicate the exposure image captured by the electronic device using an algorithm to achieve exposure using an exposure value of -2. The exposure image corresponding to EV1 is used to indicate the exposure image captured by the electronic device using an algorithm to achieve exposure using an exposure value of 1. The same applies to other images and will not be repeated here.

[0112] Each increase in the exposure value by 1 changes the exposure by one stop, essentially doubling the exposure (the integral of the light intensity received by a certain element on the surface of an object over time t). For example, doubling the exposure time or the aperture area. This increase in exposure corresponds to a slower shutter speed and a smaller f-number. Thus, an increase of EV0 relative to EV-2 represents an increase of 2, a change of two stops. Similarly, an increase of EV1 relative to EV0 represents an increase of 1, a change of one stop.

[0113] Here, when the exposure value EV is equal to 0, it is generally the optimal exposure value under the current lighting conditions. Accordingly, the exposure image obtained by the electronic device under the condition of EV0 is the optimal exposure image under the current lighting conditions. This optimal exposure image can also be called the reference exposure image.

[0114] It should be understood that the “optimal” exposure image refers to the exposure image determined by a given electronic device through an algorithm. When the electronic device is different, the algorithm is different, or the current lighting conditions are different, the determined optimal exposure image will be different.

[0115] The above is a brief introduction to the nouns involved in the embodiments of this application, and no further details will be given below.

[0116] With the popularity of electronic devices with camera functions in our lives, taking photos with electronic devices has become a daily behavior.

[0117] Most electronic devices use multi-frame photography algorithms to achieve better image quality when processing photos. However, the more frames an algorithm processes, the longer the processing time. For example, when processing 9 frames, the time from the user pressing the capture button on the electronic device's display interface to the image processing being completed and available for the user to view, or the end-to-end duration of the algorithm, is approximately 2.5 seconds. When processing more than 9 frames, the end-to-end duration of the algorithm will be greater than 2.5 seconds.

[0118] When an electronic device is processing a photo, it will only process the next photo after the previous one is completed. Based on this single-threaded processing, if the user triggers multiple photo commands in succession, and the interval between triggering photos is less than the aforementioned 2.5s, the electronic device will be unable to respond in time because the photo algorithm takes a long time to process each photo, resulting in a gradual increase in the number of photos waiting in the background. As the number of waiting photos increases, the accumulated data stored in the background to be processed also increases, continuously expanding the memory usage. When the remaining memory is insufficient to support photo processing, the background will no longer respond to the photo command triggered by the user.

[0119] In this way, no matter how the user clicks the camera button, it feels like the camera button cannot be pressed. At this time, the electronic device is equivalent to being unable to take pictures normally, resulting in a very poor experience for the user.

[0120] In view of this, an embodiment of the present application provides a photo-taking method. In response to a user's first operation on a first control, the electronic device uses a camera to capture an original image once. Then, by determining the size of the memory usage and adaptively selecting algorithms with different processing times for processing based on the size of the memory usage, for example, when the memory usage is small, an algorithm with a longer processing time is selected for processing, and when the memory usage gradually increases, an algorithm with a shorter processing time is selected for processing. This can reduce the pressure on memory increment in the electronic device, improve the photo processing efficiency, respond to the user's photo needs in a timely manner, and achieve continuous and fast photo taking.

[0121] The photographing method provided in the embodiments of the present application can be applied to various electronic devices.

[0122] In some embodiments of the present application, the electronic device may be various camera devices such as sports cameras and digital cameras, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc., or may be other devices or apparatuses capable of displaying images. The embodiments of the present application do not impose any restrictions on the specific type of electronic device.

[0123] The following description uses a mobile phone as an example electronic device. The mobile phone may include a display screen and one or more cameras. The mobile phone uses one or more cameras to capture a scene to produce images with different characteristics. The display screen is used to display the processed images.

[0124] The "to-be-captured scene" refers to the scene that the user desires to capture. If the user uses the camera of an electronic device to point at a scene containing an object, the scene containing the object becomes the to-be-captured scene. It should be understood that the to-be-captured scene does not refer to a specific scene, but rather to the scene that the camera is pointing at in real time.

[0125] Based on this, the application scenarios provided by the embodiments of the present application are introduced in conjunction with Figures 1 and 2. Figure 1 shows a schematic diagram of a scenario applicable to the embodiments of the present application, and Figure 2 shows a schematic diagram of another scenario applicable to the embodiments of the present application.

[0126] Application scenario 1: photo taking

[0127] As shown in FIG1 (a), the electronic device 100 has a camera application installed. In addition, multiple applications are installed, and the present embodiment does not impose any restrictions on this. For example, in response to a user clicking on the camera application, when the electronic device 100 runs the camera application, the electronic device 100 displays a shooting interface as shown in FIG1 (b).

[0128] The shooting interface includes multiple shooting modes of the camera application, such as large aperture mode 41, night scene mode 42, portrait mode 43, photo mode 44, video mode 45, etc. The shooting interface also includes a first control, which is a shooting key 50. The shooting key 50 is used to indicate the current shooting mode. For example, when the camera is turned on, the shooting key 50 defaults to indicating the current shooting mode as photo mode 44.

[0129] As shown in FIG1(b), the shooting interface also includes a viewfinder 60, which can be used to display a real-time preview image before taking a photo. The shooting interface also displays a second control, which is a zoom option 61. The user can select the desired zoom factor in zoom option 61, for example, 0.5x, 2x, or 50x.

[0130] As shown in (a) and (b) in Figure 1, when the user turns on the camera and the current shooting mode defaults to the photo mode, after the user points the camera on the mobile phone at the scene to be photographed, the viewfinder 60 can display a preview image before taking the photo in real time; then, in response to the user's continuous multiple clicks on the shooting key 50, the electronic device can call the photo method provided by the present application, and adaptively select different photo algorithms for processing according to the size of the memory, thereby reducing the pressure of memory increment, improving the efficiency of photo processing, responding to the user's photo needs in a timely manner, realizing fast photo taking, and avoiding the situation in the related technology where the background cannot respond to the photo command.

[0131] Application scenario 2: Scenes captured in video

[0132] As shown in (a) of FIG2 , a camera application is installed on the electronic device 100. Exemplarily, in response to a user clicking operation on the camera application, when the electronic device 100 runs the camera application, the electronic device displays a shooting interface as shown in (b) of FIG2 .

[0133] The description of this shooting interface is the same as that of FIG1( b ) and is not repeated here. When the camera is turned on, the default shooting button 50 indicates that the current shooting mode is photo mode 44. For example, in response to a left swipe operation by the user, the shooting mode can be switched from photo mode 44 to video mode 45.

[0134] As shown in FIG2(c), when the shooting mode is switched to the video mode 45, the viewfinder 60 included in the shooting interface can be used to display a preview image before recording in real time. For example, in response to a user clicking the shooting key 50, when the electronic device 100 starts recording, the electronic device can display the video interface shown in FIG2(d).

[0135] As shown in (d) in Figure 2, the video recording interface can display the currently shot video frame (as shown in the figure, a man walking with a shopping bag), the shooting progress (as shown in the figure, 00:12), the status icon (the shooting icon in front of the shooting progress in the figure is used to indicate that the current shooting state is in progress), and a third control, for example, a snapshot control 70. Of course, the shooting interface can also include other controls, such as: zoom option 61, end control 80, pause / continue control 90, etc., which is not limited in this embodiment of the present application.

[0136] Among them, the pause / resume control 90 is used to display a pause icon (see (d) in Figure 2) during video capture, and is also used to pause the current video capture process when the pause icon is clicked; and to display a capture icon when video capture is paused, and to continue the current video capture process when the user clicks the capture icon (not shown in Figure 2). The end control 80 is used to end the current video capture process. The snapshot control 70 is used to capture a photo without pausing or ending the current video capture process.

[0137] As shown in (c) and (d) in Figure 2, when a user wants to take a snapshot of a man walking with a shopping bag, the user can click the snapshot control 70 in the shooting interface to obtain a snapshot photo. The snapshot photo is stored as an image in the gallery of the electronic device. If the user clicks the snapshot control 70 in the recording interface multiple times in a row, the electronic device can call the photo taking method provided by this application, and adaptively select different photo taking algorithms for processing according to the size of the memory, thereby reducing the pressure of memory increment, improving the efficiency of snapshot processing, and promptly responding to the user's photo taking needs, achieving fast snapshot, and avoiding the situation in the related art where the background cannot respond to the photo taking command.

[0138] The above are two examples of application scenarios and do not limit the application scenarios of this application.

[0139] The photographing method provided in the embodiment of the present application is described in detail below with reference to Figures 3 to 19.

[0140] FIG3 is a flow chart of a photographing method provided by the present application. The photographing method is applied to an electronic device including a camera, such as a mobile phone. As shown in FIG3 , the method may include the following steps S11 to S19.

[0141] S11. The electronic device displays a first interface, where the first interface includes a first control.

[0142] Optionally, as shown in (b) of FIG1 , the first interface may be a shooting interface, and the first control is used to indicate a shooting key 50 in the shooting interface.

[0143] Optionally, as shown in (d) of FIG. 2 , the first interface may be a video recording interface, and the first control is used to indicate a snapshot control 70 in the video recording interface.

[0144] Of course, the first interface may also be other interfaces, and the first control may correspondingly be a control on other interfaces for instructing taking photos. This embodiment of the present application does not impose any restrictions on this.

[0145] S12: When a first operation of the user on the first control is detected, the electronic device uses a camera to capture an original image in response to the first operation.

[0146] Optionally, the first operation may be a click operation on the first control, or may be a voice instruction operation or other operation to instruct the electronic device to take a photo, and the embodiment of the present application does not impose any limitation on this.

[0147] The click operation refers to the behavior of a user touching the first control for a short time and then leaving the control.

[0148] For example, Figure 4 shows a schematic diagram of an interface for a click operation. As shown in (a) and (b) in Figure 4, the user's finger pressing the shooting key 50 and then lifting it is recorded as a click operation. The time from the finger pressing the shooting key 50 to the finger lifting the shooting key 50 is the duration of a click operation, which is usually very short.

[0149] For another example, Figure 5 shows an interface diagram of another click operation. As shown in (a) and (b) in Figure 5, the user's finger pressing the snapshot control 70 and then lifting it is recorded as a click operation, and the time from the finger pressing the snapshot control 70 to the finger lifting the snapshot control 70 is the duration of one click operation.

[0150] 5 , illustratively, when the camera is turned on for recording, when the user clicks the snapshot control 70 once, in response to the click operation, the electronic device uses the camera to capture an original image once.

[0151] When the user clicks the snapshot control 70 multiple times, the electronic device can capture multiple original images using the camera in response to the multiple clicks. It should be understood that when the user clicks the snapshot control 70 multiple times, the first click corresponds to the first photo capture, the second click corresponds to the second photo capture, the third click corresponds to the third photo capture, and so on. The subsequent clicks are not further described.

[0152] It should be noted that when the camera is turned off and then reopened, the number of photos taken needs to be counted again from 1. Alternatively, when the interval between two adjacent click operations exceeds the first preset interval and there is no data waiting to be processed in the background, the number of photos taken also needs to be counted again from 1. The first preset interval is at least longer than the processing time of a single photo. The specific length of the first preset interval can be set and modified as needed, and this embodiment of the application does not impose any restrictions on this.

[0153] For example, when a user opens the camera for the first time and clicks the capture button 50 for the first time, the electronic device captures an original image using the camera in response to the click, and the number of photos taken this time corresponds to the first photo count. Then, the user closes and reopens the camera and clicks the capture button 50 again. In response to the current click, the electronic device captures an original image using the camera again. At this time, since the camera is reopened, the number of photos taken this time should be re-recorded as the first photo count.

[0154] For example, FIG6 provides a schematic diagram of an image processing process.

[0155] As shown in FIG6 , when a user opens the camera application and performs a first click operation K1 on the capture button 50, the electronic device captures an original image using the camera in response to this click operation K1, and this number of photos can be recorded as the first number of photos. However, after a long period of time, the user performs a next click operation K1' on the capture button 50. In response to this click operation K1', the electronic device captures an original image using the camera again. Since the interval between the click operations K1 and K1' exceeds the first preset interval duration, and there is no other data waiting to be processed in the background, the corresponding number of photos must be restarted from 1, that is, the number of photos is recorded as the first number of photos. When the user subsequently performs a click operation K2' on the capture button, in response to this click operation K2', the electronic device captures an original image using the camera again. Since the interval between the click operations K2' and K1' is less than the first preset interval duration, this number of photos can be recorded as the second number of photos. When the user performs a click operation K3' on the shooting key 50 again, in response to the click operation K3', the electronic device uses the camera to collect the original image again. Since the interval time between the click operation K3' and the click operation K2' is less than the first preset interval time, the number of photos taken this time can be recorded as the third time. The subsequent operations are similar and will not be repeated here.

[0156] It should be noted that if the interval between two adjacent click operations exceeds the first preset interval, but there is still data waiting to be processed in the background, it means that the user has performed many click operations before, and some data has not been processed in time due to the high click frequency and the slow background processing. At this time, the number of photos cannot be accumulated again from 1, and the original number of records should be maintained.

[0157] When an electronic device includes multiple cameras, one or more cameras may be used to capture a raw image in response to a single click operation, with each camera capturing one or more raw image frames. In other words, one or more raw image frames may be captured each time, and the specific number of frames may be set and modified as needed, without any limitation in the present embodiment.

[0158] It should also be understood that the original image may be an image in the RAW domain, but when the camera is a black and white camera, the original image may be a grayscale image; when the camera is a multispectral camera, the original image may be a multispectral image including multiple color channel signals. The format or characteristics of the original image changes with the camera, and the embodiments of the present application do not impose any restrictions on this.

[0159] In addition, the original image collected at one time may further include at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image.

[0160] Among them, the long-exposure original image refers to the image obtained after a longer exposure time during shooting, and the short-exposure original image refers to the image obtained after a shorter exposure time during shooting. It should be understood that "long" and "short" are relative to the "normal" exposure time. When the exposure time corresponding to the normally exposed original image is different, the long exposure and short exposure will also change accordingly.

[0161] When the multiple frames of original images include long-exposure original images, and / or normal-exposure original images, and / or short-exposure original images, the number of frames of the long-exposure original images, the number of frames of the normal-exposure original images, and the number of frames of the short-exposure original images can be selected and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0162] S13: When the first operation corresponds to the first photo-taking time, perform an environment detection on the scene to be photographed.

[0163] Among them, environmental detection at least includes illuminance value (LV) detection and dynamic range detection. Of course, environmental detection can also include detection items, which can be set and modified as needed. The embodiment of the present application does not impose any restrictions on this.

[0164] It should be understood that illumination detection refers to estimating the ambient brightness in the scene to be shot. Illumination refers to the degree to which an object is illuminated, specifically refers to the energy of visible light received per unit area of ​​the object, referred to as illumination, and the unit is lux. Streets at night, dimly lit rooms, etc. are generally low illumination. In contrast, playgrounds under the sun, stages under spotlights, etc. can be called medium and high illumination. If the illumination of the scene to be shot is relatively low, when using an electronic device to shoot the scene, it will be blurry and the effect will be poor. If the illumination of the scene to be shot is relatively high, when using an electronic device to shoot the scene, it will be clearer and the shooting effect will be good.

[0165] Here, for example, a photoelectric sensor can be set in the electronic device to detect the illumination of the scene to be captured. Alternatively, the illumination can be calculated using the following formula based on the camera's exposure parameters, such as exposure time, sensitivity, aperture, and other parameters, as well as the received response value:

[0166] Among them, Exposure is the exposure time, Aperture is the aperture size, ISO is the sensitivity, and Luma is the average value of Y in the XYZ color space.

[0167] When the exposure parameters are the same, a higher response value indicates a higher illumination of the scene to be photographed, and thus a larger value of the calculated illumination is also obtained.

[0168] It should be understood that dynamic range detection refers to the detection of the dynamic range value of the scene to be captured. The dynamic range value is used to represent the brightness range of the pixels in the image, that is, the number of grayscale levels between the "brightest" pixel and the "darkest" pixel in the image. The larger the dynamic range value of an image, the richer the brightness levels it can represent and the more realistic the visual effect of the image. The expression of the dynamic range value can be:

[0169] Where dynamic range is the dynamic range value, bright is the brightness of the "brightest" pixel, and dark is the brightness of the "darkest" pixel. The unit of dynamic range is stop.

[0170] Here, based on the detected dynamic range value, it can be determined whether the scene being captured is a low dynamic range (LDR) scene or a high dynamic range (HDR) scene. A low dynamic range scene refers to a scene with either low or high ambient light intensities, resulting in a narrow dynamic range; a high dynamic range scene refers to a scene with low light intensities in some areas and high light in others, resulting in a wide dynamic range.

[0171] S14. Determine the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first number of photographs according to the environmental detection result.

[0172] In combination with the illumination detection and dynamic range detection methods, FIG7 is a flow chart of an environment detection process provided in an embodiment of the present application.

[0173] Since S13 includes the following steps: when the first operation corresponds to the first photo capture, performing environmental detection on the scene to be captured, and the environmental detection includes illumination detection, the environmental detection result accordingly includes the detected illumination. Similarly, since the environmental detection also includes dynamic range detection, the environmental detection result accordingly also includes the detected dynamic range value. The methods for illumination detection and dynamic range detection can be referred to the above description and will not be repeated here.

[0174] In combination with the illumination and dynamic range values ​​in the detected environment detection results, as shown in FIG7 , the above S14 may include the following S141 to S146 .

[0175] S141. Determine whether the detected illuminance is less than an illuminance threshold.

[0176] Among them, the illumination threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0177] S142: Determine whether the detected dynamic range value is less than a dynamic range threshold.

[0178] The dynamic range threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0179] S143. As shown in FIG8 , when the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, or when the scene to be photographed is a low-light, high-dynamic-range scene, the number of level 1 processing frames corresponding to the first photo-taking time is determined to be a1 frames, and the A1 algorithm is determined to be the level 1 processing algorithm.

[0180] Wherein, a1 is all or part of the frames in the original image collected corresponding to the first shooting time.

[0181] S144. As shown in FIG8 , when the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, or when the scene to be photographed is a low-brightness, low-dynamic-range scene, the number of level 1 processing frames corresponding to the first photo-taking time is determined to be b1 frames, and the B1 algorithm is determined to be the level 1 processing algorithm.

[0182] Wherein, b1 is all or part of the frames in the original image collected corresponding to the first shooting time.

[0183] S145. As shown in FIG8 , when the illumination is greater than or equal to the illumination threshold and the dynamic range value is also greater than or equal to the dynamic range threshold, or when the scene to be captured is a scene with high brightness and high dynamic range, the number of level 1 processing frames corresponding to the first number of shots is determined to be c1 frames, and the C1 algorithm is determined to be the level 1 processing algorithm.

[0184] Wherein, c1 is all or part of the frames in the original image collected corresponding to the first shooting time.

[0185] S146. As shown in FIG8 , when the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, or when the scene to be photographed is a high-brightness, low-dynamic-range scene, the number of level 1 processing frames corresponding to the first photo-taking time is determined to be d1 frames, and the D1 algorithm is determined to be the level 1 processing algorithm.

[0186] Wherein, d1 is all or part of the frames in the original image collected corresponding to the first shooting number.

[0187] It should be understood that A1, B1, C1 and D1 are used to refer to the names of algorithms. A1 algorithm, B1 algorithm, C1 algorithm and D1 algorithm can indicate the same algorithm or different algorithms. They can be set and modified as needed. The embodiments of this application do not impose any restrictions on this.

[0188] At the same time, a1, b1, c1 and d1 are used to refer to the number of processed frames. a1, b1, c1 and d1 can indicate the same number of frames or different numbers of frames. The specific number of frames can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0189] Among them, the a1 frame, b1 frame, c1 frame, or d1 frame can be selected from all the frames of the original image in the acquisition order for processing, or the a1 frame, b1 frame, c1 frame, or d1 frame can be arbitrarily extracted from all the frames of the original image for processing, or the images captured by one or more cameras can be selected as the a1 frame, b1 frame, c1 frame, or d1 frame for processing in a manner of specifying cameras. Of course, the methods of selecting the a1 frame, b1 frame, c1 frame, or d1 frame can be the same or different, and can be specifically set and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0190] In addition, for scenes to be captured with a high dynamic range, the determined a1 frame or c1 frame may include at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image, or at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image may be additionally added. The specific settings and modifications may be made as needed, and the embodiments of the present application do not impose any restrictions on this.

[0191] It should be understood that when the processed original images include original images with long exposure, normal exposure and short exposure, the dynamic range and image details of the image can be adjusted so that the content presented by the obtained captured image is more realistic and the image quality is relatively better.

[0192] It should be noted that the above sequence of S141 to S146 is only one example. When performing environmental detection, it is also possible to first perform an illumination detection on the scene to be captured to determine whether the detected illumination is less than the illumination threshold, and then perform a dynamic range detection on the scene to be captured to determine whether the detected dynamic range value is less than the dynamic range threshold. Alternatively, the order of the two steps can be reversed. The specific execution process can be adjusted as needed. In addition, other steps can be added or some steps can be deleted on this basis. The embodiments of the present application do not impose any restrictions on this.

[0193] S15 , as shown in FIG3 , uses a level 1 processing algorithm to process the level 1 processing frames to obtain corresponding captured images.

[0194] In combination with the example in S14 above, when the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the determined A1 algorithm is used to process the a1 frame in the original image corresponding to the first shooting time to obtain the corresponding captured image.

[0195] When the illumination is less than the illumination threshold and the dynamic range is less than the dynamic range threshold, the determined B1 algorithm is used to process the b1 frame in the original image collected corresponding to the first shooting time to obtain the corresponding captured image.

[0196] When the illumination is greater than or equal to the illumination threshold and the dynamic range value is also greater than or equal to the dynamic range threshold, the determined C1 algorithm is used to process the C1 frame in the original image corresponding to the first shooting time to obtain the corresponding captured image.

[0197] When the illumination is greater than or equal to the illumination threshold and the dynamic range is less than the dynamic range threshold, the determined D1 algorithm is used to process the d1 frame in the original image collected corresponding to the first shooting time to obtain the corresponding captured image.

[0198] For example, when the first operation is the first photo-taking time, an original image is collected correspondingly, and a total of 9 frames of original images are collected. When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the determined A1 algorithm can be used to process 6 of the 9 frames (at this time a1=6) to obtain the corresponding captured images.

[0199] When the illumination is less than the illumination threshold and the dynamic range is less than the dynamic range threshold, the determined B1 algorithm can be used to process 5 frames (in this case b1=5) of the 9 frames to obtain corresponding captured images.

[0200] When the illumination is greater than or equal to the illumination threshold and the dynamic range value is also greater than or equal to the dynamic range threshold, the determined C1 algorithm can be used to process 4 frames (c1=4 at this time) of the 9 frames to obtain corresponding captured images.

[0201] When the illumination is greater than or equal to the illumination threshold and the dynamic range is less than the dynamic range threshold, the determined D1 algorithm can be used to process 3 frames (at this time d1=3) of the 9 frames to obtain corresponding captured images.

[0202] It should be understood that the present application can determine different scenes by performing environmental detection on the shooting scene, and then, based on the different scenes that have been segmented, select different algorithms, and select different numbers of frames from the corresponding collected original images for processing, so as to adaptively improve the quality and effect of the captured images corresponding to the first number of photos in each scene.

[0203] S16. When the first operation corresponds to the xth photo-taking time, determine the memory usage, where x is an integer greater than 1.

[0204] It should be understood that the memory usage refers to the amount of data stored in the memory. Since the camera will increase the memory usage each time it collects an original image or obtains a frame of captured image, the memory usage will increase relatively.

[0205] As the number of photos taken increases, the memory usage will increase. When the memory usage reaches a certain level, the related art may fail to respond to the photo command. To avoid this, this application will monitor the memory usage in real time. Different processing algorithms and / or different processing frame rates can be set for different memory usage levels to reduce the pressure of memory increments. The memory usage can be obtained by obtaining data from the electronic device.

[0206] S17. Determine the x-level processing frame number and the x-level processing algorithm corresponding to the number of photos taken based on the memory usage.

[0207] Here, "level x processing algorithm" refers to the algorithm used for the processing corresponding to the xth number of shots, and "level x processing frames" refers to the number of original image frames used for the processing corresponding to the xth number of shots. For example, when x = 2, the second shot corresponds to level 2 processing frames and a level 2 processing algorithm; when x = 3, the third shot corresponds to level 3 processing frames and a level 3 processing algorithm; the names of subsequent processing frames and processing algorithms are similar and are not repeated here.

[0208] Exemplarily, FIG9 shows a flow chart of determining the corresponding number of processing frames and processing algorithm according to memory usage.

[0209] As shown in FIG. 9 , the above-mentioned S17 may include the following S171 to S175 .

[0210] S171. For the xth number of photos, determine whether the memory usage is less than a first memory threshold.

[0211] S172: When the memory usage is less than the first memory threshold, determine the x-level processing algorithm as the first photographing algorithm, and determine the x-level processing frame number as the first frame number.

[0212] The first frame number refers to all or part of the frames in the original image collected corresponding to the xth number of photographs.

[0213] S173: When the memory usage is greater than or equal to the first memory threshold, continue to determine whether the memory usage is less than the second memory threshold.

[0214] S174: When the memory usage is less than the second memory threshold, determine the x-level processing algorithm as the second photographing algorithm, and determine the x-level processing frame number as the second frame number.

[0215] The second frame number refers to all or part of the frames in the original image collected corresponding to the x-th number of photographs.

[0216] S175: When the memory usage is greater than or equal to the second memory threshold, determine the x-level processing algorithm as the third photographing algorithm, and determine the x-level processing frame number as the third frame number.

[0217] The third frame number refers to all or part of the frames in the original image captured corresponding to the xth number of photographs.

[0218] It should be understood that the first memory threshold is less than the second memory threshold, and the second memory threshold is less than the total memory. Here, the sizes of the first memory threshold and the second memory threshold can be set and modified as needed, and the embodiment of the present application does not impose any restrictions on this.

[0219] It should be understood that the first photographing algorithm, the second photographing algorithm and the third photographing algorithm may be the same or different.

[0220] As an example, when the first photographing algorithm, the second photographing algorithm, and the third photographing algorithm are the same, the first frame number, the second frame number, and the third frame number gradually decrease.

[0221] As another example, when the first photographing algorithm, the second photographing algorithm and the third photographing algorithm are different, the processing time of the first photographing algorithm, the second photographing algorithm and the third photographing algorithm gradually decreases. At this time, the sizes of the first frame number, the second frame number and the third frame number can be the same, or gradually decrease.

[0222] Among them, in the above two cases, the reduction amount of the second frame number relative to the first frame number and the reduction amount of the third frame number relative to the second frame number can be the same or different, and can be set and modified as needed. The embodiment of the present application does not impose any restrictions on this.

[0223] In the second scenario described above, the number of processing steps included in the first, second, and third photo algorithms gradually decreases, or the complexity of the included processing steps gradually decreases, thereby gradually reducing the processing time. The processing time reduced by the second photo algorithm relative to the first photo algorithm and the processing time reduced by the third photo algorithm relative to the second photo algorithm can be the same or different, and can be specifically set and modified as needed, and this embodiment of the application does not impose any restrictions on this.

[0224] In the present application, the first frame number, the second frame number, or the third frame number can be selected from all frames of the original image captured by the corresponding number of shots in the acquisition order for processing, or the first frame number, the second frame number, or the third frame number can be arbitrarily extracted from all frames of the original image for processing, or the images captured by one or more cameras can be selected as the first frame number, the second frame number, or the third frame number for processing in a manner of specifying a camera. Of course, the method of selecting the first frame number, the second frame number, or the third frame number can be the same or different, and can be specifically set and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0225] Furthermore, the selected first frame number of original images may include at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image. The selected second frame number of original images may include at least one of a long-exposure original image, a normal-exposure original image, and a short-exposure original image. Similarly, the selected third frame number of original images may also include a long-exposure original image, a normal-exposure original image, and a short-exposure original image. Regardless of whether the first, second, and third frame numbers are the same, the number of long-exposure original images and short-exposure original images included in the first, second, and third frame numbers may gradually decrease. In this way, as memory usage increases, dynamic range maintenance is abandoned to ensure that the user's need for fast photography is met as much as possible.

[0226] It should be understood that in the above S171 to S175, only two memory thresholds are set, namely the first memory threshold and the second memory threshold. Through these two memory thresholds, the memory size can be divided into three intervals. When the memory occupancy meets one of the intervals, the algorithm and frame number corresponding to the interval are used for processing to obtain the corresponding captured image. In addition, a memory threshold can also be set to divide the memory size into two intervals; or, three, four or more memory thresholds can be set to subdivide the memory size into multiple intervals. The specific settings and modifications can be made as needed, and the embodiments of the present application do not impose any restrictions.

[0227] Here, the number of level x processing frames corresponding to the xth photo-taking number can be the same as the number of processing frames corresponding to the 1st photo-taking number, or can be less than the number of processing frames corresponding to the 1st photo-taking number; the level x processing algorithm corresponding to the xth photo-taking number can be the same as the level 1 processing algorithm corresponding to the 1st photo-taking number, or can have a shorter processing time than the level 1 processing algorithm corresponding to the 1st photo-taking number. Specific settings and modifications can be made as needed, and this embodiment of the application does not impose any restrictions on this.

[0228] When the first, second, and third photography algorithms are the same, the processing time of the first, second, and third photography algorithms can be the same as or slightly shorter than that of the level 1 processing algorithm. In this case, the first, second, and third frame numbers gradually decrease, and the first frame number can be the same as or slightly shorter than the level 1 processing frame number. Then, the second and third frame numbers are both shorter than the level 1 processing frame number.

[0229] When the processing times of the first, second, and third photography algorithms gradually decrease, the first photography algorithm can be the same as the level 1 processing algorithm corresponding to the first photography number, or the first photography algorithm can have a shorter processing time than the level 1 processing algorithm corresponding to the first photography number. In this case, when the first, second, and third frame numbers are the same, the first, second, and third frame numbers are the same as or less than the level 1 processing frame number. When the first, second, and third frame numbers gradually decrease, the first frame number is the same as or less than the level 1 processing frame number. In this case, the second and third frame numbers are less than the level 1 processing frame number.

[0230] S18 , processing the x-level processing frames using an x-level processing algorithm to obtain corresponding captured images.

[0231] FIG10 shows a schematic diagram of the growth of memory usage.

[0232] As shown in FIG10 , in conjunction with the example in S17 above, when x=2, the memory usage corresponding to the second number of photos is determined. If the memory usage corresponding to the second number of photos is less than the first memory threshold, the second-level processing algorithm is determined to be the first photo algorithm, and the second-level processing frame number is determined to be the first frame number.

[0233] When x=3, the memory usage corresponding to the third number of photos is determined. If the memory usage corresponding to the third number of photos is greater than the first memory threshold but less than the second memory threshold, the third-level processing algorithm is determined to be the second photo algorithm, and the number of frames processed by the third level is determined to be the second number of frames.

[0234] When x = 4, the memory usage corresponding to the fourth photo count is determined. If the memory usage corresponding to the fourth photo count is greater than the first memory threshold but less than the second memory threshold, the fourth-level processing algorithm is determined to be the second photo algorithm, and the number of frames processed in the fourth-level processing is determined to be the second number of frames. In this way, the processing for the fourth photo count is equivalent to the processing for the third photo count.

[0235] When x = 5, the memory usage corresponding to the fifth photo capture is determined. If the memory usage corresponding to the fifth photo capture is greater than the second memory threshold, the level 5 processing algorithm is determined to be the third photo capture algorithm, and the level 5 processing frame number is determined to be the third frame number. The following is analogous and will not be further elaborated here.

[0236] It should be understood that based on dividing the memory size into multiple interval ranges, when the memory usage is larger and the higher interval range is satisfied, the processing algorithm is the same, then the number of processed frames is gradually reduced, or, if the number of processed frames is the same, the processing time of the processing algorithm is reduced; or, both the processing time and the number of processed frames of the processing algorithm are reduced. At this time, the growth rate of the memory usage will slow down rapidly, reducing the pressure on memory increment in electronic devices.

[0237] In this way, compared with the existing technology in which each photo is processed with a single processing algorithm and processing frame number, or the memory usage increases at a fixed rate, the present application can timely adjust the processing algorithm and processing frame number according to the memory usage. When the memory is larger, the growth rate will decrease and tend to be flat, so that the electronic device can achieve more photos under the condition of the same memory size.

[0238] It should also be understood that FIG10 does not show the memory usage corresponding to the first photo-taking sequence. At this time, the original image captured during the first photo-taking sequence is processed according to the aforementioned environmental detection results. The method for processing based on the memory usage results is applicable to the second and subsequent photo-taking sequences. Of course, the same method can be used for processing the first photo-taking sequence as for the second and subsequent photo-taking sequences, and this embodiment of the present application is not limited to this.

[0239] For example, if x is 1, the memory usage corresponding to the first number of photos is determined. When the memory usage is less than the first memory threshold, the level 1 processing algorithm is determined to be the first photo algorithm, and the level 1 processing frame number is determined to be the first frame number; when the memory usage is greater than or equal to the first memory threshold but less than the second memory threshold, the level 1 processing algorithm is determined to be the second photo algorithm, and the level 1 processing frame number is determined to be the second frame number; when the memory usage is greater than the second memory threshold, the level 1 processing algorithm is determined to be the third photo algorithm, and the level 1 processing frame number is determined to be the third frame number; then, the level 1 processing frame number is processed using the determined level 1 processing algorithm to obtain the corresponding captured image.

[0240] S19. Save the captured image.

[0241] The captured images can be stored in the gallery or displayed.

[0242] In combination with the above process, in response to one click operation, one frame of captured image can be obtained, and in response to multiple click operations, multiple frames of captured images can be obtained; the specific number of captured images is related to the number of user operations, and the embodiment of the present application does not impose any restrictions on this.

[0243] An embodiment of the present application provides a photographing method, in which, in response to a first user operation on a first control, an electronic device uses a camera to capture an original image. When the first operation corresponds to the first photographing attempt, an environmental detection is performed on the scene to be photographed, and then, based on the environmental detection results, the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first photographing attempt are determined. When the first operation corresponds to the second photographing attempt and subsequent photographing attempts, a memory usage detection is first performed, and the determined memory usage is compared with a preset memory threshold. Then, different processing frames and processing algorithms are selected based on different memory usage.

[0244] When processing the first number of photos, since the scene is subdivided according to the environmental detection results, different algorithms and frame rates are selected according to different scenes, so that the quality and effect of the captured images corresponding to the first number of photos in each scene can be adaptively improved.

[0245] When processing the second and subsequent photos, the memory usage is subdivided and different algorithms and frame rates are selected according to different memory usage. When the memory usage is larger, the algorithm with shorter processing time and / or the number of processing frames is selected, thereby reducing the pressure of memory increment, improving processing efficiency, and meeting the needs of fast photo taking.

[0246] Because electronic devices gradually generate high levels of heat when taking photos and performing other tasks, their temperature gradually rises. If photos are taken rapidly and continuously, the device temperature may rise exponentially. Consequently, overheating can cause the device to malfunction and fail to respond to photo commands.

[0247] In this regard, the embodiment of the present application can also detect the device temperature of the electronic device before processing the collected data. In the event of overheating, the processing algorithm and the number of processing frames are switched to reduce the workload and reduce the incremental heat pressure.

[0248] Figure 11 shows a flow chart of another photographing method. As shown in Figure 11, based on Figure 3, the method may further include the following S20 and S21.

[0249] The contents of S11 to S19 can refer to the above description of FIG3 and will not be repeated here.

[0250] S20: After capturing an original image once in response to a first operation, when the first operation corresponds to a second photo-taking number or another subsequent photo-taking number, determining whether the device temperature is less than a temperature threshold.

[0251] The device temperature refers to the temperature inside the electronic device. Generally, as the electronic device operates, it generates a certain amount of heat, which causes the device temperature to rise. The device temperature can be collected by a temperature sensor inside the electronic device.

[0252] S21: When the device temperature is greater than or equal to the temperature threshold, a single-frame algorithm is used to process one frame of the original image collected this time to obtain a corresponding captured image, and then the captured image is saved.

[0253] The temperature threshold can be set and modified as needed, and the present application embodiment does not impose any restrictions on this. The device temperature can be obtained by obtaining data inside the electronic device.

[0254] It should be understood that the single-frame algorithm refers to an algorithm that processes one frame of original image. The single-frame algorithm is different from the level 1 processing algorithm. The processing time of the single-frame algorithm is shorter than that of the level 1 processing algorithm (A1 algorithm, B1 algorithm, C1 algorithm or D1 algorithm).

[0255] Here, the single-frame algorithm may include processing steps such as default pixel correction (DPC), RAW domain noise reduction (RAWNF), black level correction (BLC), lens shading correction (LSC), automatic white balance, demosaicing, color correction matrix (CCM), YUV domain noise reduction, tone mapping, gamma correction, color space conversion, and color enhancement (CE), with fast processing speed and short processing time.

[0256] Bad pixel correction: Bad pixels are white dots in the output image in a completely dark environment, or black dots in the output image in a bright environment. Generally, the three primary color channel signals should respond linearly to the ambient brightness. However, due to poor signal output from the image sensor, white or black dots may appear. To address this, we can automatically detect and repair bad pixels, or create a bad pixel list to repair bad pixels at fixed locations. A dot refers to a pixel.

[0257] Denoising refers to the process of reducing noise in an image. Common methods include mean filtering, Gaussian filtering, and bilateral filtering. RAW-domain denoising refers to the process of reducing noise in RAW-domain images. YUV-domain denoising refers to the process of reducing noise in YUV-domain images.

[0258] Black level correction is due to the presence of dark current in the image sensor, which causes the pixels to have a certain output voltage even when there is no light. In addition, pixels at different positions may correspond to different output voltages. Therefore, it is necessary to correct the output voltage corresponding to the pixels when there is no light (i.e., black).

[0259] Lens shading correction can solve the problem of shadows appearing around the lens due to uneven refraction of light by the lens.

[0260] Automatic white balance is designed to eliminate the impact of light sources on image sensor imaging, simulate the color constancy of human vision, and ensure that the white seen in any scene is truly white. Therefore, it is necessary to correct the color temperature and automatically adjust the white balance to the appropriate position.

[0261] Demosaicing. Since each pixel in a RAW image corresponds to the color information of only one channel, information from surrounding pixels can be used to estimate the other colors. For example, through linear interpolation, the color information of the other two channels missing for each pixel can be determined, thereby restoring all channel information for all pixels in the image. The demosaicing process is equivalent to converting the image from the RAW domain to the RGB domain. In this context, demosaicing can also be called color interpolation.

[0262] Color correction is necessary because the image captured by the camera may not match the expected color. Since the automatic white balance has already calibrated the white color, color correction can be used to calibrate other colors besides white.

[0263] Tone mapping involves mapping all pixels in an image using a spatially invariant mapping function. This means that when tone mapping applies a dynamic range shift to an image, each pixel uses the same transformation function, resulting in a one-to-one mapping. Tone mapping is also referred to as dynamic range compression (DRC).

[0264] Gamma correction refers to editing the gamma curve of an image in a nonlinear tone editing manner, detecting the dark and light parts of the image and increasing them proportionally, thereby improving the image contrast.

[0265] Color space conversion involves converting an image from the RGB domain to the YUV domain. This allows the image processed in the previous step to be converted from the RGB domain to the YUV domain, reducing the amount of data subsequently stored and transmitted, saving bandwidth.

[0266] Color enhancement is used to make the original unsaturated color information saturated and rich. Here, color enhancement can also be called color processing.

[0267] The above is only an example. The single-frame algorithm can also delete or add some other steps, and the level 1 processing algorithm (A1 algorithm, B1 algorithm, C1 algorithm or D1 algorithm) can include other steps on the basis of the single-frame algorithm. Of course, the level 1 processing algorithm (A1 algorithm, B1 algorithm, C1 algorithm or D1 algorithm) can also be different from the steps included in the single-frame algorithm. The embodiments of the present application do not impose any restrictions on this.

[0268] It should be understood that since the single-frame algorithm can be processed only in the image signal processor, the processing speed of the single-frame algorithm is relatively fast and the processing time is relatively short.

[0269] When the first photographing algorithm, the second photographing algorithm and the third photographing algorithm are the same, the processing time of the single-frame algorithm is shorter than that of the third photographing algorithm. Thus, the processing time of the second photographing algorithm is also shorter than that of the first photographing algorithm.

[0270] When the processing times of the first photographing algorithm, the second photographing algorithm and the third photographing algorithm gradually decrease, the single-frame algorithm may be the same as the third photographing algorithm, or the single-frame algorithm may have a relatively shorter processing time than the third photographing algorithm.

[0271] In conjunction with the above, when the device temperature is above the temperature threshold, the single-frame algorithm can be switched to process one frame of the original image according to the above steps, thereby minimizing the workload of the electronic device, slowing the growth of heat, and reducing the incremental heat pressure. This allows the electronic device to take more photos before it ceases to function, thus meeting the user's photography needs. When the device temperature is below the temperature threshold, the process described in S16 to S19 above can be continued to determine the memory usage of the electronic device. Based on the different memory usage, different algorithms and different frame rates can be switched to reduce the incremental memory pressure, thereby allowing more photos to be taken before the memory is full, thus meeting the user's photography needs.

[0272] It should be noted that the execution sequence shown in FIG11 is merely an example. Alternatively, after capturing an original image, when the first operation is determined to be the first photo-taking attempt, the device temperature of the electronic device may be detected. Then, when the device temperature of the electronic device is greater than or equal to a temperature threshold, one frame of the original image may be processed using a single-frame algorithm. When the device temperature of the electronic device is less than the temperature threshold, an environmental detection of the scene to be photographed may be performed. Alternatively, device temperature detection and environmental detection of the scene to be photographed may be performed simultaneously, and the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first photo-taking attempt may be determined based on the device temperature detection results and the environmental detection results. Alternatively, a temperature detection may be performed for each photo-taking attempt to determine whether the temperature is less than the temperature threshold before executing subsequent related steps.

[0273] It should be understood that the above method only sets one temperature threshold. The present application can also set multiple or three or more temperature thresholds, which can be set specifically as needed. For example, two temperature thresholds can be set, namely the first temperature threshold and the second temperature threshold. The first temperature threshold is less than the second temperature threshold. Through the two temperature thresholds, the temperature range can be divided into three intervals. When the device temperature meets one of the intervals, the algorithm and frame number corresponding to the interval are used for processing to obtain the corresponding captured image. On this basis, different memory thresholds can also be set for different temperature intervals to execute different judgment processes.

[0274] Figure 12 shows a schematic flow chart of another photographing method. When two temperature thresholds are set, namely a first temperature threshold and a second temperature threshold, this method, based on Figure 11, provides a new process including S22, which is described below. As shown in Figure 12, S22 may also include the following S221 to S231.

[0275] S221. After capturing an original image once in response to a first operation, when the first operation corresponds to the xth number of photographs (the second number of photographs or other subsequent number of photographs), determine whether the device temperature is less than a first temperature threshold.

[0276] S222: When the device temperature is lower than the first temperature threshold, continue to determine whether the memory usage is lower than the first memory threshold.

[0277] S223. According to S222, when the device temperature is less than the first temperature threshold and the memory usage is also less than the first memory threshold, determine the x-level processing algorithm as the first photographing algorithm, and determine the x-level processing frame number as the first frame number.

[0278] S224: Process the first frame number of the original image collected this time using the first photographing algorithm to obtain a corresponding photographed image.

[0279] S225. According to S222, when the device temperature is less than the first temperature threshold, but the memory usage is greater than or equal to the first memory threshold, the second photographing algorithm may be determined to be the x-level processing algorithm, and the x-level processing frame number may be determined to be the second frame number.

[0280] S226: Process the second number of frames in the original image collected this time using a second photographing algorithm to obtain a corresponding photographed image.

[0281] S227. In the above S221, when the device temperature is greater than or equal to the first temperature threshold, continue to determine whether the device temperature is less than the second temperature threshold.

[0282] S228: When the device temperature is greater than or equal to the first temperature threshold but less than the second temperature threshold, it may be determined whether the memory usage is less than the second memory threshold.

[0283] At this time, when the device temperature is less than the second temperature threshold and the memory usage is less than the second memory threshold, it means that the device temperature is not particularly high and the memory is not full. It is possible to continue processing with an algorithm with a longer processing time or with original images with a larger number of frames, such as executing the above S225 and S226.

[0284] S229. According to S228, when the device temperature is greater than or equal to the first temperature threshold and less than the second temperature threshold, but the memory usage is greater than or equal to the second memory threshold, the third photographing algorithm may be determined to be the x-level processing algorithm, and the x-level processing frame number may be determined to be the third frame number.

[0285] S230: Process the third frame number in the original image collected this time by using a third photographing algorithm to obtain a corresponding photographed image.

[0286] S231. According to S227, when it is determined that the device temperature is greater than or equal to the first temperature threshold and also greater than the second temperature threshold, it indicates that the device temperature is very high. Therefore, a single-frame algorithm can be used to process one frame of the original image collected this time to obtain a corresponding captured image.

[0287] It should be understood that when dividing the temperature range into multiple intervals and the memory size into multiple intervals, the temperature and memory conditions can be combined. Thus, when the temperature is low and the memory usage is small, an algorithm with a longer processing time is used to process multiple frames of the original image to ensure good image quality and effects. When the temperature is low and the memory usage is high, or when the temperature is high and the memory usage is low, an adaptive adjustment is made to use an algorithm with a relatively shorter processing time to process more frames of the original image to balance image quality with the temperature and memory usage. When the temperature is high and the memory usage is high, an algorithm with a shorter processing time is switched to process fewer frames of the original image to reduce the pressure of temperature increment and memory increment, ensuring normal photography with a slight reduction in image quality. In addition, when the temperature is extremely high, the simplest algorithm is directly switched to process the fewest frames of the original image regardless of the memory usage. Although this reduces image quality, it minimizes the temperature increment to ensure that the user's photography command is responded to and the user's photography needs are met.

[0288] It should be understood that the above is only an example. Multiple temperature thresholds and memory thresholds can be set to distinguish different situations. They can be set and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0289] The above method provided by the present application can make a judgment based on the device temperature and memory usage of the electronic device at the time of the current trigger when the user triggers the photo command, and then associate the determined processing algorithm and processing frame number with the number of photos. When the user trigger frequency is not high and the electronic device background processing is relatively timely, the processing frame number can be processed according to the determined processing algorithm to obtain the corresponding captured image. When the electronic device background is unable to process in a timely manner, the determined processing algorithm and processing frame number are associated with the number of photos, and then stored together with the collected original image and other data, and then wait for the electronic device to call and process it later.

[0290] However, when the user suddenly increases the frequency of triggering the photo command, the data obtained by the electronic device surges, causing serious data congestion. In this case, when the data corresponding to a certain number of photos is subsequently called and processed, since some time has passed between the processing time and the response time, the memory usage of the electronic device has changed greatly during this period. At this time, the originally determined processing algorithm will no longer be applicable and needs to be adaptively adjusted.

[0291] Alternatively, when the data corresponding to a certain number of photos is called for processing, the memory usage is close to the memory limit. If the processing is still performed according to the previously determined processing algorithm, it will inevitably cause an unresponsive situation. At this time, it is also necessary to adaptively adjust the processing algorithm in a timely manner to reduce the memory increment again.

[0292] In response to the above two situations, the embodiment of the present application provides a photo-taking method to switch the processing algorithm to adapt to the memory situation at the actual processing moment.

[0293] FIG13 is another photographing process provided by an embodiment of the present application. As shown in FIG13 , the method 30 may include the following steps 31 to S39.

[0294] S31. The electronic device displays a first interface, where the first interface includes a first control.

[0295] S32: When a first operation of the user on the first control is detected, the electronic device uses a camera to capture an original image in response to the first operation.

[0296] S33: When the first operation corresponds to the xth photo-taking time, determine the memory usage.

[0297] S34. Determine, based on the memory usage, the number of x-level processing frames and the x-level processing algorithm corresponding to the number of photos, where x is an integer greater than 1.

[0298] The processes of S31 to S34 are the same as the processes of S11, S12, S16 and S17 in FIG. 3 , and reference may be made to the above description, which will not be repeated here.

[0299] S35: Determine whether the frequency at which the user performs the first operation on the first control meets a preset frequency condition.

[0300] The frequency can indicate the speed at which the user performs the first operation, such as the speed at which the capture button is clicked. For example, the interval between two consecutive capture commands issued by the user is detected. If the interval is less than a second preset interval for 10 or 20 consecutive times, it can be determined that the frequency meets the preset frequency condition. The second preset interval can be set as needed, for example, to 100 meters or 200 milliseconds. The second preset interval is less than the first preset interval.

[0301] It should be understood that when the frequency does not meet the preset frequency condition, it indicates that the user has triggered the photo command relatively frequently, and the user is taking multiple photos normally. When the frequency does not meet the preset frequency condition, it indicates that the user has triggered the photo command relatively frequently, and the user desires to take multiple photos quickly. The preset frequency condition, i.e., the number of consecutive detection intervals between adjacent photo attempts, and the second preset interval duration, can be set and modified as needed, and this embodiment of the application does not impose any restrictions on this.

[0302] Optionally, in an embodiment of the present application, it is also possible to first determine whether the frequency of the user performing the first operation on the first control meets the preset frequency condition, and then, when the preset frequency condition is met, determine the memory usage, and determine the x-level processing frame number and x-level processing algorithm corresponding to the number of photos taken according to the method shown in Figure 3.

[0303] As an example, when the frequency meets the preset frequency condition, the memory usage is determined, and the x-level processing frame number and x-level processing algorithm corresponding to the number of photos are determined according to the method shown in Figure 3, the x-level processing frame number may not include long-exposure original images and short-exposure original images, and the x-level processing algorithm does not include a high dynamic range (HDR) algorithm.

[0304] It should be understood that when the frequency meets the preset frequency condition, it means that the user suddenly and quickly issues intensive photo-taking commands. At this time, the long-exposure original images and short-exposure original images included in the x-level processing frames can be removed, and the x-level processing algorithm can be switched to a non-HDR algorithm. In this way, by giving up the protection of the dynamic range and switching to a non-HDR algorithm that only processes the original images with normal exposure, the subsequent processing efficiency can be improved to meet the user's fast photo-taking needs.

[0305] S36: Determine whether the difference between the memory waterline and the memory usage is less than a preset difference.

[0306] Among them, the memory waterline refers to the maximum usage of memory. The size of the memory waterline can be obtained by obtaining parameters provided by the manufacturer, or it may need to be set and modified. The embodiment of the present application does not impose any restrictions on this.

[0307] It should be understood that when the difference between the memory waterline and the memory usage is greater than the preset difference, it indicates that the memory usage is still far from reaching the memory waterline, and there is still a lot of available storage space; when the difference between the memory waterline and the memory usage is less than the preset difference, it indicates that the memory usage is very close to reaching the memory waterline, and there is little available storage space left. The size of the preset difference can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0308] S37. When the frequency meets the preset frequency condition or the difference between the memory waterline and the memory usage is less than the preset difference, determine the number of photos waiting in the background, and use the x-level processing algorithm corresponding to each waiting number of photos as the initial processing algorithm.

[0309] Determining the number of photos waiting in the background refers to determining which photo counts are waiting in the background. The number of photos waiting in the background may include one, two, or more. For example, there may be two, namely, the 5th level photo count and the 6th level photo count. Then, the 5th level processing algorithm corresponding to the 5th photo count is used as the initial processing algorithm, and the 6th level processing algorithm corresponding to the 6th photo count is used as the initial processing algorithm. Of course, there may be no photo counts waiting in the background. If there are no photo counts waiting, no subsequent steps are required.

[0310] It should be understood that when the frequency meets the preset frequency condition, it means that the user has increased the frequency of triggering the photo command; when the difference between the memory waterline and the memory usage is less than the preset difference, it means that the memory is approaching the upper limit. In these two cases, it is necessary to switch the processing algorithm, and when switching, it is necessary to first clarify the processing algorithm that needs to be replaced.

[0311] The above are just two example conditions. You can also set other conditions to start the replacement process, or you can cancel the above conditions and replace the secondary processing algorithm when processing the number of photos waiting in the background. The specific settings and adjustments can be made as needed, and this application does not impose any restrictions on this.

[0312] S38. The x-level processing algorithm corresponding to the most recently determined number of photographing times is used as the target processing algorithm, and the initial processing algorithm corresponding to the waiting number of photographing times is replaced by the target processing algorithm.

[0313] It should be understood that the most recent number of photos refers to the number of photos currently recorded in response to the user's first operation. The process of determining the processing algorithm and processing frame number corresponding to the most recent number of photos can be determined using the method described in S17 above, which will not be repeated here.

[0314] Since the processing algorithm and number of processing frames corresponding to the number of photos waiting in the background are determined based on the previous memory usage, the previous memory usage is different from the current memory usage. Therefore, based on the currently determined memory usage, all processing algorithms corresponding to the number of photos waiting in the background can be replaced with the currently determined processing algorithms to adapt to the current memory situation.

[0315] Optionally, before replacing the algorithm, the processing algorithm corresponding to the most recently determined number of photos can be compared with the initial processing algorithm corresponding to the number of photos to be waited for. If they are the same, no replacement is required; if they are different, then replacement is performed. Thus, by comparing the processing algorithms and only replacing those that differ, ineffective replacements can be avoided, reducing workload.

[0316] Optionally, the number of processing frames corresponding to the number of photos to be waited for can be used as the initial processing frame number, and the number of processing frames corresponding to the most recently determined number of photos to be taken can be used as the target processing frame number. Then, the target processing frame number can be used to replace the initial processing frame number corresponding to the number of photos to be waited for.

[0317] It should be understood that when the currently determined memory usage is taken into consideration, the number of processing frames corresponding to the number of photos waiting in the background can be completely replaced with the target number of processing frames.

[0318] Before replacing, the number of processed frames corresponding to the most recently determined number of shots can be compared with the number of processed frames corresponding to the waiting number of shots. If they are the same, no replacement is required; if they are different, replacement is performed. Thus, by comparing the number of processed frames, only replacement is performed when the number of frames is different, which can avoid invalid replacement and reduce workload.

[0319] As shown in Figure 3, when the number of processing frames corresponding to the number of photos is first determined, the number of processed frames remains the same, or, as the memory usage increases, the number of processed frames gradually decreases. Therefore, when the number of processing frames corresponding to the number of photos is determined for the second time, if a replacement is required, the situation where the number of processed frames remains the same must be eliminated, and the previously larger number of processed frames will be replaced with the most recent smaller number of processed frames. This is equivalent to reducing the number of processing frames corresponding to the number of photos being waited for each time. The smaller the number of processed frames after replacement, the higher the efficiency of subsequent determination of the captured images and the smaller the increase in memory usage.

[0320] It should be noted that when replacing the initial processing frame count with the target processing frame count, only the numerical value of the processing frame count is changed; the corresponding original image is not replaced. For example, if the waiting number of photos is the 10th, and in response to the first operation, 9 original image frames are captured, 5 of these original images are selected as the processing frames when determining the processing frame count. "5 frames" is the initial processing frame count. If the target processing frame count is subsequently determined to be 1 frame, the initial processing frame count of "5 frames" can be replaced with "1 frame", and the original image corresponding to this 1 frame can be the 1 frame selected from the original 5 original images.

[0321] It should also be understood that the above replacement method, once initiated, can be executed multiple times in a loop. In other words, the processing algorithm and / or number of processing frames corresponding to the number of pending photos can be replaced each time a new photo command is subsequently responded to. Alternatively, the processing algorithm and / or number of processing frames corresponding to a certain number of pending photos can be replaced multiple times before being used to process and obtain the corresponding captured image.

[0322] Therefore, the processing algorithm corresponding to the number of photos waiting in the background is the processing algorithm determined according to the memory usage in response to the user's first operation on the first control, or the target processing algorithm after the last replacement during the waiting process.

[0323] For example, when the most recently determined number of photographs is the 20th, the processing algorithm corresponding to the 20th photograph is used as the target processing algorithm, and the corresponding number of processing frames is used as the target number of frames; then, the processing algorithms corresponding to the multiple waiting number of photographs can be used as the initial processing algorithm, and the corresponding number of processing frames can be used as the initial processing frame number, and then the initial processing algorithm is replaced with the target processing algorithm, and the initial processing frame number is replaced with the target processing frame number.

[0324] In response to another first operation by the user, the number of photos corresponding to the most recent one is updated to the 21st photo number. The processing algorithm corresponding to the 21st photo number is used as the target processing algorithm, and the corresponding number of processing frames is used as the target number of processing frames. Then, the processing algorithm corresponding to the waiting multiple photo numbers (the target processing algorithm corresponding to the 20th photo number) can be used as the initial processing algorithm, and the corresponding number of processing frames (the target number of processing frames corresponding to the 20th photo number) can be used as the initial processing algorithm. The initial processing algorithm is then replaced with the target processing algorithm for the 21st photo number, and the initial processing frame number is replaced with the target number of processing frames for the 21st photo number. If the user continues to perform the first operation, the subsequent replacement process is analogous and will not be repeated here.

[0325] S39: When any one of the photographing times waiting in the background is called for processing, the x-level processing frames in the original image collected by the photographing time are processed using the corresponding target processing algorithm to obtain the corresponding photographed image.

[0326] In addition, if the frequency does not meet the preset frequency condition, or the difference is greater than or equal to the preset difference, the above steps are not required, and processing continues according to the previously determined processing algorithm. For example, as shown in S18, the previously determined x-level processing algorithm is used to process x-level processing frames to obtain the corresponding captured image. Alternatively, if the above-mentioned frequency or difference conditions are met, but there are no waiting times for the background to take photos, the above steps are not required.

[0327] S40: Save the captured image.

[0328] The captured images can be stored in the gallery or displayed.

[0329] In the photographing method provided in an embodiment of the present application, in response to a user's first operation on a first control, the electronic device uses a camera to capture a raw image. When the frequency of the first operation on the first control meets a preset frequency condition or the difference between the memory watermark and the memory usage is less than a preset difference, the number of photos waiting in the background is determined and the processing algorithm corresponding to each waiting photo number is used as the initial processing algorithm for each waiting photo number. Then, the processing algorithm corresponding to the most recently determined photo number is used as the target processing algorithm, and the initial processing algorithm corresponding to the waiting photo number is replaced with the target processing algorithm. In this way, when the waiting photo number is subsequently called for processing, the replaced target processing algorithm can be used to process the initial processing frames of the raw image captured for that photo number, thereby obtaining the corresponding captured image. Since the processing algorithm corresponding to the most recently determined photo number is determined based on the current memory status, the processing algorithm for the waiting photo number is adjusted accordingly, thereby controlling the degree of memory accumulation, reducing memory usage, and achieving continuous and rapid photography.

[0330] Furthermore, the number of processing frames can be replaced. When the number of processing frames decreases as the number of photos taken increases, the number of processing frames corresponding to the most recently determined number of photos is used as the target number of processing frames, and the initial number of processing frames corresponding to the number of photos to be taken is replaced with the target number of processing frames. After the replacement, the number of processing frames corresponding to the number of photos to be taken is reduced. Thus, when the number of photos to be taken is called for processing, the reduced target number of processing frames can be used to determine the captured image. Since the number of processing frames corresponding to the most recently determined number of photos is determined based on the current memory situation, the number of processing frames for the number of photos to be taken is adjusted accordingly, which can also control the degree of memory accumulation and reduce memory usage.

[0331] On this basis, if the number of photos waiting in the background takes a long time to be processed, the corresponding processing algorithm and / or processing frame number can be replaced multiple times as the most recently determined number of photos changes, using the latest target processing algorithm to replace the previous target processing algorithm, and / or using the latest target processing frame number to replace the previous target processing frame number. In this way, when the number of photos waiting in the background is actually processed, the latest target processing algorithm can be used for processing. In this way, through multiple replacements, the latest data can be used as a benchmark, memory usage can be reduced as much as possible, and more continuous and rapid photos can be taken.

[0332] Compared with the previous method of only determining the processing algorithm and processing frame number once, this method performs secondary replacement, or even multiple replacements, and can update and optimize the processing algorithm and / or processing frame number in real time, adapting to the current memory situation more flexibly and ensuring that photo taking can proceed normally.

[0333] The above generally introduces the process of the photographing method provided by the present application, which will be further illustrated using an embodiment below.

[0334] Example 1

[0335] FIG14 is a table provided in an embodiment of the present application, which is used to illustrate the processing algorithm and number of processing frames corresponding to each number of photographs.

[0336] As shown in FIG4 , when a user opens a camera application, the phone displays a shooting interface including a shooting button. When the user takes a photo, the user clicks the shooting button multiple times with a finger.

[0337] When the mobile phone detects the user's first click operation, in response to the first click operation, the mobile phone uses the camera to capture an original image, for example, 9 frames of original images in the RAW domain are captured. The number of photos taken this time can be recorded as the first number of photos taken.

[0338] Since this click operation corresponds to the first photo capture, the phone can perform environmental detection for the scene to be captured, such as illumination detection and dynamic range detection. At this time, the phone can also determine whether the detected illumination is less than the illumination threshold and whether the detected dynamic range value is less than the dynamic range threshold based on the environmental detection results.

[0339] When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, or when the scene to be captured is low-light and low-dynamic-range, as shown in Figure 14, the Level 1 processing algorithm can be determined to be the A1 algorithm, which is, for example, the first night scene algorithm. Furthermore, the Level 1 processing frame number corresponding to the first capture can be determined to be, for example, 6 frames. These 6 frames are the 6 original image frames selected from the 9 original image frames collected corresponding to the first capture.

[0340] When the mobile phone detects the user's second click operation, in response to the click operation, the mobile phone uses the camera to capture an original image, or 9 frames of original images, and the number of photos taken this time can be recorded as the second number of photos taken.

[0341] Since this photo count corresponds to the second photo count, and x is greater than 1, the phone continues to determine the memory usage. Since the memory usage at this point is less than the first memory threshold, it can be determined that the Level 2 processing algorithm corresponding to the second photo count is the first photo algorithm, which can also be, for example, the first night scene algorithm. At the same time, it can be determined that the Level 2 processing frame count corresponding to the second photo count is 6 frames, which is the first frame count. These 6 frames are the 6 original image frames selected from the 9 original image frames collected corresponding to the second photo count.

[0342] When the mobile phone detects the user's third click operation, in response to the click operation, the mobile phone uses the camera to capture an original image, or 9 frames of original images, and the number of photos taken this time can be recorded as the third photo taking time.

[0343] Since this photo count corresponds to the third photo count, and x is greater than 1, the phone determines the memory usage at that time. This memory usage is greater than the first memory threshold but less than the second memory threshold. Therefore, it can be determined that the Level 3 processing algorithm corresponding to the third photo count is the second photo algorithm. The preset second photo algorithm is the same as the first photo algorithm, that is, the second photo algorithm is the first night scene algorithm. Furthermore, it can be determined that the Level 3 processing frame count corresponding to the third photo count is 5 frames, which is the second frame count, and is less than the first frame count. These 5 frames are the 5 original image frames selected from the 9 original image frames collected corresponding to the third photo count.

[0344] When the mobile phone detects the user's fourth click operation, in response to the click operation, the mobile phone uses the camera to capture an original image, or 9 frames of original images, and the number of photos taken this time can be recorded as the fourth photo taking time.

[0345] Since this photo count corresponds to the fourth photo count, and x is greater than 1, the phone determines the memory usage at that time. Since the memory usage at that time is greater than the second memory threshold, it can be determined that the fourth-level processing algorithm corresponding to the fourth photo count is the third photo algorithm. The preset third photo algorithm is the same as the second and first photo algorithms, that is, the third photo algorithm is the first night scene algorithm. At the same time, it can be determined that the fourth-level processing frame count corresponding to the fourth photo algorithm is 4 frames, which is the third frame count, and the third frame count is less than the second frame count. These 4 frames are the 4 original image frames selected from the 9 original image frames collected corresponding to the fourth photo count.

[0346] The processing of subsequent photo taking times is similar and will not be described in detail here.

[0347] For example, the first night scene algorithm may include all steps included in the single-frame algorithm, as well as a night scene algorithm module generated based on the Unet network model. The night scene algorithm module may fuse multiple frames of original images in the RAW domain into a single frame of image in the RAW domain. Of course, the night scene algorithm module may also be generated based on other models, and the first night scene algorithm may further include other modules or steps, which are not limited in any way by the present embodiment.

[0348] In addition, the reduction amount of the number of processing frames corresponding to each number of photos can be the same or different, and can be set and modified according to needs. The embodiments of the present application do not impose any restrictions on this.

[0349] In the above embodiment, when the scene to be captured is a low-light, low-dynamic-range scene, it means that the ambient light in the scene to be captured is relatively dim. If a general photography algorithm is used for processing, the resulting captured image will be unclear, which in turn causes the user to have difficulty seeing the details in the captured image. In this regard, in the embodiment of the present application, the first night scene algorithm can be selected for processing to increase the brightness of the image during processing, so that the subsequently acquired captured image can present more details.

[0350] On this basis, based on dividing the memory size into multiple intervals, during multiple photo-taking processes, as the memory usage increases, the number of processing frames determined for the number of photos taken in different memory ranges gradually decreases. At this time, although the selected processing algorithm is the same, during subsequent processing, due to the decrease in the number of processing frames, the processing time of the processing algorithm will be correspondingly shortened, and the growth rate of memory usage will tend to be flat. Therefore, in the embodiment of the present application, under multiple photo-taking times, as the number of photos increases, based on the same algorithm, the number of processing frames corresponding to each photo-taking time is reduced, thereby reducing the processing time of the processing algorithm, improving processing efficiency, and at the same time slowing down the growth rate of memory usage and reducing the pressure of memory increment.

[0351] Example 2

[0352] FIG15 is another table provided in an embodiment of the present application, which is used to illustrate the processing algorithm and number of processing frames corresponding to each number of photographs.

[0353] As shown in FIG4 , when a user opens a camera application, the phone displays a shooting interface including a shooting button. When the user takes a photo, the user clicks the shooting button multiple times with a finger.

[0354] When the mobile phone detects the user's first click operation, in response to the click operation, the mobile phone uses the camera to capture a raw image, for example, capturing 6 frames of raw images in the RAW domain. The number of photos taken this time can be recorded as the first photo taking time.

[0355] Since this click operation corresponds to the first photo capture, the phone can perform environmental detection for the scene to be captured, such as illumination detection and dynamic range detection. At this time, the phone can also determine whether the detected illumination is less than the illumination threshold and whether the detected dynamic range value is less than the dynamic range threshold based on the environmental detection results.

[0356] When the illumination is greater than the illumination threshold and the dynamic range value is less than the dynamic range threshold, or when the scene to be captured is a high-brightness scene with a low dynamic range, as shown in Figure 15, the Level 1 processing algorithm can be determined to be the D1 algorithm, which is the first multi-frame noise reduction (MFNR) algorithm. Furthermore, the Level 1 processing frame number corresponding to the first shot can be determined to be 6 frames, which refers to all 6 frames of the original image captured corresponding to the first shot.

[0357] When the mobile phone detects the user's second click operation, in response to the click operation, the mobile phone uses the camera to capture an original image, or 6 frames of original images, and the number of photos taken this time can be recorded as the second number of photos taken.

[0358] Since this photo count corresponds to the second photo count, and x is greater than 1, the phone can then test the device temperature to determine whether it is below the temperature threshold. If so, it continues to determine the memory usage. Since the memory usage at this point is below the first memory threshold, it can be determined that the second-level processing algorithm corresponding to the second photo count is the first photo algorithm, such as the second MFNR algorithm. At the same time, it can be determined that the number of second-level processing frames corresponding to the second photo count is 6, which is the first frame number. These 6 frames refer to all 6 frames of original images collected corresponding to the second photo count.

[0359] When the mobile phone detects the user's third click operation, in response to the click operation, the mobile phone uses the camera to capture an original image, or 6 frames of original images, and the number of photos taken this time can be recorded as the third photo taking time.

[0360] Since this photo count corresponds to the third photo count, and x is greater than 1, the phone can then detect the device temperature to determine whether it is less than the temperature threshold. If it is, it continues to determine the memory usage at that time. The memory usage at this time is greater than the first memory threshold but less than the second memory threshold. Therefore, it can be determined that the third-level processing algorithm corresponding to the third photo count is the second photo algorithm. The preset second photo algorithm is different from the first photo algorithm, and the processing time of the second photo algorithm is shorter than the processing time of the first photo algorithm. The second photo algorithm is, for example, the third MFNR algorithm. At the same time, it can be determined that the number of third-level processing frames corresponding to the third photo count is 6 frames. 6 frames is the second frame number, which is the same as the first frame number. The 6 frames refer to all 6 frames of original images collected corresponding to the third photo count.

[0361] When the mobile phone detects the user's fourth click operation, in response to the click operation, the mobile phone uses the camera to capture an original image, or 6 frames of original images, and the number of photos taken this time can be recorded as the fourth photo taking time.

[0362] Since this photo count corresponds to the fourth photo count, and x is greater than 1, the device temperature can be detected to determine whether the device temperature is less than the temperature threshold. If it is less than the temperature threshold, the phone continues to determine the memory usage at this time. The memory usage at this time is greater than the second memory threshold. Therefore, it can be determined that the 4-level processing algorithm corresponding to the fourth photo count is the third photo algorithm. The preset third photo algorithm is different from the second photo algorithm and the first photo algorithm, and the processing time of the third photo algorithm is shorter than the processing time of the second photo algorithm. The third photo algorithm is, for example, the fourth MFNR algorithm. At the same time, it can be determined that the 4-level processing frame number corresponding to the fourth photo count is 6 frames. 6 frames is the third frame number, and the third frame number is the same as the second frame number and the first frame number. The 6 frames refer to all 6 frames of original images collected corresponding to the fourth photo count.

[0363] When the mobile phone detects the user's fifth click operation, in response to the click operation, the mobile phone uses the camera to capture an original image, or 6 frames of original images, and the number of photos taken this time can be recorded as the fifth number of photos taken.

[0364] Since this photo capture corresponds to the fifth photo capture, and x is greater than 1, the device temperature can be tested to determine whether it is less than the temperature threshold. If it is, there is no need to determine the memory usage. The processing algorithm at this time is directly determined to be a single-frame algorithm, and the number of processed frames is 1 frame. This 1 frame refers to one original image among the 6 original images captured corresponding to the fifth photo capture.

[0365] The processing of subsequent photo taking times is similar and will not be described in detail here.

[0366] For example, the first MFNR algorithm may be a process of first aligning multiple frames in the original image and then processing the aligned original image using a wavelet fusion algorithm; the second MFNR algorithm may be an optical flow algorithm, and the third MFNR algorithm and the fourth MFNR algorithm may be other algorithms. The embodiments of the present application have no restrictions on this.

[0367] In the above embodiment, when the scene to be captured is a high-brightness, low-dynamic-range scene, it means that the ambient light in the scene to be captured is strong but the dynamic range is small. In this case, if a general photography algorithm is used for processing, the resulting captured image will have high noise and a small dynamic range. In this regard, in the embodiment of the present application, the MFNR algorithm can be selected for processing to reduce the noise in the captured image and improve the dynamic range of the captured image.

[0368] On this basis, based on dividing the memory size into multiple intervals, during multiple photo-taking processes, as the memory usage increases, the processing time of the processing algorithm determined by the number of photos taken in different memory ranges gradually decreases. At this time, although the number of processing frames selected is the same, during subsequent processing, the processing time of the processing algorithm gradually becomes shorter, making the growth rate of memory usage tend to be flat. Therefore, in the embodiment of the present application, under multiple photo-taking times, as the number of photos increases, based on the same number of processing frames, the processing time or performance of the processing algorithm corresponding to each photo-taking time can be reduced, thereby improving processing efficiency, while slowing down the growth rate of memory usage and reducing the pressure of memory increment.

[0369] In addition, when the device temperature is higher than the temperature threshold, it can directly switch to a single-frame algorithm to process one frame of the original image to reduce the workload, slow down the growth of heat, and reduce the incremental heat pressure, so that the mobile phone can take more photos to meet the user's photography needs.

[0370] Example 3

[0371] Figure 16 is another table provided in an embodiment of the present application, which is used to illustrate the processing algorithm and processing frame number corresponding to each number of photo shooting. Figure 17 is a schematic diagram of the processing process corresponding to Figure 16.

[0372] As shown in FIG5 , when a user opens the camera application to record a video, the phone displays a recording interface including a snapshot control. The user clicks the snapshot control multiple times with a finger while recording the video.

[0373] When the phone detects the user's first click operation, in response to this click operation, the phone uses the camera to capture a raw image. For example, nine frames of raw images are captured, including six frames of normal exposure raw images, one frame of long exposure raw image, and two frames of short exposure raw images. This number of photos can be recorded as the first photo capture number.

[0374] Since this click corresponds to the first photo capture, the phone performs environmental detection for the scene to be captured, such as illumination detection and dynamic range detection. Based on the detection results, it also determines whether the detected illumination is less than the illumination threshold and whether the detected dynamic range value is less than the dynamic range threshold.

[0375] When the illumination is greater than the illumination threshold and the dynamic range value is greater than the dynamic range threshold, or when the scene to be captured is a bright scene with a high dynamic range, as shown in Figure 16(a), the Level 1 processing algorithm can be determined to be the C1 algorithm, which is the first HDR algorithm. Furthermore, the Level 1 processing frame number corresponding to the first shot can be determined to be 9 frames, which refers to all 9 frames of original images captured corresponding to the first shot.

[0376] The first HDR algorithm may include an HDR algorithm module, and the electronic device may implement the processing process of the first HDR algorithm through the HDR algorithm module.

[0377] For example, the first HDR algorithm module is generated based on multi-exposure fusion processing and a tone mapping model. It can fuse multiple frames of original images with different exposure values ​​or exposure levels into a single image in the RAW domain and perform color mapping and transformation processing on the image. The tone mapping model can be any one of a Unet network model, a Resnet network model, and an Hdrnet network model. Of course, the tone mapping model can also be other models, and the embodiments of the present application do not impose any restrictions on this.

[0378] At this time, as shown in Figure 17, the background starts to process the level 1 processing frames using the first HDR algorithm determined for the first number of photos, that is, all 9 frames of original images collected corresponding to the first number of photos are processed. This process can be called the first photo processing.

[0379] When the phone detects the user's second click, in response to this click, the phone uses the camera to capture a raw image, again nine frames of raw images, including seven frames of normal exposure, one frame of long exposure, and one frame of short exposure. This number of photos can be recorded as the second number of photos.

[0380] Since this photo count corresponds to the second photo count, and x is greater than 1, the phone can then determine whether the frequency of the user clicking the snapshot control meets a preset frequency condition. If the frequency does not meet the preset frequency condition, the phone continues to determine the memory usage. Since the memory usage at this time is less than the first memory threshold, the second-level processing algorithm corresponding to the second photo count can be determined to be the first photo algorithm. The first photo algorithm is, for example, a second HDR algorithm that is different from the first HDR algorithm. At the same time, the number of second-level processing frames corresponding to the second photo count can be determined to be 8 frames, which is the first number of frames. These 8 frames refer to 5 normally exposed original images, 1 long-exposure original image, and 2 short-exposure original images from the 9 original images collected corresponding to the second photo count. Alternatively, these 8 frames refer to 5 normally exposed original images from the 9 original images collected corresponding to the second photo count, as well as 1 long-exposure original image and 2 short-exposure original images collected from the first photo count, which are reused. Alternatively, the 8 frames refer to 1 frame of long-exposure original image and 2 frames of short-exposure original images captured by the first photo-taking time, as well as 5 frames of normal-exposure original images that are captured close to the long-exposure original image or the short-exposure original image.

[0381] It should be noted that when the user clicks the photo button very quickly, that is, when the photo command is triggered frequently, the content differences between the images are relatively small. Therefore, it is possible to partially or fully reuse the original image captured in the first photo shot. This way, the electronic device does not need to re-frame, reducing workload and memory usage.

[0382] The subsequent process of determining the number of photos taken from the third to the fifth time is similar to the process of determining the number of photos taken from the second time, and will not be repeated here.

[0383] When the phone detects the user's sixth click, in response to this click, the phone uses the camera to capture a raw image, again nine frames of raw images. These nine frames include six frames of normally exposed raw images, one frame of long-exposure raw images, and two frames of short-exposure raw images. This capture can be recorded as the sixth capture.

[0384] Since this photo count corresponds to the sixth photo count and x is greater than 1, the phone can then determine whether the frequency of the user's clicks on the snapshot control meets the preset frequency condition. If the frequency does not meet the preset frequency condition, the phone continues to determine the memory usage at that time. The memory usage at this time is greater than the first memory threshold but less than the second memory threshold. Therefore, it can be determined that the 6-level processing algorithm corresponding to the sixth photo count is the second photo algorithm. The preset second photo algorithm is different from the first photo algorithm, and the processing time of the second photo algorithm is shorter than the processing time of the first photo algorithm. The second photo algorithm is, for example, a third HDR algorithm, which is different from the first and second HDR algorithms. At the same time, it can be determined that the number of 6-level processing frames corresponding to the sixth photo count is 7 frames, which is the second frame number, which is smaller than the first frame number. The 7 frames refer to the 4 normal exposure original images, 1 long exposure original image, and 2 short exposure original images among the 9 original images captured corresponding to the sixth photo count. Alternatively, the seven frames refer to the four normally exposed original images among the nine original images acquired corresponding to the sixth shooting time, and the one long-exposure original image and two short-exposure original images acquired by the first shooting time. Alternatively, the seven frames refer to the one long-exposure original image and two short-exposure original images acquired by the first shooting time, and the five normally exposed original images acquired close to the long-exposure original image or the short-exposure original image.

[0385] As shown in Figure 17, when the user performs the second to sixth click operations, since the first photo processing is still in progress, only all the original images corresponding to the second to sixth photos, as well as the determined processing algorithms at each level, the number of processing frames at each level and other related data can be stored, and the system enters the background waiting state.

[0386] Then, after the first shot processing is completed, the background waits for the second to sixth shots, processing the data related to the sixth shot first, according to the principle of recent first. The background begins to process the seven frames of original images using the determined third HDR algorithm for the sixth shot. That is, it processes the four frames of original images with normal exposure, one frame of original image with long exposure, and two frames of original image with short exposure corresponding to the sixth shot. This process can be called the second shot processing.

[0387] Next, while processing the data related to the sixth photo capture, when the phone detects the user's seventh click, in response to this click, the phone uses the camera to capture a single original image, again nine frames of original images. These nine frames include six frames of normal exposure, one frame of long exposure, and two frames of short exposure. This photo capture can be recorded as the seventh photo capture.

[0388] Since this photo count corresponds to the 7th photo count, and x is greater than 1, the phone can then determine whether the frequency of the user clicking the snapshot control meets the preset frequency condition. If the frequency meets the preset frequency condition, it is necessary to abandon the dynamic range protection and switch to a non-HDR algorithm that only processes normally exposed original images. For example, the memory usage at this time can be determined. If the memory usage at this time is greater than the first memory threshold but less than the second memory threshold, the 7-level processing algorithm corresponding to the 7th photo count can be switched from the third HDR algorithm to the third MFNR algorithm. At the same time, it can be determined that the 7-level processing frames corresponding to the 7th photo count are 4 frames of normally exposed original images, and other long-exposure original images and short-exposure original images are removed.

[0389] In addition, since the frequency meets the preset frequency condition, it can also be determined that the number of photos waiting in the background at this time is the second to fifth photo taking times. Based on this, the processing algorithms corresponding to the second to fifth photo taking times can be stored as the initial processing algorithm, and the 7-level processing algorithm determined for the seventh photo taking time can be used as the target processing algorithm. Then, the target processing algorithm is used to replace the initial processing algorithm corresponding to the waiting number of photos.

[0390] As shown in Figure 16(b), the 7-level processing algorithm determined for the 7th shot is the third MFNR algorithm. Therefore, the third MFNR algorithm is now the target processing algorithm. In conjunction with Figure 16(a), the second HDR algorithm originally associated with the 2nd shot was the initial processing algorithm. After replacing the initial processing algorithm with the target processing algorithm, the 2-level processing algorithm associated with the 2nd shot is changed to the third MFNR algorithm. The replacement process for shots 3 through 5 is similar and will not be repeated here.

[0391] As shown in Figure 17, when the user performs the 7th click operation, since the 2nd photo processing is still in progress, all the original images corresponding to the 7th photo number, as well as the determined 7-level processing algorithm, 7-level processing frame number and other related data can only be stored, and the system enters the background waiting state.

[0392] When the phone detects the user's eighth click operation, in response to this click operation, the phone uses the camera to capture an original image, again 9 frames of original images, including 6 frames of original images with normal exposure, 1 frame of original image with long exposure, and 2 frames of original image with short exposure. This number of photos can be recorded as the 8th number of photos.

[0393] Since this photo count corresponds to the eighth photo count and x is greater than 1, the phone can then determine whether the frequency of the user's clicks on the snapshot control meets the preset frequency condition. If the frequency meets the preset frequency condition, it is necessary to abandon dynamic range protection and switch to a non-HDR algorithm that only processes normally exposed original images. For example, the memory usage at this time can be determined. If the memory usage at this time is greater than the second memory threshold, the 8-level processing algorithm corresponding to the eighth photo count can be switched from the third HDR algorithm to the fourth MFNR algorithm, which is different from the third MFNR algorithm. At the same time, it can be determined that the number of 8-level processing frames corresponding to the eighth photo count is 3 frames of normally exposed original images, and other long-exposure original images and short-exposure original images are removed.

[0394] In addition, since the frequency meets the preset frequency condition, it can also be determined that the number of photos waiting in the background at this time is the 2nd to 5th photo taking times, and the 7th photo taking times. Based on this, the processing algorithm after the last replacement of the 2nd to 5th photo taking times can be used as the initial processing algorithm, and the 7-level processing algorithm determined for the 7th photo taking times can be used as the initial processing algorithm, and the 8-level processing algorithm determined for the 8th photo taking times can be used as the target processing algorithm. Then, the target processing algorithm is used to replace the initial processing algorithm corresponding to the waiting photo taking times.

[0395] As shown in Figure 16 (c), the 8-level processing algorithm determined for the 8th shot is the fourth MFNR algorithm, so the fourth MFNR algorithm is the target processing algorithm. The third MFNR algorithm originally corresponding to the second shot is the initial processing algorithm. After replacing the initial processing algorithm with the target processing algorithm, the 2-level processing algorithm corresponding to the second shot is changed to the fourth MFNR algorithm. The replacement process for the third, fifth, and seventh shots is similar and will not be repeated here.

[0396] As shown in Figure 17, when the user performs the 8th click operation, since the 2nd photo processing is still in progress, all the original images corresponding to the 8th photo number, as well as the determined 8-level processing algorithm, 8-level processing frame number and other related data can only be stored, and the system enters the background waiting state.

[0397] Then, when the user performs the 9th to 20th click operation, the processing algorithm corresponding to the number of photos waiting in the background can continue to be replaced with the x-level processing algorithm corresponding to the most recently determined number of photos, as the target processing algorithm. When processing the number of photos waiting in the background, the processing can be performed according to the processing algorithm after the last replacement.

[0398] In the above embodiment, the present application changes the processing algorithm and processing frame number for the number of photos taken in different situations by combining multiple factors such as the environmental detection results in the scene to be photographed, the frequency of user operations, and the memory usage.

[0399] On this basis, since the processing cannot be performed in time when the number of photos is large, this application will also replace the processing algorithm of the waiting number of photos with the processing algorithm determined most recently in time. In this way, the processing can be based on the latest data, adapting to the current memory situation as much as possible, reducing memory increments, and achieving more continuous and rapid photos.

[0400] The above is a detailed introduction to the photographing method provided in the embodiment of the present application. The following describes how a user enables the photographing method provided in the embodiment of the present application in conjunction with the display interface of the electronic device.

[0401] FIG18 is a schematic diagram of a display interface of an electronic device provided in an embodiment of the present application.

[0402] For example, in response to a user's click operation, when the electronic device 100 runs a camera application, the electronic device 100 displays a shooting interface as shown in (a) of Figure 18. A navigation bar is displayed above the interface, wherein an icon indicating "AI Photography" is displayed in the middle.

[0403] In response to the user clicking on "AI Photography", the electronic device 100 displays a prompt box as shown in (b) of Figure 18, which displays a prompt message that "AI Photography" is turned on. At this time, the electronic device 100 can enable the program related to the photo shooting method provided in the embodiment of the present application when shooting.

[0404] It should be understood that the above is only an example of a user enabling the photographing method provided in an embodiment of the present application from the display interface of an electronic device. Of course, the photographing method provided in an embodiment of the present application can also be enabled in other ways, or the photographing method provided in an embodiment of the present application can be directly used by default during the photographing process. The embodiment of the present application does not impose any restrictions on this.

[0405] FIG19 is a schematic diagram of a display interface of another electronic device provided in an embodiment of the present application.

[0406] As shown in Figures 18 and 19, when "AI Photography" is turned on, if the user opens the camera application and quickly and continuously clicks the capture button in the shooting interface multiple times, during the click process, due to the background processing using the shooting method provided by the embodiment of the present application, when the memory usage increases, a processing algorithm with a shorter processing time and / or a smaller number of processing frames will be used for processing. Therefore, the quality of the subsequently processed captured images will be reduced to a certain extent. In order to prompt the user that the image quality has changed, a prompt "The current image quality may be reduced" can be displayed above the preview image in the shooting interface.

[0407] Of course, the above is only an example. Other prompts may be displayed to remind the user, or no prompts may be displayed, or the user may be reminded in other ways. The embodiments of the present application do not impose any restrictions on this.

[0408] The above describes in detail the photography method and related display interface provided by the embodiments of the present application in conjunction with Figures 1 to 19; the following describes in detail the electronic device, device, and chip provided by the embodiments of the present application in conjunction with Figures 20 to 23. Figure 20 shows a hardware system of an electronic device suitable for use in the present application.

[0409] The electronic device 100 can be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc. The embodiment of the present application does not impose any restrictions on the specific type of the electronic device 100.

[0410] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0411] It should be noted that the structure shown in FIG20 does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in FIG20, or the electronic device 100 may include a combination of some of the components shown in FIG20, or the electronic device 100 may include sub-components of some of the components shown in FIG20. The components shown in FIG20 may be implemented in hardware, software, or a combination of software and hardware.

[0412] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices. The controller may generate an operation control signal based on the instruction opcode and timing signal to complete the control of instruction fetching and execution.

[0413] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0414] Exemplarily, the processor 110 can be used to execute the photographing method of the embodiment of the present application; display a first interface, which includes a first control; when a first operation on the first control is detected, in response to the first operation, collect an original image; for the x-th number of photographs, determine the memory usage; when the memory usage is less than the first memory threshold, determine the x-level processing algorithm as the first photographing algorithm, and determine the x-level processing frame number as the first frame number, where x is an integer greater than 1; when the memory usage is greater than or equal to the first memory threshold but less than the second memory threshold, determine the x-level processing algorithm as the second photographing algorithm, and determine the x-level processing frame number as the second frame number; when the memory usage is greater than the second memory threshold, determine the x-level processing algorithm as the third photographing algorithm, and determine the x-level processing frame number as the third frame number, where the first memory threshold is less than the second memory threshold; use the x-level processing algorithm to process the x-level processing frames in the original image to obtain a corresponding photographed image; and save the photographed image.

[0415] 20 is merely a schematic illustration and does not limit the connection relationship between the modules of the electronic device 100. Alternatively, the modules of the electronic device 100 may adopt a combination of the multiple connection modes described in the above embodiments.

[0416] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0417] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 is also used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance).

[0418] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0419] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0420] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0421] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0422] The camera 193 is used to capture still images or videos. It can be triggered to turn on via application instructions to implement the camera function, such as capturing images of any scene. The camera may include components such as an imaging lens, a filter, and an image sensor. Light emitted or reflected by an object enters the imaging lens, passes through the filter, and is ultimately focused on the image sensor. The imaging lens is primarily used to focus and image the light emitted or reflected by all objects within the camera's viewing angle (also referred to as the scene to be captured, the target scene, or the scene image the user expects to capture); the filter is primarily used to filter out excess light waves (e.g., light waves other than visible light, such as infrared); and the image sensor may be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The image sensor is primarily used to perform photoelectric conversion on the received light signal, converting it into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

[0423] Among them, the camera 193 can be located in the front of the electronic device 100 or in the back of the electronic device 100. The specific number and arrangement of the cameras can be set according to needs, and this application does not impose any restrictions.

[0424] Exemplarily, the electronic device 100 includes a front camera and a rear camera. For example, the front camera or the rear camera may include one or more cameras. Taking the electronic device 100 having four rear cameras as an example, when the electronic device 100 activates the four rear cameras to take pictures, the photo-taking method provided in the embodiment of the present application can be used. Alternatively, the camera is set on an external accessory of the electronic device 100, and the external accessory is rotatably connected to the frame of the mobile phone, and the angle formed between the external accessory and the display screen 194 of the electronic device 100 is any angle between 0-360 degrees. For example, when the electronic device 100 takes a selfie, the external accessory drives the camera to rotate to a position facing the user. Of course, when the mobile phone has multiple cameras, only some of the cameras can be set on the external accessories, and the remaining cameras can be set on the electronic device 100 body. The embodiment of the present application does not impose any restrictions on this.

[0425] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0426] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0427] The internal memory 121 can also store the software code of the photographing method provided in the embodiment of the present application. When the processor 110 runs the software code, the process steps of the photographing method are executed to achieve fast and continuous photographing.

[0428] The internal memory 121 may also store captured images.

[0429] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music can be stored in the external memory card.

[0430] Of course, the software code of the photographing method provided in the embodiment of the present application can also be stored in an external memory, and the processor 110 can run the software code through the external memory interface 120 to execute the process steps of the photographing method to obtain multiple frames of captured images. The captured images obtained by the electronic device 100 can also be stored in an external memory.

[0431] It should be understood that the user can specify whether to store the image in the internal memory 121 or the external memory. For example, when the electronic device 100 is connected to the external memory, if the electronic device 100 captures a frame of image, a prompt message may pop up to prompt the user to store the image in the external memory or the internal memory. Of course, other designation methods are possible, and the present embodiment does not impose any restrictions on this. Alternatively, when the electronic device 100 detects that the memory capacity of the internal memory 121 is less than a preset amount, it may automatically store the image in the external memory.

[0432] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0433] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0434] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can also be used for image stabilization.

[0435] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0436] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0437] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0438] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0439] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0440] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0441] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0442] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0443] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0444] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0445] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0446] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or touch vibration feedback. For example, touch operations on different applications (such as taking photos, playing audio, etc.) can correspond to different vibration feedback effects.

[0447] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0448] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting or removing the SIM card into or from the SIM card interface 195 .

[0449] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0450] The hardware system of the electronic device 100 is described in detail above. The software system of the electronic device 100 is introduced below.

[0451] Figure 21 is a schematic diagram of the software system of the electronic device provided in an embodiment of the present application.

[0452] As shown in FIG. 21 , the system architecture may include an application layer (APP) 210 , an application framework layer 220 , a hardware abstract layer (HAL) 230 , a driver layer 240 , and a hardware layer 250 .

[0453] The application layer 210 may include a camera application or other applications. Other applications include but are not limited to: camera, gallery, and other applications.

[0454] The application layer 210 is at the top of the entire framework and is responsible for interacting directly with the user. Once it receives a direct or indirect request from the user, such as taking a photo, it will send the request to the application framework layer 220 through the interface and wait for the application framework layer 220 to return the processing result, which includes image data and camera parameters, etc.; then the application layer 210 will feed back the result to the user.

[0455] The application framework layer 220 may provide an application programming interface (API) and a programming framework to the application programs of the application layer; the application framework layer may include some predefined functions.

[0456] For example, the application framework layer 220 may include a camera access interface; the camera access interface may include camera management and camera equipment; wherein, camera management may be used to provide an access interface for managing the camera; and camera equipment may be used to provide an interface for accessing the camera.

[0457] The hardware abstraction layer 230 is used to abstract hardware. For example, the hardware abstraction layer may include a camera hardware abstraction layer and other hardware device abstraction layers. The camera hardware abstraction layer may include camera device 1, camera device 2, etc. The camera hardware abstraction layer may be connected to the camera algorithm library, and the camera hardware abstraction layer may call the algorithms in the camera algorithm library.

[0458] In this application, the perception engine used to perform various detections can be set in the hardware abstraction layer.

[0459] The driver layer 240 is used to provide drivers for different hardware devices. For example, the driver layer may include a camera device driver, a digital signal processor driver, and a graphics processor driver.

[0460] The hardware layer 250 may include multiple image sensors, multiple image signal processors, digital signal processors, graphics processors, and other hardware devices.

[0461] For example, the hardware layer 250 includes sensors and image signal processors. The sensors may include sensor 1, sensor 2, a depth sensor (time of flight, TOF), a multispectral sensor, etc. The image signal processors may include image signal processor 1, image signal processor 2, etc.

[0462] In this application, by calling the hardware abstraction layer interface in the hardware abstraction layer 230, the connection between the application layer 210 and the application framework layer 220 above the hardware abstraction layer 230 and the driver layer 240 and the hardware layer 250 below can be realized, thereby realizing camera data transmission and function control.

[0463] Manufacturers can customize functionality within the camera hardware interface layer within the hardware abstraction layer 230. Compared to the hardware abstraction layer interface, the camera hardware interface layer is more efficient, flexible, and has lower latency. It also allows for more comprehensive access to the ISP and GPU for image processing. Images input to the hardware abstraction layer 230 can come from an image sensor or a stored image.

[0464] The scheduling layer in the hardware abstraction layer 230 includes a general functional interface for implementing management and control.

[0465] The camera service layer in the hardware abstraction layer 230 is used to access the ISP and other hardware interfaces.

[0466] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0467] The camera application in the application layer can be displayed as an icon on the screen of the electronic device 100. When the camera application icon is clicked by the user to trigger it, the electronic device 100 starts running the camera application. When the camera application is running on the electronic device 100, the camera application calls the corresponding interface of the camera application in the application framework layer 210, then starts the camera driver by calling the hardware abstraction layer 230, turns on the camera 193 on the electronic device 100, and simultaneously the camera algorithm library starts loading the photography method used in the embodiment of the present application.

[0468] Then, after one or more frames of raw images are captured by the image sensor, the captured raw images can be processed by the image signal processor and returned to the hardware abstraction layer, and processed using a processing algorithm called from the camera algorithm library to generate captured images, which are then saved and / or transmitted to the display screen for display.

[0469] The following describes an image processing device 300 for implementing the above-mentioned photographing method provided in an embodiment of the present application. FIG22 is a schematic diagram of the image processing device 300 provided in an embodiment of the present application.

[0470] As shown in FIG. 22 , the image processing apparatus 300 includes a display unit 310 , an acquisition unit 320 , and a processing unit 330 .

[0471] The display unit 310 is used to display a first interface, which includes a first control.

[0472] The acquisition unit 320 is configured to detect a first operation performed by a user on a first control.

[0473] The processing unit 330 is configured to capture an original image once in response to the first operation.

[0474] The processing unit 330 is also used to determine the memory occupancy for the x-th number of photos; when the memory occupancy is less than the first memory threshold, determine the x-level processing algorithm as the first photo algorithm, and determine the x-level processing frame number as the first frame number, where x is an integer greater than 1; when the memory occupancy is greater than or equal to the first memory threshold but less than the second memory threshold, determine the x-level processing algorithm as the second photo algorithm, and determine the x-level processing frame number as the second frame number; when the memory occupancy is greater than the second memory threshold, determine the x-level processing algorithm as the third photo algorithm, and determine the x-level processing frame number as the third frame number, where the first memory threshold is less than the second memory threshold; use the x-level processing algorithm to process the x-level processing frames in the original image to obtain a corresponding captured image; and save the captured image.

[0475] It should be noted that the image processing device 300 is implemented in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0476] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the aforementioned functionality. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0477] Therefore, the units of each example described in the embodiments of this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0478] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium runs on the image processing device 300, the image processing device 300 executes the photographing method shown above.

[0479] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium (e.g., a solid state disk (SSD)).

[0480] The embodiment of the present application further provides a computer program product including computer instructions, which, when executed on the image processing device 300, enables the image processing device 300 to execute the aforementioned photographing method.

[0481] FIG23 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip shown in FIG23 can be a general-purpose processor or a dedicated processor. The chip includes a processor 401. The processor 401 is used to support the image processing device 300 in executing the technical solution shown above.

[0482] Optionally, the chip further includes a transceiver 402 , which is configured to accept control from the processor 401 and to support the image processing apparatus 300 in executing the aforementioned technical solution.

[0483] Optionally, the chip shown in FIG23 may further include: a storage medium 403 .

[0484] It should be noted that the chip shown in Figure 23 can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.

[0485] The electronic device, image processing device 300, computer storage medium, computer program product, and chip provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0486] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application. Based on the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, certain steps in each embodiment of the above detection method may be unnecessary, or certain new steps may be added. Or a combination of any two or any multiple embodiments described above. Such modifications, changes, or combined solutions also fall within the scope of the embodiments of the present application.

[0487] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced with each other. For the sake of brevity, they will not be repeated here.

[0488] It should also be understood that the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0489] It should also be understood that in the embodiments of the present application, "pre-setting" and "pre-definition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device (for example, including an electronic device), and the present application does not limit its specific implementation method.

[0490] It should also be understood that the division of the modes, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various modes, categories, situations and embodiments can be combined without contradiction.

[0491] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0492] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A photographing method, characterized in that: Applied to electronic equipment, the method includes: Displaying a first interface, wherein the first interface includes a first control; When a first operation on the first control is detected, capturing an original image in response to the first operation; Determine the memory usage for the xth number of photos; When the memory usage is less than the first memory threshold, determining the x-level processing algorithm as the first photographing algorithm, and determining the x-level processing frame number as the first frame number, where x is an integer greater than 1; When the memory usage is greater than or equal to the first memory threshold but less than a second memory threshold, determining that the x-level processing algorithm is a second photographing algorithm, and determining that the x-level processing frame number is a second frame number; When the memory usage is greater than the second memory threshold, determining that the x-level processing algorithm is a third photographing algorithm, and determining that the x-level processing frame number is a third frame number, and the first memory threshold is less than the second memory threshold; Processing the x-level processing frames in the original image using the x-level processing algorithm to obtain a corresponding captured image; Saving the captured image; Wherein, when the first photographing algorithm, the second photographing algorithm and the third photographing algorithm are the same, the first frame number, the second frame number and the third frame number gradually decrease; When the processing times of the first photographing algorithm, the second photographing algorithm, and the third photographing algorithm gradually decrease, the first number of frames, the second number of frames, and the third number of frames are the same or gradually decrease.

2. The photographing method according to claim 1, wherein: The method further comprises: For the x-th number of photographs, when the device temperature is greater than or equal to the temperature threshold, processing one frame of the original image using a single-frame algorithm to obtain a corresponding photographed image; When the first photographing algorithm, the second photographing algorithm, and the third photographing algorithm are the same, the processing time of the single-frame algorithm is shorter than the processing time of the third photographing algorithm. When the processing time of the first photographing algorithm, the second photographing algorithm and the third photographing algorithm gradually decreases, the processing time of the single-frame algorithm is the same as the processing time of the third photographing algorithm, or the processing time of the single-frame algorithm is shorter than the processing time of the third photographing algorithm.

3. The photographing method according to claim 1 or 2, characterized in that: The method further comprises: When the first operation corresponds to the first photo-taking time, performing environmental detection for the scene to be photographed; Determine the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first number of photographs according to the environmental detection result; The first-level processing algorithm is used to process the first-level processing frame number in the original image collected corresponding to the first number of photographs to obtain a corresponding photographed image.

4. The photographing method according to claim 3, wherein: The environmental detection includes at least illumination detection and dynamic range detection, and the environmental detection result includes at least illumination and dynamic range values; Determining the number of level 1 processing frames and the level 1 processing algorithm corresponding to the first number of photographs according to the environmental detection result includes: When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the The number of level 1 processing frames is a1 frames, and the level 1 processing algorithm is determined to be the A1 algorithm; When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, determining the number of level 1 processing frames to be b1 frames, and determining the level 1 processing algorithm to be the B1 algorithm; When the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, determining the number of level 1 processing frames to be c1 frames and determining the level 1 processing algorithm to be the C1 algorithm; When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, determining the number of level 1 processing frames to be d1 frames, and determining the level 1 processing algorithm to be the D1 algorithm; Wherein, a1, b1, c1 and d1 are all integers greater than or equal to 1.

5. The photographing method according to claim 4, characterized in that: Using the first-level processing algorithm, processing the first-level processing frame number in the original image corresponding to the first shooting number to obtain the corresponding captured image, including: When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the A1 algorithm is used to process the a1 frame in the original image corresponding to the first photographing time to obtain a corresponding photographed image; When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, the B1 algorithm is used to process the b1 frame in the original image corresponding to the first photographing time to obtain a corresponding photographed image; When the illumination is greater than or equal to the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the C1 algorithm is used to process the C1 frame of the original image collected corresponding to the first shooting time to obtain a corresponding captured image; When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, the D1 algorithm is used to process the d1 frame in the original image collected corresponding to the first shooting time to obtain a corresponding captured image.

6. The photographing method according to any one of claims 1 to 5, characterized in that: After determining that the x-level processing algorithm is the third photographing algorithm and determining that the x-level processing frame number is the third frame number, the method further includes: Determining a frequency of a first operation on the first control, and a difference between a memory watermark and the memory usage; When the frequency meets the preset frequency condition or the difference is less than the preset difference, determining the number of photos to be taken in the background waiting; The x-level processing algorithms corresponding to the number of times of photos waiting in the background are respectively used as the initial processing algorithms; The x-level processing algorithm corresponding to the most recently determined number of photos is used as the target processing algorithm; Using the target processing algorithm to replace the initial processing algorithm of each of the number of photos waiting in the background; When any one of the photographing times waiting in the background is called for processing, the target processing algorithm is used to process the x-level processing frames in the original image corresponding to the any one of the photographing times to obtain a corresponding photographed image.

7. The photographing method according to any one of claims 1 to 6, characterized in that: The first interface refers to a photo taking interface, and the first control refers to a control for instructing photo taking.

8. The photographing method according to any one of claims 1 to 6, characterized in that: The first interface refers to a video recording interface, and the first control refers to a control for instructing snapshot.

9. An electronic device, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 8.

10. A chip, characterized in that: The chip is applied to an electronic device, and the chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 8.

Images