Image processing method and related equipment thereof

Abstract

The invention provides an image processing method and related equipment, and relates to the field of image processing, and the image processing method comprises the steps: displaying a first interface, the first interface comprises a preview image and a first control, and the first control indicates a photographing mode; when a first operation on a control is detected, in response to the first operation, collecting an original image once by adopting a camera; determining background photographing waiting times; when the number of times of waiting for photographing by the background is less than or equal to M, processing the original image by using a first photographing algorithm to obtain a corresponding photographed image; and when the background waiting photographing algorithm is greater than M and less than or equal to N, processing the original image by using a second photographing algorithm to obtain a corresponding photographed image, and storing the photographed image. According to the method, the photographing waiting times are counted in the background, the user behavior and the user intention are distinguished, and different time-consuming algorithms are adaptively selected for processing, so that the photographing efficiency can be improved, and the effect and the performance are considered.

Description

Image processing method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on July 28, 2022, with application number 202210900690.0 and application name “Image Processing Method and Related Equipment Thereof”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of image technology, and in particular to an image processing method and related equipment. Background Art

[0003] With the popularity of electronic devices with camera functions in our lives, taking photos with electronic devices has become a daily behavior.

[0004] Currently, most electronic devices use camera parameters to determine the capture algorithm when taking photos. The resulting image is then processed and generated based on the algorithm. Commonly used camera parameters include brightness, dynamic range, focal length, and object distance. However, these algorithms often take a long time to run, or they can run quickly but produce poor quality images, failing to achieve the desired balance between quality and performance. A new approach is urgently needed to address this issue.

[0005] Summary of the Invention

[0006] The present application provides an image processing method and related equipment, which can improve the efficiency of taking pictures by distinguishing the number of times the background waits for taking pictures and adaptively selecting different time-consuming algorithms for processing, thereby taking into account both effects and performance.

[0007] To achieve the above objectives, this application adopts the following technical solutions:

[0008] In a first aspect, an image processing method is provided, which is applied to an electronic device including a camera, the method comprising:

[0009] The electronic device displays a first interface, the first interface including a preview image and a first control, and the first control indicates a photo taking mode;

[0010] When a first operation on the first control is detected, in response to the first operation, the electronic device uses the camera to capture an original image once;

[0011] Determine the number of backend waiting photos, where the number of backend waiting photos is equal to the current cumulative number of photos minus the number of photos that have been processed and the number of photos being processed;

[0012] When the number of times the background waits for taking photos is less than or equal to M, the original image is processed using a first photo-taking algorithm to obtain a corresponding photographed image;

[0013] When the number of times the background waits for taking photos is greater than M but less than or equal to N, the original image is processed using a second photo-taking algorithm to obtain a corresponding captured image;

[0014] Saving the captured image;

[0015] Wherein, M<N, M and N are both positive integers, and the processing time of the second photographing algorithm is less than the processing time of the first photographing algorithm.

[0016] Optionally, the first operation is a click operation. The first operation may also include a voice instruction operation or other operations that instruct the electronic device to take a photo.

[0017] An embodiment of the present application provides an image processing method. In response to a user's first operation on a first control, the electronic device uses a camera to capture an original image. Then, by determining the number of times the background waits for taking photos, and according to the number of times the background waits for taking photos, different photo algorithms are called for processing. For example, when the number of times the background waits for taking photos is large, a photo algorithm with a relatively short time consumption is called for processing; when the number of times the background waits for taking photos is large, a photo algorithm with a relatively long time consumption is called for processing. Thus, by dynamically calling different algorithms, the running time of the photo algorithm is coordinated, the filming rate is improved, and both effect and performance are taken into account.

[0018] In a possible implementation of the first aspect, the method further includes:

[0019] When the number of times the background waits for taking photos is greater than N, the original image is processed using a third photo algorithm to obtain a corresponding captured image, and the processing time of the third photo algorithm is less than the processing time of the second photo algorithm.

[0020] Optionally, the third photographing algorithm is a single-frame algorithm.

[0021] In this implementation, when the number of times the background waits for taking photos is large, it means that the frequency of user clicks is very high, and there is a lot of data that the electronic device cannot process in time. At this time, a relatively short algorithm can be used to process the original image to improve processing efficiency and ensure that data related to the number of times the background waits for taking photos can be processed in time, reducing waiting and thus avoiding the problem of frame loss.

[0022] In a possible implementation of the first aspect, the method further includes:

[0023] When a second operation on the first control is detected, in response to the second operation, the electronic device uses the camera to capture multiple original images;

[0024] The original image collected each time is processed using a fourth photographing algorithm to obtain corresponding photographed images respectively; the processing time of the fourth photographing algorithm is less than the processing time of the second photographing algorithm.

[0025] Optionally, the second operation may be a long press operation. The second operation may also include a voice instruction operation or other operations that instruct the electronic device to take multiple photos continuously.

[0026] A long press refers to the user touching the capture button for a longer period of time. The time from the moment the finger is pressed down to the moment the finger is lifted up is the duration of a long press. The duration of a long press is relatively longer than that of a tap.

[0027] Optionally, the fourth photographing algorithm may be the same as the third photographing algorithm, and the fourth photographing algorithm is a single-frame algorithm.

[0028] In this implementation, when the user performs a long press, indicating that the user wishes to take multiple photos in succession, the electronic device captures raw images at a high frequency. If the backend were to use a relatively time-consuming algorithm to process each captured raw image, it would undoubtedly be unable to process all of them, resulting in data congestion. Therefore, when the user performs a long press, a relatively quick algorithm should be used to process each captured raw image to improve processing efficiency.

[0029] In a possible implementation of the first aspect, before processing the original image using the first photographing algorithm to obtain a corresponding captured image, the method further includes:

[0030] Detecting the scene to be captured and obtaining detection results, wherein the detection includes at least one of AI scene recognition, motion detection, face detection, face attribute detection, illumination detection, and dynamic range detection;

[0031] In this implementation, the embodiment of the present application detects the scene to be shot, refines the situation in the scene, and accordingly refines the processing method of the original image, thereby adaptively improving the quality and effect of the captured image obtained in each situation.

[0032] In a possible implementation of the first aspect, detecting a scene to be photographed and obtaining a detection result includes:

[0033] Performing front-end processing on the original image, wherein the front-end processing is used to correct the original image;

[0034] Perform detection on the original image after front-end processing to obtain the detection results.

[0035] In the embodiment of the present application, the image after the first front-end processing is in the RAW domain, and the image after the second front-end processing is in the YUV domain.

[0036] In a possible implementation of the first aspect, the front-end processing includes a first front-end processing and a second front-end processing;

[0037] The first front-end processing includes: at least one of bad pixel correction, RAW domain noise reduction, black level correction, lens shading correction, and automatic white balance;

[0038] The second front-end processing includes at least one of color interpolation, color correction, global tone mapping processing, and gamma correction.

[0039] In this implementation, the image in the YUV domain after the first and second front-end processing undergoes some corrections and adjustments relative to the original image, resulting in a more accurate image. Subsequently, the image in the YUV domain can be tested to obtain a more accurate test result. This test result is the test result corresponding to the scene to be captured.

[0040] In a possible implementation of the first aspect, when the number of times the background waits for taking photos is less than or equal to M, processing the original image using a first photo-taking algorithm to obtain a corresponding captured image includes:

[0041] When the number of times the background waits for taking photos is less than or equal to M, determining whether the preset subject in the scene to be photographed is static according to the detection result;

[0042] When the scene is static, determining whether the illumination in the scene to be captured is greater than a first illumination threshold;

[0043] When the illumination value is less than or equal to the first illumination threshold, determining that the first night scene algorithm is the first photographing algorithm, and using the first night scene algorithm to process the P frame in the original image to obtain a corresponding photographed image;

[0044] When the illumination intensity is greater than the first illumination threshold, determining whether the dynamic range of the scene to be captured is greater than a first dynamic range threshold;

[0045] When the dynamic range is less than or equal to the first dynamic range threshold, determining a first MFNR algorithm as the first photographing algorithm, and processing the P frame in the original image using the first MFNR algorithm to obtain a corresponding photographed image;

[0046] When it is greater than the first dynamic range threshold, the first HDR algorithm is determined to be the first photographing algorithm, and the first HDR algorithm is used to process the P frame in the original image to obtain a corresponding photographed image, where P is an integer greater than 1.

[0047] In this implementation, when the number of photos waiting in the background is determined to be small, which can also be called judging that the camera is in a normal photo-taking state, the application performs multiple tests on the scene to be photographed. Based on the test results, on the basis of determining whether the preset photographed object is static, the scene to be photographed is further divided according to the detected illumination and dynamic range, so that the scene to be photographed can be divided into multiple situations. Then, different algorithms are used for each situation to process the original images of different frame numbers, so that the processing method of the original image can be refined, and the quality and effect of the captured image obtained in each situation can be adaptively improved.

[0048] In a possible implementation of the first aspect, processing the P frame in the original image using the first night scene algorithm to obtain a corresponding captured image includes:

[0049] Performing a third front-end processing on the P frame in the original image to obtain a first intermediate image of the P frame;

[0050] Using the first night scene algorithm, processing the P frames of the first intermediate image to obtain a frame of the second intermediate image;

[0051] Perform first back-end processing on one frame of the second intermediate image to obtain a corresponding captured image.

[0052] The third front-end processing is used to correct the P frame in the original image, and the first back-end processing is used to correct the second intermediate image.

[0053] Optionally, when multiple frames of original images are selected for processing, the multiple frames of original images may include at least one of long-exposure original images, normal-exposure original images, and short-exposure original images.

[0054] Optionally, a night scene algorithm module may be provided in the electronic device, and the electronic device may implement the processing of the first night scene algorithm through the night scene algorithm module.

[0055] In this implementation, when the illumination of the scene to be captured is detected to be less than or equal to the first illumination threshold, it indicates that the ambient brightness of the scene to be captured is relatively dark. In this case, if a general photography algorithm is used for processing, the resulting captured image will be unclear, and the user will not be able to see the details in the captured image. To address this, in this embodiment of the application, the first night scene algorithm can be selected for processing to increase the brightness of the image during processing, so that the captured image can present more details.

[0056] In a possible implementation of the first aspect, processing the P frame in the original image using the first MFNR algorithm to obtain a corresponding captured image includes:

[0057] performing a fourth front-end processing on the P frame in the original image to obtain a third intermediate image of the P frame;

[0058] Processing the third intermediate image of the P frame using the first MFNR algorithm to obtain a fourth intermediate image;

[0059] Performing second back-end processing on one frame of the fourth intermediate image to obtain a corresponding captured image.

[0060] The fourth front-end processing is used to correct the P frame in the original image, and the second back-end processing is used to correct the fourth intermediate image.

[0061] In this implementation, when it is detected that the preset subject in the scene to be captured is static, the illumination is greater than the first illumination threshold, and the dynamic range is less than or equal to the first dynamic range threshold, it indicates that the ambient light in the scene to be captured is strong, but the dynamic range is small. In this case, if a general photography algorithm is used for processing, the resulting captured image will have high noise and a small dynamic range. To address this, in this embodiment of the application, the first MFNR algorithm can be selected for processing to reduce noise in the captured image and improve the dynamic range of the captured image.

[0062] In a possible implementation of the first aspect, processing the P frame in the original image using the first HDR algorithm to obtain a corresponding captured image includes:

[0063] performing a fifth front-end processing on the P frame in the original image to obtain a fifth intermediate image of the P frame;

[0064] Using the first HDR algorithm, processing the fifth intermediate image of the P frame to obtain a sixth intermediate image;

[0065] Perform third back-end processing on one frame of the sixth intermediate image to obtain a corresponding captured image.

[0066] The fifth front-end processing is used to correct the P frame in the original image, and the third back-end processing is used to correct the sixth intermediate image.

[0067] In an embodiment of the present application, when multiple frames of original images are selected to be processed using the first HDR algorithm, the multiple frames of original images may correspond to different exposure values ​​respectively.

[0068] In this implementation, when the dynamic range of the scene being captured is detected to be greater than the first dynamic range threshold, it indicates that the ambient light intensity in the scene being captured is high, and the scene being captured can be considered a high dynamic range scene. In this case, if a conventional photography algorithm is used for processing, the resulting image may be overexposed. To address this issue, in this embodiment of the present application, a first HDR algorithm can be selected for processing to adjust the image's dynamic range and image details, making the captured image more realistic and improving image quality.

[0069] In a possible implementation of the first aspect, the method further includes:

[0070] If the subject is not static, determining whether the moving speed of the preset subject is greater than a preset speed threshold;

[0071] When the speed is greater than the preset speed threshold, determining whether the illumination in the scene to be captured is greater than a second illumination threshold;

[0072] When the value is less than or equal to the second illumination threshold, determining that the second night scene algorithm is the first photographing algorithm, and using the second night scene algorithm to process the Q frame in the original image to obtain a corresponding photographed image;

[0073] When the illumination intensity is greater than the second illumination threshold, determining whether the dynamic range of the scene to be captured is greater than a second dynamic range threshold;

[0074] When the dynamic range is less than or equal to the second dynamic range threshold, determining that the second MFNR algorithm is the first photographing algorithm, and using the second MFNR algorithm to process the Q frame in the original image to obtain a corresponding photographed image;

[0075] When it is greater than the second dynamic range threshold, the second HDR algorithm is determined to be the first photographing algorithm, and the second HDR algorithm is used to process Q frames in the original image to obtain a corresponding photographed image, where Q is an integer greater than or equal to 1, and Q<P.

[0076] In a possible implementation of the first aspect, processing the Q frame in the original image using the second night scene algorithm to obtain a corresponding captured image includes:

[0077] performing a sixth front-end processing on the Q frame in the original image to obtain a seventh intermediate image of the Q frame;

[0078] Using the second night scene algorithm, processing the Q frames of the seventh intermediate image to obtain one frame of the eighth intermediate image;

[0079] Perform fourth back-end processing on one frame of the eighth intermediate image to obtain a corresponding captured image.

[0080] The sixth front-end processing is used to correct the Q frame in the original image, and the fourth back-end processing is used to correct the eighth intermediate image.

[0081] In this implementation, when the speed of a preset subject in the scene to be captured is detected to be greater than a preset speed threshold, and the illumination in the scene to be captured is less than or equal to a second illumination threshold, it indicates that the preset subject is moving in a low-light environment, and the movement amplitude is relatively large, which can be called a fast motion state. In this case, using a general photography algorithm for processing may result in blurred or ghosting images. To address this issue, in this embodiment of the present application, a second night scene algorithm can be selected for processing to improve image clarity and ensure image quality.

[0082] In a possible implementation of the first aspect, processing the Q frame in the original image using the second MFNR algorithm to obtain a corresponding captured image includes:

[0083] performing a seventh front-end processing on the Q frame in the original image to obtain a ninth intermediate image of the Q frame;

[0084] Processing the ninth intermediate image of Q frames using the second MFNR algorithm to obtain a tenth intermediate image of one frame;

[0085] Perform fifth back-end processing on one frame of the tenth intermediate image to obtain a corresponding captured image.

[0086] The seventh front-end processing is used to correct the Q frame in the original image, and the fifth back-end processing is used to correct the tenth intermediate image.

[0087] In this implementation, when the preset subject is dynamic and its motion speed is greater than a preset speed threshold, the illumination in the scene to be captured is greater than a second illumination threshold, but the dynamic range is less than or equal to a second dynamic range threshold, this indicates that the preset subject is moving at a relatively large amplitude and is rapidly moving in a bright scene with a small dynamic range. In this case, if a general photography algorithm is used for processing, the resulting captured image will have high noise and a small dynamic range. Therefore, in this embodiment of the present application, a second MFNR algorithm can be selected for processing to reduce noise in the captured image and improve the dynamic range of the captured image.

[0088] In a possible implementation of the first aspect, processing the Q frame in the original image using the second HDR algorithm to obtain a corresponding captured image includes:

[0089] performing an eighth front-end processing on the Q frame in the original image to obtain an eleventh intermediate image of the Q frame;

[0090] Using the second HDR algorithm, processing the Q frames of the eleventh intermediate image to obtain a frame of the twelfth intermediate image;

[0091] Perform sixth back-end processing on one frame of the twelfth intermediate image to obtain a corresponding captured image.

[0092] The eighth front-end processing is used to correct the Q frame in the original image, and the sixth back-end processing is used to correct the twelfth intermediate image.

[0093] In this implementation, when the preset subject is dynamic, the motion speed is greater than the preset speed threshold, the illumination in the scene to be captured is greater than the second illumination threshold, and the dynamic range is greater than the second dynamic range threshold, it indicates that the preset subject has a relatively large motion amplitude and is in a fast motion state in a high dynamic range scene. In this case, if a general photography algorithm is used for processing, the obtained captured image may be overexposed. To address this issue, in an embodiment of the present application, a second HDR algorithm can be selected for processing to adjust the dynamic range and image details of the image, making the content presented in the captured image more realistic and improving image quality.

[0094] In a possible implementation of the first aspect, the method further includes:

[0095] When the speed is less than or equal to the preset speed threshold, the single-frame algorithm is determined to be the first photographing algorithm, and the single-frame algorithm is used to process one frame of the original image to obtain a corresponding photographed image.

[0096] In this implementation, when it is detected that the preset subject in the scene to be photographed is in motion and the motion speed is less than or equal to the preset speed threshold, it indicates that the motion amplitude of the preset subject is relatively small, which can be called a slow motion state. In this case, a general photography algorithm can be used to process it, for example, a single-frame algorithm can be used for processing.

[0097] In a possible implementation of the first aspect, processing one frame of the original image using the single-frame algorithm to obtain a corresponding captured image includes:

[0098] performing the ninth front-end processing on one frame of the original image to obtain a thirteenth intermediate image;

[0099] The seventh back-end processing is performed on one frame of the thirteenth intermediate image to obtain a corresponding captured image.

[0100] The ninth front-end processing is used to correct one frame of the original image, and the seventh back-end processing is used to correct the thirteenth intermediate image.

[0101] In a possible implementation of the first aspect, when the number of times the background waits for taking photos is greater than M but less than or equal to N, processing the original image using a second photo-taking algorithm to obtain a corresponding captured image includes:

[0102] When the number of times the background waiting for taking photos is greater than M but less than or equal to N, determining whether the illumination in the scene to be photographed is greater than a third illumination threshold according to the detection result;

[0103] When the illumination value is less than or equal to the third illumination threshold, determining the third night scene algorithm as the second photographing algorithm, and using the third night scene algorithm to process the R frame in the original image to obtain a corresponding photographed image;

[0104] When the illumination intensity is greater than the third illumination threshold, determining whether the dynamic range of the scene to be captured is greater than the third dynamic range threshold;

[0105] When the value is less than or equal to the third dynamic range threshold, determining a third MFNR algorithm as the second photographing algorithm, and processing the R frame in the original image using the third MFNR algorithm to obtain a corresponding photographed image;

[0106] When it is greater than the third dynamic range threshold, the third HDR algorithm is determined to be the second photographing algorithm, and the third HDR algorithm is used to process the R frame in the original image to obtain a corresponding captured image, where R is an integer greater than or equal to 1, and R≤Q<P.

[0107] In this implementation, when the present application determines that the number of photos waiting in the background is relatively large, which can also be called determining that it is in a continuous photo-taking state, it divides the scene to be photographed according to the detected illumination and dynamic range, so that the scene to be photographed can be divided into multiple situations. Then, different algorithms are used to process the original images with a relatively small number of frames for each situation, so that the processing method of the original image can be refined. While ensuring the quality of the captured image, the processing efficiency is improved, the number of photos waiting in the background is reduced, and both effect and performance are taken into consideration.

[0108] In a possible implementation of the first aspect, processing the R frame in the original image using the third night scene algorithm to obtain a corresponding captured image includes:

[0109] performing a tenth front-end processing on the R frame in the original image to obtain a fourteenth intermediate image of the R frame;

[0110] Using the third night scene algorithm, processing the fourteenth intermediate image of the R frame to obtain a fifteenth intermediate image of the R frame;

[0111] Perform the eighth back-end processing on one frame of the fifteenth intermediate image to obtain a corresponding captured image.

[0112] The tenth front-end processing is used to correct the R frame in the original image, and the eighth back-end processing is used to correct the fifteenth intermediate image.

[0113] In this implementation, when it is detected that the number of background waiting times for photos is moderate and the illumination of the scene to be captured is less than or equal to the third illumination threshold, it indicates that the user is continuously taking photos in a low-light environment. In this case, using a general photo algorithm for processing will result in a blurry image with unclear details. To address this, in this embodiment of the application, the third night scene algorithm, which takes a relatively short time, can be selected for processing. While improving image clarity, it can also improve processing efficiency, reduce the number of background waiting times for photos, and balance effect and performance.

[0114] In a possible implementation of the first aspect, processing the R frame in the original image using the third MFNR algorithm to obtain a corresponding captured image includes:

[0115] performing an eleventh front-end processing on the R frame in the original image to obtain a sixteenth intermediate image of the R frame;

[0116] Processing the sixteenth intermediate image of the R frame using the third MFNR algorithm to obtain a seventeenth intermediate image of the R frame;

[0117] The ninth back-end processing is performed on the seventeenth intermediate image of one frame to obtain a corresponding captured image.

[0118] The eleventh front-end processing is used to correct the R frame in the original image, and the ninth back-end processing is used to correct the seventeenth intermediate image.

[0119] In this implementation, when it is detected that the number of background waiting times for photos is moderate, and the illumination in the scene to be photographed is greater than the third illumination threshold, while the dynamic range is less than or equal to the third dynamic range threshold, it indicates that the user is taking continuous photos in a bright environment but with a small dynamic range. In this case, using a general photo algorithm for processing will result in a captured image with high noise, a small dynamic range, and unrealistic content. To address this, in an embodiment of the present application, a third MFNR algorithm with a relatively short processing time can be selected for processing. While reducing noise and increasing the dynamic range of the captured image, it can also improve processing efficiency, reduce the number of background waiting times for photos, and balance effects and performance.

[0120] In a possible implementation of the first aspect, processing the R frame in the original image using a third HDR algorithm to obtain a corresponding captured image includes:

[0121] performing a twelfth front-end processing on the R frame in the original image to obtain an eighteenth intermediate image of the R frame;

[0122] Processing the eighteenth intermediate image of frame R using the third HDR algorithm to obtain a nineteenth intermediate image of frame R;

[0123] The tenth back-end processing is performed on one frame of the nineteenth intermediate image to obtain a corresponding captured image.

[0124] The twelfth front-end processing is used to correct the R frame in the original image, and the tenth back-end processing is used to correct the nineteenth intermediate image.

[0125] In this implementation, when it is detected that the number of background waiting times for taking photos is in the middle, and the illumination in the scene to be photographed is greater than the third illumination threshold, and the dynamic range is also greater than the third dynamic range threshold, it indicates that the user is taking continuous photos in a bright high dynamic range environment. At this time, using a general photo algorithm for processing, the obtained captured image may be overexposed. To this end, in an embodiment of the present application, a third HDR algorithm with a relatively short processing time can be selected for processing. While adjusting the dynamic range of the image, enhancing the image details, and making the content presented in the captured image more realistic, it can also improve processing efficiency, reduce the number of background waiting times for taking photos, and take into account both effects and performance.

[0126] In a possible implementation of the first aspect, when the number of times the background waits for taking photos is greater than N, processing the original image using a third photo-taking algorithm to obtain a corresponding captured image includes:

[0127] When the number of times the background waits for taking photos is greater than N, the single-frame algorithm is determined to be the third photo-taking algorithm, and the single-frame algorithm is used to process one frame of the original image to obtain a corresponding captured image.

[0128] In this implementation, when the present application determines that the number of photos waiting in the background is relatively large, which can also be called determining that it is in a super continuous photography state, it can use a single-frame algorithm with the simplest processing steps and the shortest time to process the minimum number of original images, thereby maximizing the processing efficiency, reducing the number of photos waiting in the background, and avoiding the problem of frame loss.

[0129] In a possible implementation manner of the first aspect, the third front-end processing to the twelfth front-end processing respectively include at least one of bad pixel correction, RAW domain noise reduction, black level correction, lens shading correction, and automatic white balance.

[0130] In this implementation, the third to twelfth front-end processes may all be the same as the first front-end process.

[0131] In a possible implementation manner of the first aspect, the first back-end processing to the tenth back-end processing respectively include: at least one of: distortion correction, YUV domain noise reduction, brightness adjustment, color processing, and image scaling.

[0132] In this implementation, the captured images obtained after the first back-end processing to the tenth back-end processing have improved clarity compared to the original images, and have better image quality.

[0133] In a second aspect, an image processing apparatus is provided, which includes a unit for executing each step in the above third aspect or any possible implementation of the third aspect.

[0134] In a third aspect, an electronic device is provided, comprising a processor and a memory; the memory is used to store a computer program that can be run on the processor; and the processor is used to execute the image processing method provided in the first aspect or any possible implementation of the first aspect.

[0135] In a fourth aspect, a chip is provided, comprising: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the image processing method provided in the first aspect or any possible implementation of the first aspect.

[0136] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the image processing method provided in the first aspect or any possible implementation of the first aspect.

[0137] In a sixth aspect, a computer program product is provided, which includes a computer-readable storage medium storing a computer program, and the computer program enables a computer to execute the image processing method provided in the first aspect or any possible implementation of the first aspect.

[0138] The image processing method and related device provided by the embodiment of the present application, in response to the user's first operation on the first control, the electronic device uses the camera to capture an original image once, then determines the number of times the background waits for photos, and calls different algorithms for processing according to the number of times the background waits for photos. For example, when the number of times the background waits for photos is large, the third photo algorithm with a relatively short time consumption is called for processing; when the number of times the background waits for photos is small, the first photo algorithm with a relatively long time consumption is called for processing; when the number of times the background waits for photos is medium, the second photo algorithm with a relatively medium time consumption is called for processing. Thus, by dynamically calling different algorithms, the running time of the photo algorithm can be coordinated, the success rate of the film can be improved, and both effect and performance can be taken into account.

[0139] In addition, in response to the user's second operation on the second control, the electronic device uses the camera to collect multiple original images, and uses the fourth photo-taking algorithm with shorter processing time to process them, thereby improving processing efficiency and achieving "unlimited shooting". As a result, there will be no frame loss problem when taking pictures under any circumstances. BRIEF DESCRIPTION OF THE DRAWINGS

[0140] FIG1 is a schematic diagram of a display interface provided in an embodiment of the present application;

[0141] FIG2 is a schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0142] FIG3 is a schematic diagram of an interface for a click operation provided in an embodiment of the present application;

[0143] FIG4 is a schematic diagram of an image processing process provided by an embodiment of the present application;

[0144] FIG5 is a schematic diagram of an interface for a long press operation provided in an embodiment of the present application;

[0145] FIG6 is a table provided in an embodiment of the present application;

[0146] FIG7 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0147] FIG8 is a schematic diagram of a flow chart of a first photographing algorithm provided in an embodiment of the present application;

[0148] FIG9 is a schematic diagram of a front-end processing flow according to an embodiment of the present application;

[0149] FIG10 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0150] FIG11 is a schematic diagram of a first backend processing flow provided in an embodiment of the present application;

[0151] FIG12 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0152] FIG13 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0153] FIG14 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0154] FIG15 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0155] FIG16 is a flow chart of another image processing method provided in an embodiment of the present application;

[0156] FIG17 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0157] FIG18 is a schematic diagram of a flow chart of a second photographing algorithm provided in an embodiment of the present application;

[0158] FIG19 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0159] FIG20 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0160] FIG21 is a schematic flow chart of another image processing method provided in an embodiment of the present application;

[0161] FIG22 is a schematic diagram of a flow chart of a third photographing algorithm provided in an embodiment of the present application;

[0162] FIG23 is a schematic diagram of a flow chart of a fourth photographing algorithm provided in an embodiment of the present application;

[0163] FIG24 is another schematic diagram of a photographing process according to an embodiment of the present application;

[0164] FIG25 is another schematic diagram of a photographing process according to an embodiment of the present application;

[0165] FIG26 is a schematic diagram of a display interface provided in an embodiment of the present application;

[0166] FIG27 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0167] FIG28 is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0168] FIG29 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0169] Figure 30 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0170] The technical solution in this application will be described below with reference to the accompanying drawings.

[0171] In the description of the embodiments of this application, unless otherwise specified, " / " represents or. For example, A / B can represent A or B. "And / or" in this article is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" means two or more than two.

[0172] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0173] First, some of the terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0174] 1. The RGB (red, green, blue) color space refers to a color model related to the structure of the human visual system. Based on the structure of the human eye, all colors are considered to be different combinations of red, green, and blue.

[0175] 2. YUV color space refers to a color encoding method, where Y represents brightness, and U and V represent chrominance. While the RGB color space focuses on the human eye's perception of color, the YUV color space emphasizes the visual sensitivity to brightness. RGB and YUV color spaces are convertible.

[0176] 3. Pixel value refers to the set of color components corresponding to each pixel in a color image in the RGB color space. For example, each pixel corresponds to a set of three primary color components, where the three primary color components are red (R), green (G), and blue (B).

[0177] 4. Bayer pattern color filter array (CFA). When an image is converted from an actual scene into image data, the image sensor usually receives the red channel signal, the green channel signal, and the blue channel signal respectively, and then synthesizes the information of the three channel signals into a color image. However, this solution requires three filters at each pixel position, which is expensive and difficult to manufacture. Therefore, a layer of color filter array can be covered on the surface of the image sensor to obtain information of the three channel signals. The Bayer pattern color filter array refers to the arrangement of filters in a checkerboard pattern. For example, the minimum repeating unit in the Bayer pattern color filter array is: a filter for obtaining a red channel signal, two filters for obtaining a green channel signal, and a filter for obtaining a blue channel signal arranged in a 2×2 manner.

[0178] 5. Bayer format image, that is, the image output by an image sensor based on a Bayer format color filter array. The pixels of multiple colors in the image are arranged in a Bayer format. Among them, each pixel in the Bayer format image corresponds to a channel signal of only one color. For example, since human vision is more sensitive to green, it can be set that green pixels (pixels corresponding to green channel signals) account for 50% of all pixels, blue pixels (pixels corresponding to blue channel signals) and red pixels (pixels corresponding to red channel signals) each account for 25% of all pixels. Among them, the minimum repeating unit of the Bayer format image is: one red pixel, two green pixels and one blue pixel arranged in a 2×2 manner. The Bayer format image is an image in the RAW domain.

[0179] 6. Shooting Parameters

[0180] Shooting parameters include shutter speed, exposure time, aperture value (AV), exposure value (EV), and ISO. These are described below.

[0181] The shutter controls the amount of time light enters the camera, thereby determining the image exposure time. The longer the shutter remains open, the more light enters the camera, and the corresponding image exposure time becomes longer. Conversely, the shorter the shutter remains open, the less light enters the camera, and the corresponding image exposure time becomes shorter.

[0182] Exposure time refers to the amount of time the shutter remains open to allow light to reach the camera's photosensitive surface. Exposure time is determined by the sensitivity of the photosensitive material and the illumination on the photosensitive surface. Longer exposure times allow more light to enter the camera, while shorter exposure times reduce the amount of light entering. Therefore, long exposure times are required in low-light scenes, while short exposure times are required in backlit scenes.

[0183] The aperture value (f-number) is the ratio of the focal length of the camera lens to the diameter of the lens' aperture. The larger the aperture value, the more light enters the camera. The smaller the aperture value, the less light enters the camera.

[0184] The exposure value is a combination of exposure time and aperture value to represent the light transmission capacity of the camera lens. The exposure value can be defined as:

[0185] Where N is the aperture value; t is the exposure time in seconds.

[0186] ISO is a measure of a film's sensitivity to light, also known as its speed or gain. Insensitive film requires a longer exposure time to achieve an image with the same brightness as a more sensitive film. Sensitive film requires a shorter exposure time to achieve an image with the same brightness as an insensitive film.

[0187] Among the shooting parameters, shutter, exposure time, aperture value, exposure value and ISO, electronic devices can use algorithms to implement at least one of auto focus (AF), automatic exposure (AE), and auto white balance (AWB) to achieve automatic adjustment of these shooting parameters.

[0188] Autofocus involves electronic devices adjusting the focus lens position to obtain the highest image frequency components, thereby achieving higher image contrast. Focusing is a cumulative process, where the electronic device compares the contrast of images captured with the lens in different positions to determine the lens position that maximizes image contrast, and thus determines the focal length for focus.

[0189] Automatic exposure refers to electronic devices automatically setting the exposure value based on available lighting conditions. Electronic devices can automatically set the shutter speed and aperture value based on the exposure value of the currently captured image to achieve automatic exposure setting.

[0190] The color of objects changes depending on the color of the light projected onto them. Images captured by electronic devices under different lighting colors will have different color temperatures. White balance is closely related to ambient light. Regardless of ambient lighting conditions, electronic device cameras can detect white and use it as a reference to reproduce other colors. Automatic white balance allows electronic devices to adjust the color fidelity of images based on lighting conditions. The 3A functions are autofocus, auto exposure, and auto white balance.

[0191] Exemplarily, the exposure value may be any one of -24, -4, -3, -2, -1, 0, 1, 2, 3, 4, and 24.

[0192] The exposure image corresponding to EV0 is used to indicate the exposure image captured by the electronic device using an algorithm to achieve exposure using an exposure value of 0. The exposure image corresponding to EV-2 is used to indicate the exposure image captured by the electronic device using an algorithm to achieve exposure using an exposure value of -2. The exposure image corresponding to EV1 is used to indicate the exposure image captured by the electronic device using an algorithm to achieve exposure using an exposure value of 1. The same applies to other images and will not be repeated here.

[0193] Each increase in the exposure value by 1 changes the exposure by one stop, essentially doubling the exposure (the integral of the light intensity received by a certain element on the surface of an object over time t). For example, doubling the exposure time or the aperture area. This increase in exposure corresponds to a slower shutter speed and a smaller f-number. Thus, an increase of EV0 relative to EV-2 represents an increase of 2, a change of two stops. Similarly, an increase of EV1 relative to EV0 represents an increase of 1, a change of one stop.

[0194] Here, when the exposure value EV is equal to 0, it is generally the optimal exposure value under the current lighting conditions. Accordingly, the exposure image obtained by the electronic device under the condition of EV0 is the optimal exposure image under the current lighting conditions. This optimal exposure image can also be called the reference exposure image.

[0195] It should be understood that the “optimal” exposure image refers to the exposure image determined by a given electronic device through an algorithm. When the electronic device is different, the algorithm is different, or the current lighting conditions are different, the determined optimal exposure image will be different.

[0196] The above is a brief introduction to the nouns involved in the embodiments of this application, and no further details will be given below.

[0197] With the popularity of electronic devices with camera functions in our lives, taking photos with electronic devices has become a daily behavior.

[0198] Nowadays, people's requirements for electronic devices are becoming increasingly demanding when taking photos. Simply relying on the raw images captured by the camera on the electronic device can no longer meet people's needs. Different photography algorithms are needed to process different raw images to obtain images that meet different user needs. The raw image refers to the image output by the image sensor in the camera.

[0199] Currently, most electronic devices use camera parameters to determine a photography algorithm when taking photos. The algorithm then processes and generates a captured image based on the determined algorithm. This captured image is then used for display or storage. Camera parameters commonly used to determine the photography algorithm include brightness, dynamic range, focal length, and object distance. However, in the prior art, the photography algorithms determined by these parameters often take a long time to run, or even run quickly but produce poor quality images, failing to strike a balance between quality and performance.

[0200] Furthermore, when an electronic device uses a photo-taking algorithm to process photos, it can only process the next photo after the previous one has completed. Based on this processing, if a user triggers multiple photo-taking commands within a short period of time, the electronic device will have to wait for more photos in the background because the photo-taking algorithm takes longer to process each time. Consequently, if the memory is full, the data waiting for processing may be lost, which can lead to frame loss. As a result, the actual number of frames captured is less than the number of frames the user intended, and the electronic device is effectively unable to achieve the continuous photo-taking function.

[0201] In view of this, an embodiment of the present application provides an image processing method. In response to the user's first operation on the first control, the electronic device uses a camera to capture an original image once. Then, by determining the number of times the background waits for photos, and according to the number of times the background waits for photos, different photo algorithms are called for processing. For example, when the number of times the background waits for photos is large, a photo algorithm with a relatively short time consumption is called for processing; when the number of times the background waits for photos is large, a photo algorithm with a relatively long time consumption is called for processing. Thus, by dynamically calling different algorithms, the running time of the photo algorithm is coordinated, the filming rate is improved, and both effect and performance are taken into account.

[0202] The image processing method provided in the embodiments of the present application can be applied to various electronic devices.

[0203] In some embodiments of the present application, the electronic device may be various camera devices such as sports cameras and digital cameras, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc., or may be other devices or apparatuses capable of displaying images. The embodiments of the present application do not impose any restrictions on the specific type of electronic device.

[0204] The following description uses a mobile phone as an example electronic device. The mobile phone includes a display and one or more cameras. The mobile phone can use one or more cameras to capture a scene, thereby obtaining images with different characteristics. The display is used to display the captured images.

[0205] The "to-be-captured scene" refers to the scene that the user desires to capture. If the user uses the camera of an electronic device to point at a scene containing an object, the scene containing the object becomes the to-be-captured scene. It should be understood that the to-be-captured scene does not refer to a specific scene, but rather to the scene that the camera is pointing at in real time.

[0206] FIG1 shows a schematic diagram of an interface of an electronic device provided in an embodiment of the present application.

[0207] As shown in FIG1(a), the electronic device 100 has a camera application installed. In addition, multiple applications are installed, and the present embodiment does not impose any restrictions on this. For example, in response to a user's touch operation, when the electronic device 100 runs the camera application, the electronic device 100 displays a shooting interface as shown in FIG1(b).

[0208] The shooting interface includes multiple shooting modes of the camera application, such as large aperture mode 41, night scene mode 42, portrait mode 43, photo mode 44, video mode 45, etc. The shooting interface also includes a first control, which is a shooting key 50. The shooting key 50 is used to indicate the current shooting mode. For example, when the camera is turned on by default, the shooting key 50 indicates that the current shooting mode is photo mode 44.

[0209] As shown in FIG1(b), the shooting interface also includes a viewfinder 60, which can be used to display a real-time preview image before taking a photo or recording a video. The shooting interface also displays a second control, which is a zoom option 61. In zoom option 61, the user can select the desired zoom factor, for example, 0.5x, 2x, or 50x.

[0210] As shown in (a) and (b) in Figure 1, when the user turns on the camera and the current shooting mode defaults to the photo mode, after the user points the camera on the mobile phone at the scene to be photographed, the viewfinder 60 can display a preview image before taking the photo in real time; in response to the user's touch operation on the shooting key 50, the electronic device can call the image processing method provided by this application, and use different photo algorithms for processing according to different situations, thereby reducing the running time of the photo algorithm, improving the filming rate, and taking into account both effect and performance.

[0211] Based on this, FIG2 shows a flow chart of an image processing method provided in an embodiment of the present application.

[0212] As shown in FIG. 2 , the method 10 is applied to an electronic device including a display screen and a camera. The method 10 may include the following S11 to S20 .

[0213] S11. The electronic device displays a first interface, where the first interface includes a preview image and a first control, and the first control indicates a photo taking mode.

[0214] Exemplarily, as shown in (b) of FIG1 , the electronic device displays a preview interface, and the first control included in the preview interface is a shooting key 50 , which indicates that the current shooting mode is a photo mode, that is, the user can take photos using the photo mode.

[0215] S12: When a first operation of the user on the first control is detected, the electronic device uses a camera to capture an original image in response to the first operation.

[0216] Optionally, the first operation refers to a click operation on the first control. The first operation may also include a voice instruction operation, or other operations that instruct the electronic device to take a photo, which is not limited in this embodiment of the present application.

[0217] A tap operation refers to the user briefly touching and then releasing the capture button 50. Figure 3 shows a schematic diagram of a tap operation interface. As shown in (a) and (b) of Figure 3, a tap operation is recorded as a finger pressing and then lifting the capture button 50. The time from the finger pressing the capture button 50 to the finger lifting the capture button 50 is the duration of a tap operation, which is usually very short.

[0218] For example, when the camera is turned on, the shooting key 50 indicates that the current shooting mode is the photo mode 44. When the user clicks the shooting key 50 once, in response to the click operation, the electronic device uses the camera to capture an original image.

[0219] When a user clicks the capture button 50 multiple times, the electronic device can capture multiple original images using the camera in response to the multiple clicks. It should be understood that when a user clicks the capture button 50 multiple times, the first click corresponds to the first photo capture, and the current cumulative photo capture count is 1; the second click corresponds to the second photo capture, and the current cumulative photo capture count is 2; and so on, the tenth click corresponds to the tenth photo capture, and the current cumulative photo capture count is 10; and the most recent click corresponds to the most recent cumulative photo capture count.

[0220] It should be noted that when the camera is turned off and then reopened, the current cumulative number of photos must be restarted from 0. Alternatively, when the interval between two adjacent click operations exceeds the first preset interval and there is no data waiting to be processed in the background, the current cumulative number of photos must also be restarted from 0. The first preset interval is at least longer than the processing time of a single photo. The specific duration of the first preset interval can be set and modified as needed, and this embodiment of the application does not impose any restrictions on this.

[0221] In Example 1, when a user opens the camera for the first time and clicks the capture button 50 for the first time, the electronic device captures an original image using the camera in response to this click, and the current cumulative number of photos is 1. Then, the user turns off the camera. If the user opens the camera again and clicks the capture button 50 again, the electronic device captures an original image using the camera in response to this click. In this case, since the camera is turned on again, the current cumulative number of photos needs to be reset to 1.

[0222] Example 2, FIG4 shows a schematic diagram of an image processing process.

[0223] As shown in FIG4 , when the user opens the camera and performs the first click operation K1 on the capture button 50 , in response to the click operation K1 , the electronic device uses the camera to capture an original image, and the current cumulative number of photos taken is 1.

[0224] However, after a long period of time, the user performs the next click operation K1' on the capture button 50. In response to this click operation K1', the electronic device uses the camera to capture the original image again. Since the interval between the click operations K1 and K1' exceeds the first preset interval time, and there is no other data waiting to be processed in the background, the current cumulative number of photos needs to be restarted from 0, that is, the current cumulative number of photos is recorded as 1. When the user subsequently performs a click operation K2' on the capture button, in response to this click operation K2', the electronic device uses the camera to capture the original image again. Since the interval between the click operations K2' and K1' is less than the first preset interval time, the current cumulative number of photos can be recorded as 2. When the user subsequently performs a click operation K3' on the capture button, in response to this click operation K3', the electronic device uses the camera to capture the original image again. Since the interval between the click operations K3' and K2' is less than the first preset interval time, the current cumulative number of photos can be recorded as 3. Other similar operations are not described in detail here.

[0225] It should be noted that if the interval between two adjacent click operations exceeds the first preset interval, but there is still data waiting to be processed in the background, it means that the user has performed many click operations before, and some data has not been processed in time due to the high click frequency and the slow background processing. Therefore, the current cumulative number of photos cannot be restarted from 0, and the original number of records should be maintained.

[0226] When an electronic device includes multiple cameras, one or more cameras may be used to capture a raw image in response to a single click operation, with each camera capturing one or more raw image frames. In other words, one or more raw image frames may be captured each time, and the specific number of frames captured may be determined as needed. This is not a limitation imposed by the present embodiment.

[0227] It should also be understood that the original image may be an image in the RAW domain, but when the camera is a black and white camera, the original image may be a grayscale image; when the camera is a multispectral camera, the original image may be a multispectral image including multiple color channel signals. The embodiments of the present application do not impose any restrictions on this.

[0228] S13, determining the number of times the background waits for taking photos.

[0229] The number of photos waiting in the background is equal to the current cumulative number of photos minus the number of photos that have been processed and the number of photos being processed.

[0230] It should be understood that in response to a click operation, the electronic device needs to call the camera to capture an original image, and use the relevant photo algorithm to process the original image, and then generate the corresponding captured image for storage or display. The time it takes for the electronic device to process in response to a click operation is the photo processing time or the photo processing time. Since the photo processing time is much longer than the click operation time, when the user clicks the camera button multiple times in a short period of time and triggers multiple processes, the background is single-threaded and cannot be completed in time, which will cause data congestion. In this case, the electronic device can only store part of the data first, and after the ongoing photo process in the background is completed, it calls the stored data, processes the stored data one by one, and then generates the corresponding captured image.

[0231] During this process, if there is an ongoing photo process in the background, the number of photos being processed is recorded as 1. If there is no ongoing photo process in the background, the number of photos being processed can be recorded as 0. At the same time, in the process of accumulating the current cumulative number of photos from 0 to the present, the number of photo processes that have ended is the number of photos that have been processed.

[0232] For example, if the current cumulative number of photos is 1, the number of photos waiting in the background is 0. If the current cumulative number of photos is 2, the number of photos being processed is 1, and the number of photos that have been processed is 0, then the number of photos waiting in the background can be determined to be 1. If the current cumulative number of photos is 10, the number of photos being processed is 1, and the number of photos that have been processed is 5, then the number of photos waiting in the background can be determined to be 4. The above are just a few examples. Other processes for determining the number of photos waiting in the background are similar to the above and will not be repeated here.

[0233] It should also be noted that if the current cumulative number of photos is counted again from 0, the number of photos that have been processed also needs to be counted again from 0.

[0234] S14: When the number of times the background waits for taking photos is less than or equal to M, the original image is processed using a first photo-taking algorithm to obtain a corresponding photographed image.

[0235] It should be understood that the original image refers to the original image captured by the electronic device using the camera in response to the most recent first operation. That is, the original image corresponding to the current cumulative number of photos. The original image includes one or more frames. When the original image includes multiple frames, the multiple frames can be captured by a single camera or by multiple cameras separately, and this embodiment of the application does not impose any restrictions on this.

[0236] S15. Determine whether the number of times the background waits for taking photos is less than or equal to N; M<N, and both M and N are positive integers.

[0237] S16. When the number of times the background waits for taking photos is greater than M but less than or equal to N, the original image is processed using a second photo algorithm to obtain a corresponding captured image. The processing time of the second photo algorithm is less than the processing time of the first photo algorithm.

[0238] The processing time of the first photo algorithm refers to the time it takes to process the original image using the first photo algorithm; the processing time of the second photo algorithm refers to the time it takes to process the original image using the second photo algorithm. The memory required by the second photo algorithm is smaller than that of the first photo algorithm.

[0239] S17: When the number of times the background waits for taking photos is greater than N, the original image is processed using a third photo algorithm to obtain a corresponding captured image. The processing time of the third photo algorithm is less than the processing time of the second photo algorithm.

[0240] The processing time of the third photographing algorithm refers to the time it takes to process the original image using the third photographing algorithm. The memory corresponding to the third photographing algorithm is smaller than the memory corresponding to the second photographing algorithm.

[0241] When the number of times the background waits for photos is low, it indicates that the user's click frequency is not very high, and the electronic device processes the data relatively promptly. In this case, a relatively long-term algorithm can be used to process the original image to ensure good image quality. When the number of times the background waits for photos is high, it indicates that the user's click frequency is very high, and the electronic device has a lot of data that it cannot process in time. In this case, a relatively short-term algorithm can be used to process the original image to improve processing efficiency, ensure that data related to the number of times the background waits for photos can be processed promptly, reduce waiting time, and thus avoid frame loss.

[0242] It should be understood that it can generally be considered that an algorithm that takes a relatively long time to process includes more processing steps, or processes a larger number of original image frames, and the quality of the obtained captured images is relatively better.

[0243] For example, the number of times the background waits for taking photos can be divided into two situations. For the situation where the number of times the background waits for taking photos is small, the first photo-taking algorithm which takes a relatively long time can be used for processing; for the situation where the number of times the background waits for taking photos is large, the second photo-taking algorithm which takes a relatively short time can be used for processing.

[0244] For example, as described above, the number of times the background waits for taking photos can be divided into three situations. For the situation where the number of times the background waits for taking photos is small, the first photo-taking algorithm with the relatively longest time consumption can be used for processing; for the situation where the number of times the background waits for taking photos is large, the third photo-taking algorithm with the relatively shortest time consumption can be used for processing; for the situation where the number of times the background waits for taking photos is medium, the second photo-taking algorithm with a relatively medium time consumption can be used for processing.

[0245] It should be understood that the above is only an example of dividing the number of times the background waits for taking pictures. The number of times the background waits for taking pictures can also be divided into four, five or more situations. For situations where the number of times the background waits for taking pictures increases, a photo-taking algorithm with relatively shorter time consumption is used for processing. The specific division method and photo-taking algorithm may need to be set and modified, and the embodiments of the present application do not impose any restrictions on this.

[0246] S18. When a second operation of the user on the first control is detected, the electronic device uses a camera to capture multiple original images in response to the second operation.

[0247] Optionally, the second operation refers to a long press operation on the first control. The second operation may also include a voice instruction operation or other operations that instruct the electronic device to take multiple photos continuously, which is not limited in this embodiment of the present application.

[0248] A long press refers to the user touching the capture button for an extended period of time. The time from the moment the finger is pressed down to the moment it is lifted is the duration of a long press. The duration of a long press is relatively longer than the duration of a click. The durations of click and long press settings may vary between electronic devices, but they must all be longer than the duration of a click.

[0249] For example, Figure 5 shows a schematic diagram of an interface for a long press operation. As shown in (a) in Figure 5, after the finger presses the shooting button 50, the electronic device detects that the pressing time is equal to the duration of a click operation. If the finger is not lifted, the electronic device can determine that this is a long press operation. Therefore, the camera can be instructed to capture an original image once every second preset interval, and as shown in (b) in Figure 5, the preview interface is displayed in real time from "01" in the form of scrolling numbers to display how many times the camera has been instructed to capture the original image.

[0250] As shown in Figure 5(c), if the number on the preview screen scrolls to "30," the camera has been instructed to capture 30 raw images. As shown in Figure 5(d), if the user's finger immediately lifts off the capture button 50 after the number on the preview screen shows "30," the entire process from pressing the capture button 50 to lifting it off is counted as a long press. The duration of the long press is the time from the finger pressing the capture button 50 to the finger lifting it off. During this time, the electronic device instructed the camera to capture a total of 30 raw images.

[0251] Exemplarily, when the camera is turned on, the shooting key 50 indicates that the current shooting mode is the photo mode 44. When the user performs a long press operation on the shooting key 50, in response to the long press operation, the electronic device uses the camera to capture multiple original images.

[0252] It should be noted that the longer the user performs the long press operation, the more times the electronic device uses the camera to capture the original image. In addition, the second preset interval between two adjacent times can be set and modified as needed, and the embodiment of the present application does not impose any restrictions on this.

[0253] S19: Process the original image collected each time using the fourth photographing algorithm to obtain corresponding photographed images.

[0254] Among them, the processing time of the fourth photographing algorithm is less than the processing time of the second photographing algorithm.

[0255] The processing time of the fourth photographing algorithm refers to the time it takes to process the original image using the fourth photographing algorithm. The memory corresponding to the fourth photographing algorithm is smaller than the memory corresponding to the second photographing algorithm.

[0256] It should be understood that when a user performs a long-press operation, it indicates that the user wishes to take multiple photos in succession. At this time, the electronic device captures raw images at a high frequency. If the background uses a relatively time-consuming algorithm to process each captured raw image, it will inevitably be unable to process all the captured raw images, resulting in data congestion. Therefore, when the user performs a long-press operation, it is necessary to use a relatively time-consuming algorithm to process each captured raw image to improve processing efficiency.

[0257] Optionally, the fourth photographing algorithm may be the same as the third photographing algorithm. The processing time of the fourth photographing algorithm is equal to the processing time of the third photographing algorithm. The memory corresponding to the fourth photographing algorithm is equal to the memory corresponding to the third photographing algorithm.

[0258] Since the processing time of the third photographing algorithm is less than that of the second photographing algorithm, when the fourth photographing algorithm is the same as the third photographing algorithm, the processing time of the fourth photographing algorithm is also less than that of the second photographing algorithm.

[0259] Optionally, the fourth photographing algorithm may be different from the third photographing algorithm. The processing time of the fourth photographing algorithm is less than the processing time of the third photographing algorithm. The memory corresponding to the fourth photographing algorithm is less than the memory corresponding to the third photographing algorithm.

[0260] For example, the fourth photographing algorithm may include fewer processing steps than the third photographing algorithm. In this case, the processing time of the fourth photographing algorithm is less than the processing time of the third photographing algorithm.

[0261] It should be understood that since the processing time of the third photographing algorithm is less than that of the second photographing algorithm, the processing time of the fourth photographing algorithm is less than that of the third photographing algorithm, so the processing time of the fourth photographing algorithm is still less than that of the second photographing algorithm.

[0262] For example, as shown in Figure 6, the processing time corresponding to the first photo algorithm is 2000ms, and the memory usage is 87.5M; the processing time corresponding to the second photo algorithm is 1200ms, and the memory usage is 50M; the processing time corresponding to the third photo algorithm is 500ms, and the memory usage is 25M; the processing time corresponding to the fourth photo algorithm is 1200ms, and the memory usage is 12.5M.

[0263] It should be understood that the above is only an example, and the processing time and memory usage corresponding to each photo-taking algorithm can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0264] S20: Save the captured image.

[0265] The captured images can be stored in the gallery or displayed.

[0266] In response to a single click, one frame of image can be captured. In response to multiple clicks, multiple frames of image can be captured. In response to a long press, multiple frames of image can be captured. The specific number of captured images depends on the user's operation and is not limited in this embodiment of the application.

[0267] The embodiment of the present application provides an image processing method. In response to a user's first operation on a first control, the electronic device uses a camera to capture an original image. Then, by determining the number of times the background waits for photos, different algorithms are called for processing based on the number of times the background waits for photos. For example, when the number of times the background waits for photos is large, a third photo algorithm with a relatively short processing time is called for processing; when the number of times the background waits for photos is small, a first photo algorithm with a relatively long processing time is called for processing; and when the number of times the background waits for photos is medium, a second photo algorithm with a relatively medium processing time is called for processing. Thus, by dynamically calling different algorithms, the running time of the photo algorithms can be coordinated, the success rate of the photos can be improved, and both the effect and performance can be taken into account.

[0268] In addition, in response to the user's second operation on the second control, the electronic device uses the camera to collect multiple original images, and uses the fourth photo-taking algorithm with shorter processing time to process them, thereby improving processing efficiency and achieving "unlimited shooting". As a result, there will be no frame loss problem when taking pictures under any circumstances.

[0269] On the basis of the above, since the information contained in the scene to be photographed is relatively diverse, when the number of times waiting for photographing in the background is small, the scene to be photographed can be detected and analyzed, further classified, and then different photographing algorithms can be used to process different scenes to improve the quality of the captured images.

[0270] Optionally, as shown in FIG7 , before S14 , after determining that the number of times the background waits for taking photos is less than or equal to M, the method further includes the following S21 and S22 .

[0271] S21: Detect the scene to be photographed and obtain the detection result.

[0272] The detection includes illumination value (LV) detection and dynamic range detection. In addition, the detection also includes at least one of artificial intelligence (AI) scene recognition, motion detection, face detection, and face attribute detection.

[0273] It should be understood that illumination detection refers to estimating the ambient brightness in the scene to be shot. Illumination refers to the degree to which an object is illuminated, specifically refers to the energy of visible light received per unit area of ​​the object, referred to as illumination, and the unit is lux. Streets at night, dimly lit rooms, etc. are generally low illumination. In contrast, playgrounds under the sun, stages under spotlights, etc. can be called medium and high illumination. If the illumination of the scene to be shot is relatively low, when using an electronic device to shoot the scene, it will be blurry and the effect will be poor. If the illumination of the scene to be shot is relatively high, when using an electronic device to shoot the scene, it will be clearer and the shooting effect will be good.

[0274] Here, for example, a photoelectric sensor can be set in the electronic device to detect the illumination of the scene to be captured. Alternatively, the illumination can be calculated using the following formula based on the camera's exposure parameters, such as exposure time, sensitivity, aperture, and other parameters, as well as the received response value:

[0275] Among them, Exposure is the exposure time, Aperture is the aperture size, ISO is the sensitivity, and Luma is the average value of Y in the XYZ color space.

[0276] When the exposure parameters are the same, a higher response value indicates a higher illumination of the scene to be photographed, and thus a larger value of the calculated illumination is also obtained.

[0277] It should be understood that dynamic range detection refers to the detection of the dynamic range value of the scene to be captured. The dynamic range value is used to represent the brightness range of the pixels in the image, that is, the number of grayscale levels between the "brightest" pixel and the "darkest" pixel in the image. The larger the dynamic range value of an image, the richer the brightness levels it can represent and the more realistic the visual effect of the image. The expression of the dynamic range value can be:

[0278] Where dynamic range is the dynamic range value, bright is the brightness of the "brightest" pixel, and dark is the brightness of the "darkest" pixel. The unit of dynamic range is stop.

[0279] Here, based on the detected dynamic range value, it can be determined whether the scene to be captured is a low-light scene or a high dynamic range (HDR) scene. A low-light scene refers to a scene with low ambient light intensity, while a high dynamic range scene refers to a scene with relatively strong ambient light, or a scene with low light intensity in some parts and high light intensity in other parts.

[0280] AI scene recognition is used to detect whether the scene to be photographed contains a preset photographic object, such as: landscape, portrait, text, sky, etc. The preset photographic object can be set and modified as needed, and the embodiment of the present application does not impose any restrictions on this.

[0281] Here, a scene recognition model can be set in the electronic device, and the electronic device can use the scene recognition model to identify the scenery, portraits, text, etc. in the scene to be photographed. That is, AI scene recognition can be detected using a scene recognition model, and the scene recognition model can be a recognition model based on a convolutional neural network. The embodiment of the present application does not limit the specific construction process and specific training process of the scene recognition model. It can be constructed using the existing construction process, or it can be trained using the existing training process. It should be understood that the scene recognition model set in the electronic device can be trained by a cloud server, and can also be updated in real time according to the update operation of the cloud server.

[0282] Motion detection is used to detect whether people or objects in the scene being captured are moving, as well as the speed of the movement. For example, people are running, children are chasing, pets are running, etc.

[0283] Here, the electronic device may be provided with an acceleration sensor and / or a gyroscope sensor. The electronic device may use the acceleration sensor and / or the gyroscope sensor to assist in identifying whether a person or object in the scene to be captured is moving, and the speed of the movement. The unit of the movement speed may be meters per second (m / s) or centimeters per second (cm / s), which may be set as needed, and the embodiments of the present application are not limited thereto.

[0284] Face detection is used to determine whether a scene contains any human faces. Face attribute detection is used to estimate the number, gender, and age of detected faces.

[0285] Here, a face detection model can be set in the electronic device. The electronic device can use the face detection model to identify the number of people, gender, age, etc. in the scene to be photographed. That is, face detection and face attribute detection can be detected using a face recognition model. The face recognition model can be a recognition model constructed based on a convolutional neural network. The embodiment of the present application does not limit the specific construction process and specific training process of the face recognition model. It can be constructed using the existing construction process or trained using the existing training process. It should be understood that the face recognition model set in the electronic device can be trained by a cloud server and can also be updated in real time according to the update operation of the cloud server.

[0286] It should be understood that the detection may include one or more of the above-mentioned detection items. Of course, other detection items may also be included, which can be added as needed. The embodiments of the present application do not impose any restrictions on this.

[0287] It should be noted that when performing detection, the execution order of multiple detections can be set as needed. For example, illumination detection, dynamic range detection, AI scene recognition, motion detection, face detection and facial attribute detection can be performed in sequence, or illumination detection, dynamic range detection, AI scene recognition, motion detection, face detection and facial attribute detection can be performed simultaneously; of course, they can also be executed in other orders, and the embodiments of the present application do not impose any restrictions on this.

[0288] S22: Determine a first photographing algorithm based on the detection result.

[0289] According to the detection results, the preset shooting objects in the scene to be shot can be divided into static, slow-motion and fast-motion states. For different states, different algorithms are used as the first shooting algorithm to process the original image.

[0290] The embodiment of the present application detects the scene to be shot, refines the conditions in the scene, and accordingly refines the method for processing the original image, thereby adaptively improving the quality and effect of the shot image obtained in each case.

[0291] The image processing method provided by the present application is described in detail below with reference to the embodiments. In the following embodiments, M is 3 and N is 10 as an example.

[0292] It should be understood that the various numbers are for illustration only and can be set and modified as needed. This embodiment of the present application does not display any of this.

[0293] Embodiment 1, an image processing method, as shown in FIG8 , may include the following steps S101 to S117 .

[0294] S101: The electronic device displays a first interface, where the first interface includes a preview image and a shooting button, and the shooting button indicates a photo taking mode.

[0295] S102: When a click operation of a user on a capture button is detected, the electronic device uses a camera to capture an original image in response to the click operation.

[0296] For example, for one click operation, the camera collects 10 frames of original images. It should be understood that the number of frames included in the original image collected by the electronic device using the camera can be set and modified as needed, and the embodiment of the present application does not impose any limitation on this.

[0297] S103: Determine the number of times the background waits for taking photos.

[0298] For example, after the user opens the camera, the current cumulative number of photos starts at 0. When the user clicks the capture button A1, the electronic device uses the camera to capture an original image in response to the click A1, and the current cumulative number of photos is 1. Since there are no completed photos or photos being processed at this time, the number of photos waiting in the background is deducted from the number of photos being processed by the current cumulative number of photos, which is 1.

[0299] S104: Since the number of times of waiting for photos in the background is 1, which is less than 3, the number of photos in the background is small. Therefore, the scene to be photographed can be detected first to obtain the detection result.

[0300] Here, illumination detection, dynamic range detection, AI scene recognition, motion detection, face detection, and face attribute detection can be performed in a certain order; or, multiple detections can be performed simultaneously. The order of detection can be set and modified as needed, and this embodiment of the application does not impose any restrictions on this.

[0301] In addition to the above methods, multiple detection algorithms and models such as illumination detection, dynamic range detection, AI scene recognition, motion detection, face detection, and facial attribute detection can also be integrated into the perception engine. Then, when the perception engine is used to detect the scene to be photographed, all relevant detection results can be obtained at once in a relatively convenient and rapid manner. Of course, the specific types and methods of integration can be set and modified as needed, and the embodiments of this application do not impose any restrictions on this.

[0302] Optionally, before detection, the original image may be subjected to front-end processing to make some corrections to the original image.

[0303] FIG9 shows a schematic diagram of a front-end processing.

[0304] As shown in FIG9 , the front-end processing may include a first front-end processing and a second front-end processing.

[0305] The first front-end processing may include: at least one of: default pixel correction (DPC), RAW domain noise reduction (RAWNF), black level correction (BLC), lens shading correction (LSC), and automatic white balance; the second front-end processing may include: at least one of: color interpolation, color correction matrix (CCM), global tone mapping, gamma correction, and color space conversion.

[0306] Bad pixel correction: Bad pixels are white dots in the output image in a completely dark environment, or black dots in the output image in a bright environment. Generally, the three primary color channel signals should respond linearly to the ambient brightness. However, due to poor signal output from the image sensor, white or black dots may appear. To address this, we can automatically detect and repair bad pixels, or create a bad pixel list to repair bad pixels at fixed locations. A dot refers to a pixel.

[0307] Noise reduction refers to the process of reducing noise in an image. Common methods include mean filtering, Gaussian filtering, and bilateral filtering. RAW-domain noise reduction refers to the process of reducing noise in RAW-domain images.

[0308] Black level correction is due to the presence of dark current in the image sensor, which causes the pixels to have a certain output voltage even when there is no light. In addition, pixels at different positions may correspond to different output voltages. Therefore, it is necessary to correct the output voltage corresponding to the pixels when there is no light (i.e., black).

[0309] Lens shading correction can solve the problem of shadows appearing around the lens due to uneven refraction of light by the lens.

[0310] Automatic white balance is designed to eliminate the impact of light sources on image sensor imaging, simulate the color constancy of human vision, and ensure that the white seen in any scene is truly white. Therefore, it is necessary to correct the color temperature and automatically adjust the white balance to the appropriate position.

[0311] Color interpolation: Since each pixel in a RAW image corresponds to the color information of only one channel, information from surrounding pixels can be used to estimate the other colors. For example, through linear interpolation, the color information of the other two channels missing for each pixel can be determined, thereby restoring all channel information for all pixels in the image. The process of color interpolation is equivalent to converting the image from the RAW domain to the RGB domain. In this context, color interpolation can also be called demosaicing.

[0312] Color correction is necessary because the image captured by the camera may not match the expected color. Since the automatic white balance has already calibrated the white color, color correction can be used to calibrate other colors besides white.

[0313] Global tone mapping involves mapping all pixels in an image using a spatially invariant mapping function. This means that when performing a dynamic range transformation on an image, each pixel in the image uses the same transformation function, resulting in a one-to-one mapping. Global tone mapping is also referred to as dynamic range compression (DRC).

[0314] Gamma correction refers to editing the gamma curve of an image in a nonlinear tone editing manner, detecting the dark and light parts of the image and increasing them proportionally, thereby improving the image contrast.

[0315] Color space conversion involves converting an image from the RGB domain to the YUV domain. This allows the image processed in the previous step to be converted from the RGB domain to the YUV domain, reducing the amount of data subsequently stored and transmitted, saving bandwidth.

[0316] In the embodiment of the present application, the image after the first front-end processing is in the RAW domain, and the image after the second front-end processing is in the YUV domain. The image in the RAW domain contains more information, and the image in the YUV domain corresponds to less data and is faster to transmit.

[0317] The first front-end processing and / or the second front-end processing may include one or more of the aforementioned processing steps. When the first front-end processing and / or the second front-end processing include multiple processing steps, the processing order of the multiple processing steps may be adjusted as needed, and the embodiments of the present application do not impose any restrictions on this. In addition, the first front-end processing and the second front-end processing may also include other steps, which may be added and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0318] It should be understood that the image in the YUV domain after the second front-end processing has undergone some corrections and adjustments compared to the original image, and the information in the image is more accurate. Therefore, the image in the YUV domain can be input into the perception engine for detection, thereby obtaining a correspondingly more accurate detection result. This detection result is the detection result corresponding to the scene to be captured.

[0319] S105: Determine, based on the detection result, whether the preset shooting object in the scene to be shot is static.

[0320] Static refers to a scene where the subject is stationary. Examples include landscapes, standing figures, text, and the sky. The opposite of static is dynamic, which refers to a scene where the subject is moving. Examples include a running child and a thrown basketball.

[0321] It should be understood that the preset shooting objects can be added and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0322] S106: When it is determined that the preset shooting object in the scene to be shot is static, it is determined whether the detected illumination in the scene to be shot is greater than a first illumination threshold (eg, 100 Lux).

[0323] It should be understood that the size of the first illumination threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0324] S107: If it is detected that the illumination in the scene to be photographed is less than or equal to the first illumination threshold, determine the first night scene algorithm as the first photographing algorithm, and use the first night scene algorithm to process the original image collected by the camera to obtain a corresponding photographed image.

[0325] When the illumination of the scene to be captured is detected to be less than or equal to the first illumination threshold, it indicates that the ambient brightness of the scene to be captured is relatively dark. In this case, if a general photography algorithm is used for processing, the resulting captured image will be unclear, and the user will not be able to see the details in the captured image. Therefore, in this embodiment of the application, the first night scene algorithm can be selected for processing to increase the brightness of the image during processing, so that the captured image can present more details.

[0326] Optionally, the first night scene algorithm may be used to process the P frame in the original image captured by the camera to obtain a corresponding captured image.

[0327] For example, since the camera can capture 10 frames of raw images in response to a single click, when processing using the first night scene algorithm, some or all frames of raw images can be selected for processing, and this embodiment of the application does not impose any restrictions on this. Where P should be greater than 1 but less than or equal to the total number of frames of raw images captured by the camera, that is, 10 frames; the value of P should be an integer.

[0328] Optionally, when multiple frames of original images are selected for processing, the multiple frames of original images may include at least one of long-exposure original images, normal-exposure original images, and short-exposure original images.

[0329] Among them, the long-exposure original image refers to the image obtained after a longer exposure time during shooting, and the short-exposure original image refers to the image obtained after a shorter exposure time during shooting. Among them, "long" and "short" are both relative to the "normal" exposure time. When the exposure time corresponding to the normally exposed original image is different, the long exposure and short exposure will also change accordingly.

[0330] When the multiple frames of original images include long-exposure original images, and / or normal-exposure original images, and / or short-exposure original images, the number of frames of the long-exposure original images, the number of frames of the normal-exposure original images, and the number of frames of the short-exposure original images can be selected and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0331] Optionally, a night scene algorithm module may be provided in the electronic device, and the electronic device may implement the processing of the first night scene algorithm through the night scene algorithm module.

[0332] For example, the night scene algorithm module is generated based on the Unet network model and can fuse multiple frames of original images in the RAW domain into a single frame of image in the RAW domain. Of course, the night scene algorithm module can also be generated based on other models, and the embodiments of the present application have no limitation thereto.

[0333] On this basis, the first night scene algorithm is used to process the P frame in the original image captured by the camera to obtain a corresponding captured image, as shown in FIG10 , which may include the following S1071 to S1073 .

[0334] S1071 , performing third front-end processing on the P frame in the original image captured by the camera this time to obtain a first intermediate image of the P frame.

[0335] For example, when the camera collects 10 frames of original images, assuming P=9, 9 frames can be selected to perform the third front-end processing to obtain 9 frames of first intermediate images.

[0336] The third front-end processing may include the same steps as the first front-end processing, and the details may refer to the above description. Of course, the third front-end processing may also include other steps, which are not limited in this embodiment of the present application.

[0337] S1072: Use the first night scene algorithm to process the P-frame first intermediate image to obtain a frame of second intermediate image.

[0338] The first intermediate image is an image in the RAW domain, and the second intermediate image is an image in the YUV domain.

[0339] S1073: Perform first back-end processing on one frame of the second intermediate image to obtain a corresponding captured image.

[0340] It should be understood that the first back-end processing is used to correct the second intermediate image.

[0341] FIG11 shows a schematic diagram of a first backend processing.

[0342] As shown in FIG11 , the first back-end processing may include at least one of distortion correction, YUV domain noise reduction, brightness adjustment, color processing, and image scaling.

[0343] Distortion correction: Since the camera lens has different bending abilities for colored light, the imaging point is at different distances from the optical axis, resulting in different lateral magnifications. This causes actual imaging distortion and creates distortion problems. Therefore, the image needs to be corrected to restore it to normal.

[0344] YUV domain denoising refers to the process of reducing noise in YUV domain images.

[0345] Brightness adjustment refers to increasing or decreasing the brightness of an image.

[0346] Color processing is used to make the original unsaturated color information saturated and rich. Here, color processing can also be called color enhancement (CE).

[0347] Image scaling is used to enlarge or reduce the size of an image.

[0348] In the embodiment of the present application, the image after being processed by the first back-end is in the YUV domain.

[0349] It should be understood that the first back-end processing may include one or more of the above-mentioned processing steps. When the first back-end processing includes multiple processing steps, the processing order of the multiple processing steps can be adjusted as needed, and the embodiments of the present application do not impose any restrictions on this. In addition, the first back-end processing may also include other steps, which can be added and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0350] After being processed by the first back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0351] It should be understood that the captured image obtained after the first back-end processing has improved clarity and better image quality compared to the original image.

[0352] S108: If it is detected that the illumination in the scene to be captured is greater than the first illumination threshold, then continue to determine whether the dynamic range result detected in the scene to be captured is greater than the first dynamic range threshold (for example, 8 stops).

[0353] It should be understood that the size of the first dynamic range threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0354] S109. If the detected dynamic range of the scene to be captured is less than or equal to a first dynamic range threshold, determining a first multi-frame noise reduction (MFNR) algorithm as a first photographing algorithm, and using the first MFNR algorithm to process the original image captured by the camera to obtain a corresponding captured image.

[0355] In conjunction with S105, S106, and S108, when it is detected that the preset subject in the scene to be captured is static, the illumination is greater than the first illumination threshold, and the dynamic range is less than or equal to the first dynamic range threshold, it indicates that the ambient light in the scene to be captured is strong, but the dynamic range is small. In this case, if a general photography algorithm is used for processing, the resulting captured image will have high noise and a small dynamic range. In this regard, in this embodiment of the present application, the first MFNR algorithm can be selected for processing to reduce the noise in the captured image and improve the dynamic range of the captured image.

[0356] Optionally, the first MFNR algorithm is used to process the P frame in the original image captured by the camera to obtain a corresponding captured image.

[0357] It should be understood that since the camera can capture 10 frames of raw images in response to a single click, the first MFNR algorithm can be used to process a portion of the raw images, or all of the raw images. This is not limited in this embodiment of the present application. Here, the number of frames processed by the first MFNR algorithm and the number of frames processed by the first night scene algorithm can be the same.

[0358] In the embodiment of the present application, when multiple frames of original images are selected to be processed using the first MFNR algorithm, the exposure time of the multiple frames of original images is the same.

[0359] Optionally, in an embodiment of the present application, the first MFNR algorithm may be a process of first aligning the P frames in the original image and then processing the aligned original image using a wavelet fusion algorithm; or, the first MFNR algorithm may also be an optical flow algorithm, and the embodiment of the present application has no restrictions on this.

[0360] On this basis, the first MFNR algorithm is used to process the P frame in the original image captured by the camera to obtain a corresponding captured image, as shown in FIG12 , which may include the following S1091 to S1093.

[0361] S1091: Perform fourth front-end processing on the P frame in the original image captured by the camera to obtain a third intermediate image of the P frame.

[0362] The fourth front-end processing may include the same steps as those included in the first front-end processing. For details, please refer to the above description. Of course, the fourth front-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0363] S1092: Use the first MFNR algorithm to process the third intermediate image of the P frame to obtain a fourth intermediate image of one frame.

[0364] The third intermediate image is an image in the RAW domain, and the fourth intermediate image is an image in the YUV domain.

[0365] S1093. Perform second back-end processing on one frame of the fourth intermediate image to obtain a corresponding captured image.

[0366] The second back-end processing may include the same steps as those included in the first back-end processing described above. For details, please refer to the description of Figure 11. Of course, the second back-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0367] After being processed by the second back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0368] S110: If the detected dynamic range of the scene to be photographed is greater than the first dynamic range threshold, determine the first HDR algorithm as the first photographing algorithm, and use the first HDR algorithm to process the original image captured by the camera to obtain a corresponding photographed image.

[0369] When the dynamic range of the scene to be captured is detected to be greater than the first dynamic range threshold, it indicates that the ambient light intensity in the scene to be captured is high, and the scene to be captured can be considered a high dynamic range scene. In this case, if a general photography algorithm is used for processing, the captured image may be overexposed. In this embodiment of the present application, the first HDR algorithm can be selected for processing to adjust the dynamic range and image details of the image, making the captured image more realistic and improving image quality.

[0370] Optionally, the first HDR algorithm is used to process the P frame in the original image captured by the camera to obtain a corresponding captured image.

[0371] For example, in response to a single click, the camera may capture 10 frames of original images. When processing using the first HDR algorithm, a portion of the original images may be selected for processing, or all of the original images may be selected for processing. This embodiment of the present application does not impose any restrictions on this. Here, the number of frames processed using the first HDR algorithm may be the same as the number of frames processed using the first night scene algorithm.

[0372] In an embodiment of the present application, when multiple frames of original images are selected for processing using the first HDR algorithm, the multiple frames of original images may correspond to different exposure values. For example, some of the multiple frames of original images may correspond to EV-1, some may correspond to EV0, and some may correspond to EV1.

[0373] Optionally, an HDR algorithm module may be provided in the electronic device, and the electronic device may implement the processing of the first HDR algorithm through the HDR algorithm module.

[0374] For example, the first HDR algorithm module is generated based on multi-exposure fusion processing and a tone mapping model. It can fuse multiple frames of original images with different exposure values ​​or exposure levels into a single image in the RAW domain and perform color mapping and transformation processing on the image. The tone mapping model can be any one of a Unet network model, a Resnet network model, and an Hdrnet network model. Of course, the tone mapping model can also be other models, and the embodiments of the present application do not impose any restrictions on this.

[0375] On this basis, the first HDR algorithm is used to process the P frame in the original image captured by the camera to obtain a corresponding captured image, as shown in FIG13 , which may include the following S1101 to S1103 .

[0376] S1101 , performing fifth front-end processing on a P frame in the original image captured by the camera this time to obtain a fifth intermediate image of the P frame.

[0377] The fifth front-end processing can be the same as the steps included in the above-mentioned first front-end processing. Please refer to the above description for details. Currently, the fifth front-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0378] S1102 : Using a first HDR algorithm, process the fifth intermediate image of the P frame to obtain a sixth intermediate image of the P frame.

[0379] The fifth intermediate image is an image in the RAW domain, and the sixth intermediate image is an image in the YUV domain.

[0380] S1103 : Perform third back-end processing on one frame of the sixth intermediate image to obtain a corresponding captured image.

[0381] The third back-end processing can be the same as the steps included in the first back-end processing mentioned above. For details, please refer to the description of Figure 11. Of course, the third back-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0382] After being processed by the third back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0383] S111. After S105, when it is determined that the preset shooting object in the scene to be shot is not static, it means that the preset shooting object in the scene to be shot is dynamic. At this time, it can be further determined based on the motion speed detected by motion detection whether the motion speed of the preset shooting object is greater than a preset speed threshold (for example, 1m / s)?

[0384] It should be understood that the size of the preset speed threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0385] S112: If it is detected that the movement speed of the preset shooting object in the scene to be shot is greater than the preset speed threshold, it can be further determined based on the detected illumination result of the scene to be shot whether the illumination of the scene to be shot is greater than a second illumination threshold (e.g., 100 Lux).

[0386] It should be understood that the size of the second illumination threshold can be set and modified as needed, and the size of the second illumination threshold can be the same as or different from the size of the first illumination threshold. The embodiment of the present application does not impose any restrictions on this.

[0387] S113: If it is detected that the illumination in the scene to be photographed is less than or equal to the second illumination threshold, determine the second night scene algorithm as the first photographing algorithm, and use the second night scene algorithm to process the original image captured by the camera to obtain a corresponding photographed image.

[0388] In conjunction with S111 and S113, when it is detected that the speed of the preset subject in the scene to be captured is greater than the preset speed threshold, and the illumination in the scene to be captured is less than or equal to the second illumination threshold, it indicates that the preset subject is moving in a low-light environment, and the movement amplitude is relatively large, which can be called a fast motion state. In this case, using a general photography algorithm for processing may result in blurring or ghosting of the captured image. To address this issue, in this embodiment of the present application, a second night scene algorithm can be selected for processing to improve image clarity and ensure image quality.

[0389] Optionally, the second night scene algorithm may be used to process the Q frame in the original image captured by the camera to obtain a corresponding captured image.

[0390] It should be understood that since the camera can capture 10 frames of raw images in response to a single click, when processing using the second night scene algorithm, a portion of the raw images can be selected for processing, or all frames can be selected for processing. This embodiment of the present application does not impose any restrictions on this. Q should be greater than or equal to 1, but less than the total number of frames of raw images captured by the camera. The value of Q should be an integer, and 1≤Q<P.

[0391] For the selection of multiple frames of original images, reference may be made to the description in S107 , which will not be repeated here.

[0392] Optionally, the second night scene algorithm may be the same as the first night scene algorithm, or may be different, and this embodiment of the present application does not impose any limitation on this.

[0393] For example, the second night scene algorithm can be the same as the first night scene algorithm, both of which are AI algorithms. Noise reduction, demosaicing, dynamic range compression, and other processing are achieved through an AI network. The processing process is relatively complex and requires a lot of software and computing resources, such as graphics processing units (GPUs) and neural-network processing units (NPUs).

[0394] The second night scene algorithm may also be different from the first night scene algorithm. The first night scene algorithm is an AI algorithm, and the second night scene algorithm is a traditional algorithm. It completes noise reduction, de-mosaicing, dynamic range compression, etc. through the collaboration of software and hardware. The resources that need to be called include image signal processor (ISP), central processing unit (CPU), digital signal processor (DSP), etc., and the completion time is relatively shorter.

[0395] On this basis, the second night scene algorithm is used to process the Q frame in the original image collected by the camera this time to obtain the corresponding captured image, as shown in FIG14 , which may include the following S1131 to S1133 .

[0396] S1131 , performing sixth front-end processing on the Q frame in the original image captured by the camera this time to obtain a seventh intermediate image of the Q frame.

[0397] For example, when the camera collects 10 frames of original images, assuming Q=3, 3 frames can be selected to perform the sixth front-end processing to obtain 3 frames of seventh intermediate images.

[0398] The sixth front-end processing may include the same steps as the first front-end processing, and the details may refer to the above description. Of course, the sixth front-end processing may also include other steps, which are not limited in this embodiment of the present application.

[0399] S1132. Use the second night scene algorithm to process the Q-frame seventh intermediate image to obtain one-frame eighth intermediate image.

[0400] The seventh intermediate image is an image in the RAW domain, and the eighth intermediate image is an image in the YUV domain.

[0401] S1133 . Perform fourth back-end processing on one frame of the eighth intermediate image to obtain a corresponding captured image.

[0402] The fourth back-end processing is used to correct the eighth intermediate image.

[0403] The fourth back-end processing can be the same as the steps included in the above-mentioned first back-end processing. For details, please refer to the description of Figure 11. Of course, the fourth back-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0404] After being processed by the fourth back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0405] S114: If it is detected that the illumination in the scene to be captured is greater than the second illumination threshold, then continue to determine whether the dynamic range result detected in the scene to be captured is greater than the second dynamic range threshold (for example, 8 stops).

[0406] It should be understood that the size of the second dynamic range threshold can be set and modified as needed, and the size of the second dynamic range threshold can be the same as or different from the size of the first dynamic range threshold. The embodiment of the present application does not impose any restrictions on this.

[0407] S115. If the detected dynamic range of the scene to be captured is less than or equal to the second dynamic range threshold, determine the second MFNR algorithm as the first photographing algorithm, and use the second MFNR algorithm to process the original image captured by the camera to obtain a corresponding captured image.

[0408] In conjunction with S105, S111, S112, and S114, when the preset subject is dynamic, the motion speed is greater than the preset speed threshold, the illumination in the scene to be captured is greater than the second illumination threshold, but the dynamic range is less than or equal to the second dynamic range threshold, it indicates that the preset subject is moving at a relatively large amplitude and is in a fast motion state in a bright scene with a small dynamic range. In this case, if a general photography algorithm is used for processing, the resulting captured image will have high noise and a small dynamic range. To address this, in this embodiment of the present application, a second MFNR algorithm can be selected for processing to reduce noise in the captured image and improve the dynamic range of the captured image.

[0409] Optionally, a second MFNR algorithm is used to process the Q frame in the original image captured by the camera to obtain a corresponding captured image.

[0410] It should be understood that since the camera can capture 10 frames of original images in response to a single click, the second MFNR algorithm can be used to process a portion of the original images, or all of the original images. This is not limited in this embodiment of the present application. Here, the number of frames processed by the second MFNR algorithm and the number of frames processed by the second night scene algorithm can be the same.

[0411] In the embodiment of the present application, when multiple frames of original images are selected to be processed using the second MFNR algorithm, the exposure time of the multiple frames of original images is the same.

[0412] Optionally, the second MFNR algorithm may be the same as the first MFNR algorithm, or may be different. This embodiment of the present application does not impose any limitation on this.

[0413] For example, the second MFNR algorithm can be the same as the first MFNR algorithm, both being AI algorithms. Noise reduction and demosaicing are performed through an AI network. The processing is relatively complex and requires a lot of software and computing resources, such as GPUs and NPUs.

[0414] The second MFNR algorithm can also be different from the first MFNR algorithm. The first MFNR algorithm is an AI algorithm, and the second MFNR algorithm is a traditional algorithm. It completes noise reduction, demosaicing, etc. through the collaboration of software and hardware. The resources that need to be called include ISP, CPU, DSP, etc., and the completion time is relatively shorter.

[0415] On this basis, the second MFNR algorithm is used to process the Q frame in the original image captured by the camera to obtain a corresponding captured image, as shown in FIG15 , which may include the following S1151 to S1153.

[0416] S1151 , performing seventh front-end processing on the Q frame in the original image captured by the camera this time to obtain a ninth intermediate image of the Q frame.

[0417] The seventh front-end processing can be the same as the steps included in the above-mentioned first front-end processing. Please refer to the above description for details. Of course, the seventh front-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0418] S1152 . Use the second MFNR algorithm to process the ninth intermediate image of the Q frames to obtain a tenth intermediate image of the Q frames.

[0419] The ninth intermediate image is an image in the RAW domain, and the tenth intermediate image is an image in the YUV domain.

[0420] S1153 , performing fifth back-end processing on one frame of the tenth intermediate image to obtain a corresponding captured image.

[0421] The fifth back-end processing is used to correct the tenth intermediate image.

[0422] The fifth back-end processing can be the same as the steps included in the above-mentioned first back-end processing. For details, please refer to the description of Figure 11. Of course, the fifth back-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0423] After being processed by the fifth back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0424] S116. If the detected dynamic range of the scene to be photographed is greater than the second dynamic range threshold, determine the second HDR algorithm as the first photographing algorithm, and use the second HDR algorithm to process the original image captured by the camera to obtain a corresponding photographed image.

[0425] In combination with S105, S111, S112, and S114, when the preset subject is dynamic, the motion speed is greater than the preset speed threshold, the illumination in the scene to be captured is greater than the second illumination threshold, and the dynamic range is greater than the second dynamic range threshold, it indicates that the preset subject has a relatively large motion amplitude and is in a fast motion state in a high dynamic range scene. In this case, if a general photography algorithm is used for processing, the obtained captured image may be overexposed. To address this issue, in the embodiment of the present application, a second HDR algorithm can be selected for processing to adjust the dynamic range and image details of the image, making the content presented in the obtained captured image more realistic and improving image quality.

[0426] Optionally, a second HDR algorithm is used to process the Q frame in the original image captured by the camera to obtain a corresponding captured image.

[0427] It should be understood that since the camera can capture 10 frames of original images in response to a single click, the second HDR algorithm can be used to process a portion of the original images, or all of the original images. This is not limited in this embodiment of the present application. Here, the number of frames processed by the second HDR algorithm can be the same as the number of frames processed by the second night scene algorithm.

[0428] In an embodiment of the present application, when multiple frames of original images are selected to be processed using the second HDR algorithm, the multiple frames of original images may correspond to different exposure values ​​respectively.

[0429] Optionally, the second HDR algorithm may be the same as the first HDR algorithm, or may be different. This embodiment of the present application does not impose any limitation on this.

[0430] For example, the second HDR algorithm can be the same as the first HDR algorithm, both being AI algorithms. Noise reduction, demosaicing, dynamic range compression, and other processing are achieved through an AI network. The processing is relatively complex and requires a lot of software and computing resources, such as GPUs and NPUs.

[0431] The second HDR algorithm can also be different from the first HDR algorithm. The first HDR algorithm is an AI algorithm, and the second HDR algorithm is a traditional algorithm. It uses software and hardware collaboration to complete noise reduction, de-mosaicing, dynamic range compression, etc. The resources that need to be called include ISP, CPU, DSP, etc., and the completion time is relatively shorter.

[0432] On this basis, the Q frame in the original image captured by the camera is processed using the second HDR algorithm to obtain a corresponding captured image, as shown in FIG16 , which may include the following S1161 to S1163 .

[0433] S1161 , performing an eighth front-end processing on the Q frame in the original image captured by the camera this time to obtain an eleventh intermediate image of the Q frame.

[0434] The eighth front-end processing can be the same as the steps included in the above-mentioned first front-end processing. Please refer to the above description for details. Currently, the eighth front-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0435] S1162 . Process the Q-frame eleventh intermediate image using a second HDR algorithm to obtain a 12th intermediate image.

[0436] The eleventh intermediate image is an image in the RAW domain, and the twelfth intermediate image is an image in the YUV domain.

[0437] S1163 , performing sixth back-end processing on the twelfth intermediate image of one frame to obtain a corresponding captured image.

[0438] The sixth back-end processing is used to correct the twelfth intermediate image.

[0439] The sixth back-end processing can be the same as the steps included in the above-mentioned first back-end processing. For details, please refer to the description of Figure 11. Of course, the sixth back-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0440] After being processed by the sixth back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0441] S117. In S111, if it is detected that the movement speed of the preset shooting object in the scene to be shot is less than or equal to the preset speed threshold, the single-frame algorithm can be determined as the first shooting algorithm, and the single-frame algorithm can be used to process the original image captured by the camera this time to obtain the corresponding shot image.

[0442] In conjunction with S111 and S117, when it is detected that the preset subject in the scene to be photographed is in motion and the motion speed is less than or equal to the preset speed threshold, it indicates that the motion amplitude of the preset subject is relatively small, which can be called a slow motion state. In this case, a general photography algorithm can be used to process it, for example, a single-frame algorithm can be used for processing.

[0443] Optionally, a single-frame algorithm is used to process one frame of the original image captured by the camera to obtain a corresponding captured image.

[0444] Optionally, the single-frame algorithm includes a ninth front-end process and a seventh back-end process.

[0445] On this basis, a single-frame algorithm is used to process one frame of the original image captured by the camera to obtain a corresponding captured image, as shown in FIG17 , which may include the following S1171 and S1172 .

[0446] S1171 . Perform the ninth front-end processing on one frame of the original image captured by the camera to obtain a thirteenth intermediate image.

[0447] The ninth front-end processing may include the same steps as those included in the first front-end processing. For details, please refer to the above description. Currently, the ninth front-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0448] The thirteenth intermediate image is an image in the RAW domain.

[0449] S1172 , performing seventh back-end processing on one frame of the thirteenth intermediate image to obtain a corresponding captured image.

[0450] The seventh back-end processing is used to correct the thirteenth intermediate image.

[0451] The seventh back-end processing can be the same as the steps included in the above-mentioned first back-end processing. For details, please refer to the description of Figure 11. Of course, the seventh back-end processing can also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0452] After being processed by the seventh back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0453] In this embodiment, when the application determines that the number of photos waiting in the background is small, which can also be called determining that the scene is in a normal photo shooting state, multiple tests are performed on the scene to be shot. Based on the test results, on the basis of determining whether the preset shooting object is static, the scene to be shot is further divided according to the detected illumination and dynamic range, so that the scene to be shot can be divided into multiple situations. Then, different algorithms are used for each situation to process the original images of different frame numbers, so that the processing method of the original image can be refined, and the quality and effect of the captured image obtained in each situation can be adaptively improved.

[0454] Embodiment 2, an image processing method, as shown in FIG18 , may include the following S201 to S209 .

[0455] S201: The electronic device displays a first interface, which includes a preview image and a shooting button, and the shooting button indicates a photo taking mode.

[0456] S202: When a click operation of a user on a capture button is detected, the electronic device uses a camera to capture an original image in response to the click operation.

[0457] S203: Determine the number of times the background waits for taking photos.

[0458] For example, after the user opens the camera, the current cumulative number of photos starts at 0. When the user detects that the user has performed click operations A1 to A6 on the capture button, the electronic device uses the camera to capture an original image in response to click operation A6, and the current cumulative number of photos is 6. Assuming that the number of photos processed at this time is 1 and the number of photos being processed is 1, the number of photos waiting in the background is 4, which can be obtained by subtracting the number of photos processed and the number of photos being processed from the current cumulative number of photos.

[0459] S204: Since the number of times the background waits for taking photos is 4, which is greater than 3, it can be determined whether the number of times the background waits for taking photos is less than or equal to 10.

[0460] S205: Since the number of times of waiting for photos in the background is 4, which is less than 10, it can be further determined based on the detected illumination result of the scene to be photographed whether the illumination is greater than a third illumination threshold.

[0461] The number of times the background waits for taking photos is greater than M times but less than N times, which means that although there is data waiting to be processed in the background, it is not particularly large. This situation can also be called a continuous photo-taking state.

[0462] It should be understood that the size of the third illumination threshold can be set and modified as needed. The size of the third illumination threshold can be the same as or different from the size of the second illumination threshold and the first illumination threshold. The embodiment of the present application does not impose any restrictions on this.

[0463] Here, the third illumination threshold is the same as the first illumination threshold and the second illumination threshold, that is, 100 Lux.

[0464] S206: If it is detected that the illumination in the scene to be photographed is less than or equal to the third illumination threshold, determine the third night scene algorithm as the second photographing algorithm, and use the third night scene algorithm to process the original image captured by the camera to obtain a corresponding photographed image.

[0465] In conjunction with S203, S204, and S205, when it is detected that the number of background waiting times for photos is moderate and the illumination of the scene to be captured is less than or equal to the third illumination threshold, it indicates that the user is continuously taking photos in a low-light environment. In this case, using a general photo processing algorithm will result in a blurry image with unclear details. Therefore, in this embodiment of the present application, the third night scene algorithm, which takes a relatively short time, can be selected for processing. This improves image clarity while also increasing processing efficiency and reducing the number of background waiting times for photos, achieving a balanced effect between effect and performance.

[0466] Optionally, the third night scene algorithm may be used to process the R frame in the original image captured by the camera to obtain a corresponding captured image.

[0467] It should be understood that due to the large number of backend waiting times for photos, a smaller number of original images can be selected for processing using the third night scene algorithm. The specific number of frames can be selected as needed, and this embodiment of the application does not impose any restrictions on this. Wherein, 1≤R≤Q<P; the value of R should be an integer.

[0468] For the rotation of multiple frames of original images, reference may be made to the description in S107 , which will not be repeated here.

[0469] Optionally, the third night scene algorithm may be the same as the second night scene algorithm and the first night scene algorithm, or of course, may be different. This embodiment of the present application does not impose any limitation on this.

[0470] On this basis, the third night scene algorithm is used to process the R frame in the original image collected by the camera this time to obtain the corresponding captured image, as shown in FIG19 , which may include the following S2061 to S2063 .

[0471] S2061 , performing the tenth front-end processing on the R frame in the original image captured by the camera this time to obtain the fourteenth intermediate image of the R frame.

[0472] For example, when the camera collects 10 frames of original images, assuming R=1, one of the frames can be selected to perform the tenth front-end processing to obtain a fourteenth intermediate image.

[0473] The tenth front-end processing may include the same steps as the first front-end processing, and the details may refer to the above description. Of course, the tenth front-end processing may also include other steps, which are not limited in this embodiment of the present application.

[0474] S2061. Use the third night scene algorithm to process the fourteenth intermediate image of the R frame to obtain a fifteenth intermediate image of the R frame.

[0475] The fourteenth intermediate image is an image in the RAW domain, and the fifteenth intermediate image is an image in the YUV domain.

[0476] S2063 , performing the eighth back-end processing on the fifteenth intermediate image of one frame to obtain a corresponding captured image.

[0477] The eighth back-end processing is used to correct the fifteenth intermediate image.

[0478] The eighth back-end processing may include the same steps as those included in the first back-end processing described above. For details, please refer to the description of Figure 11. Of course, the eighth back-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0479] After being processed by the eighth back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0480] S207: If it is detected that the illumination in the scene to be photographed is greater than the third illumination threshold, continue to determine whether the dynamic range result detected in the scene to be photographed is greater than the third dynamic range threshold.

[0481] It should be understood that the size of the third dynamic range threshold can be set and modified as needed. The size of the third dynamic range threshold can be the same as or different from the size of the second dynamic range threshold and the first dynamic range threshold. The embodiment of the present application does not impose any restrictions on this.

[0482] 208. If the detected dynamic range of the scene to be captured is less than or equal to the third dynamic range threshold, determine the third MFNR algorithm as the second shooting algorithm, and use the third MFNR algorithm to process the original image captured by the camera to obtain a corresponding captured image.

[0483] In combination with S203, S204, S205, and S208, when it is detected that the number of background waiting times for photos is moderate, and the illumination in the scene to be captured is greater than the third illumination threshold, while the dynamic range is less than or equal to the third dynamic range threshold, it indicates that the user is continuously taking photos in a bright environment but with a small dynamic range. In this case, using a general photo algorithm for processing will result in a captured image with high noise, a small dynamic range, and unrealistic content. To address this, in an embodiment of the present application, a third MFNR algorithm, which takes a relatively short time, can be selected for processing. While reducing noise and increasing the dynamic range of the captured image, it can also improve processing efficiency, reduce the number of background waiting times for photos, and balance effect and performance.

[0484] Optionally, the third MFNR algorithm is used to process the R frame in the original image captured by the camera to obtain a corresponding captured image.

[0485] It should be understood that due to the large number of background waiting times for photo taking, a smaller number of original images can be selected for processing using the third MFNR algorithm. The specific number of frames can be selected as needed, and this embodiment of the present application does not impose any restrictions on this. Here, the number of frames processed using the third MFNR algorithm can be the same as the number of frames processed using the third night scene algorithm.

[0486] In the embodiment of the present application, when multiple frames of original images are selected to be processed using the third MFNR algorithm, the exposure time of the multiple frames of original images is the same.

[0487] Optionally, the third MFNR algorithm may be the same as the first MFNR algorithm and the second MFNR algorithm, or may be different. This embodiment of the present application does not impose any limitation on this.

[0488] On this basis, the third MFNR algorithm is used to process the R frame in the original image captured by the camera to obtain a corresponding captured image, as shown in FIG20 , which may include the following S2081 to S2083.

[0489] S2081. Perform the eleventh front-end processing on the R frame in the original image captured by the camera to obtain the sixteenth intermediate image of the R frame.

[0490] The eleventh front-end processing may include the same steps as those included in the first front-end processing. For details, please refer to the above description. Of course, the eleventh front-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0491] S2082. Use the third MFNR algorithm to process the sixteenth intermediate image of the R frame to obtain a seventeenth intermediate image of the R frame.

[0492] The sixteenth intermediate image is an image in the RAW domain, and the seventeenth intermediate image is an image in the YUV domain.

[0493] S2083: Perform the ninth back-end processing on the seventeenth intermediate image of one frame to obtain a corresponding captured image.

[0494] The ninth back-end processing is used to correct the seventeenth intermediate image.

[0495] The ninth back-end processing may include the same steps as those included in the first back-end processing described above. For details, please refer to the description of Figure 11. Of course, the ninth back-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0496] After being processed by the ninth back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0497] S209: If the detected dynamic range of the scene to be photographed is greater than the third dynamic range threshold, determine the third HDR algorithm as the second photographing algorithm, and use the third HDR algorithm to process the original image captured by the camera to obtain a corresponding photographed image.

[0498] In combination with S203, S204, S205 and S207, when it is detected that the number of background waiting times for taking photos is in the middle, and the illumination in the scene to be photographed is greater than the third illumination threshold, and the dynamic range is also greater than the third dynamic range threshold, it indicates that the user is taking continuous photos in a bright high dynamic range environment. At this time, using a general photo algorithm for processing, the obtained captured image may be overexposed. To this end, in an embodiment of the present application, a third HDR algorithm with a relatively short processing time can be selected for processing. While adjusting the dynamic range of the image, enhancing the image details, and making the content presented in the captured image more realistic, it can also improve processing efficiency, reduce the number of background waiting times for taking photos, and take into account both effect and performance.

[0499] Optionally, a third HDR algorithm is used to process the R frame in the original image captured by the camera to obtain a corresponding captured image.

[0500] It should be understood that due to the large number of backstage waiting times for taking photos, a smaller number of original images can be selected for processing using the third HDR algorithm. The specific number of frames can be selected as needed, and this embodiment of the application does not impose any restrictions on this. Here, the number of frames processed using the third HDR algorithm can be the same as the number of frames processed using the third night scene algorithm.

[0501] In an embodiment of the present application, when multiple frames of original images are selected to be processed using the third HDR algorithm, the multiple frames of original images may correspond to different exposure values ​​respectively.

[0502] Optionally, the third HDR algorithm may be the same as the second HDR algorithm and the first HDR algorithm, or of course, may be different. This embodiment of the present application does not impose any limitation on this.

[0503] On this basis, the R frame in the original image captured by the camera is processed using the third HDR algorithm to obtain a corresponding captured image, as shown in FIG21 , which may include the following S2091 to S2093.

[0504] S2091: Perform the twelfth front-end processing on the R frame in the original image captured by the camera to obtain the eighteenth intermediate image of the R frame.

[0505] The twelfth front-end processing may include the same steps as those included in the first front-end processing. For details, please refer to the above description. Currently, the twelfth front-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0506] S2092. Use the third HDR algorithm to process the eighteenth intermediate image of the R frame to obtain a nineteenth intermediate image of the R frame.

[0507] The eighteenth intermediate image is an image in the RAW domain, and the nineteenth intermediate image is an image in the YUV domain.

[0508] S2093 , performing tenth back-end processing on one frame of the nineteenth intermediate image to obtain a corresponding captured image.

[0509] The tenth back-end processing is used to correct the nineteenth intermediate image.

[0510] The tenth back-end processing may include the same steps as those included in the first back-end processing described above. For details, please refer to the description of Figure 11. Of course, the tenth back-end processing may also include other steps, and the embodiment of the present application does not impose any restrictions on this.

[0511] After being processed by the tenth back-end, the captured image is an image in the YUV domain. Alternatively, further format conversion may be performed to convert the captured image into other formats commonly used for display, such as JPEG.

[0512] In this embodiment, when the present application determines that the number of photos waiting in the background is relatively large, which can also be called determining that it is in a continuous photo-taking state, it divides the scene to be photographed into multiple situations based on the detected illumination and dynamic range. Then, different algorithms are used to process the original images with a relatively small number of frames for each situation, so that the processing method of the original image can be refined. While ensuring the quality of the captured image, the processing efficiency is improved, the number of photos waiting in the background is reduced, and both effect and performance are taken into consideration.

[0513] Embodiment 3, an image processing method, as shown in FIG22 , may include the following S301 to S305 .

[0514] S301: The electronic device displays a first interface, which includes a preview image and a shooting button, and the shooting button indicates a photo taking mode.

[0515] S302: When a click operation of a user on a shooting key is detected, the electronic device uses a camera to capture an original image in response to the click operation.

[0516] S303: Determine the number of times the background waits for taking photos.

[0517] For example, after the user opens the camera, the current cumulative number of photos starts at 0. When the user detects that the user has performed click operations A1 to A20 on the capture button, the electronic device uses the camera to capture an original image in response to click operation A20, and the current cumulative number of photos is 20. Assuming that the number of photos processed at this time is 5 and the number of photos being processed is 1, the number of photos waiting in the background is 14, which can be obtained by subtracting the number of photos processed and the number of photos being processed from the current cumulative number of photos.

[0518] S304: Since the number of times the background waits for taking photos is 14 times, which is greater than 3 times, it can be determined whether the number of times the background waits for taking photos is less than or equal to 10.

[0519] S305. Since the number of times the background waits for taking photos is 14 times, which is greater than 10 times, it can be determined that the single-frame algorithm is the third photo-taking algorithm, and the single-frame algorithm is used to process the original image collected by the camera this time to obtain the corresponding captured image.

[0520] Combining S303 and S304, if the number of background waiting times exceeds 10, it indicates that the user has clicked too many times and there is too much data waiting to be processed in the background. This situation can also be called a super continuous shooting state. In this case, the single-frame algorithm with the shortest processing time can be used to reduce the number of background waiting times and improve processing efficiency.

[0521] Optionally, a single-frame algorithm is used to process one frame of the original image captured by the camera to obtain a corresponding captured image.

[0522] For the description of the single-frame algorithm, please refer to the content in S117 above, which will not be repeated here.

[0523] In this embodiment, when the present application determines that the number of photos waiting in the background is relatively large, which can also be called determining that it is in a super continuous photo shooting state, it can use a single-frame algorithm with the simplest processing steps and the shortest time to process the minimum number of original images, thereby maximizing the processing efficiency, reducing the number of photos waiting in the background, and avoiding the problem of frame loss.

[0524] Embodiment 4, an image processing method, as shown in FIG23 , may include the following S401 to S403 .

[0525] S401: The electronic device displays a first interface, which includes a preview image and a shooting button, and the shooting button indicates a photo taking mode.

[0526] S402: When a long press operation of a capture button by a user is detected, the electronic device uses a camera to capture multiple original images in response to the long press operation.

[0527] S403: Process the original image collected each time using a single-frame algorithm to obtain corresponding captured images.

[0528] When a long press is detected, indicating the user intends to take multiple photos, a large amount of data will need to be processed in the background, which is equivalent to a super-continuous shooting state. In this case, a single-frame algorithm with the shortest processing time can be used to reduce the number of background waiting times and improve processing efficiency.

[0529] Optionally, a single-frame algorithm is used to process one frame of the original image captured by the camera to obtain a corresponding captured image.

[0530] For the description of the single-frame algorithm, please refer to the content in S117 above, which will not be repeated here.

[0531] In this embodiment, when the present application determines that the user performs a long press operation, it can use a single-frame algorithm with the simplest processing steps and the shortest time to process the minimum number of original images, thereby maximizing processing efficiency, reducing the number of times the background waits for taking photos, and avoiding the problem of frame loss.

[0532] In combination with the steps of the above embodiment, two examples are provided below to illustrate the overall flow of image processing.

[0533] In Example 1, as shown in FIG24 , after the user opens the camera, the current cumulative number of photos begins at 0. When a user clicks the capture button A1, the electronic device uses the camera to capture a single original image in response to the click A1, and the current cumulative number of photos is 1. Since there are no completed or in-progress photos at this point, the number of photos waiting in the background is 1, which is deducted from the number of photos waiting in the background. At this point, since the number of photos waiting in the background is less than 3, S104 to S117 described above can be executed to obtain the captured image triggered by the click A1.

[0534] At the same time, after performing the click operation A1, the user quickly performs click operations A2, A3 and A4 on the capture key.

[0535] When the user clicks the capture button A2, the electronic device captures an original image using the camera in response to A2, bringing the total number of captures to 2. The original image captured in response to A1 is still being processed, so the captured original image and related data are first stored in the background for processing.

[0536] When the user clicks the capture button A3, the electronic device captures an original image using the camera in response to the click A3, bringing the total number of captures to three. The original image captured in response to the click A1 is still being processed, so the original image and related data are first stored in the background for processing.

[0537] Similarly, when the user clicks the capture button A4, the electronic device captures a raw image using the camera in response to the click A4, bringing the total number of captured images to four. The raw image captured in response to the click A1 is still being processed, so the captured raw image and related data are first stored in the background for processing.

[0538] Then, after the processing of the original image captured in response to click operation A1 is completed, the background waits for three photo-taking times, starting with the original image captured in response to click operation A4, in descending order of time. In other words, the second photo-taking process should process the original image captured by the electronic device's camera in response to click operation A4.

[0539] Next, the electronic device processes the original image captured by the camera in response to click operation A4. Since the number of photos processed at this point is 1 (the original image captured in response to click operation A1) and the number of photos being processed is 0, the number of photos waiting in the background is 2, which is calculated by subtracting the number of photos processed and the number of photos being processed from the current cumulative number of photos. Since the number of photos waiting in the background is still less than 3, the above steps S104 to S117 can be repeated to obtain the captured image triggered by click operation A4.

[0540] While the electronic device is processing the original image captured by the camera in response to the click operation A4, the user is also performing click operations A5 and A6.

[0541] When the user clicks the capture button A5, the electronic device captures an original image using the camera in response to the click A5, and the cumulative number of photos taken is now 5. At this time, the original image captured in response to the click A4 is still being processed, so the original image and related data captured this time are first put into the background waiting.

[0542] When the user clicks the capture button A6, the electronic device captures an original image using the camera in response to the click A6, and the cumulative number of photos taken is now 6. At this time, the original image captured in response to the click A4 is still being processed, so the original image and related data captured this time are first put into the background waiting.

[0543] As time passes, the processing of the original image captured in response to click operation A4 is completed. The four waiting backstage photo processing times are then processed, starting with the original image captured in response to click operation A6, in descending order of time. In other words, the third photo processing should be for the original image captured by the electronic device's camera in response to click operation A6.

[0544] Therefore, after processing the original image captured in response to click operation A4, the electronic device then processes the original image captured by the camera in response to click operation A6. Since the number of photos processed at this point is 2 (the original images captured in response to click operation A1 and click operation A4 have been processed), the number of photos being processed is 0. Therefore, by subtracting the number of photos processed and the number of photos being processed from the current cumulative number of photos, the number of photos waiting in the background can be calculated as 4. Since the number of photos waiting in the background is greater than 3, it is necessary to execute the above steps S204 to S208 to obtain the captured image triggered by click operation A6.

[0545] Then, after processing the original image captured in response to click operation A6, the electronic device processes the original image captured by the camera in response to click operation A5. Since the number of photos processed at this point is 3 (the original images captured in response to click operation A1, click operation A4, and click operation A6 have been processed), and the number of photos being processed is 0, the number of photos waiting in the background is 3, which can be calculated by subtracting the number of photos processed and the number of photos being processed from the current cumulative number of photos. Since the number of photos waiting in the background is 3, the above steps S104 to S117 can be executed again to obtain the captured image triggered by click operation A5.

[0546] After processing the original image captured in response to click operation A5, the electronic device processes the original image captured by the camera in response to click operation A3. Since the number of photos processed at this point is 4 (the original images captured in response to click operation A1, click operation A4, click operation A6, and click operation A5 have been processed), and the number of photos being processed is 0, the number of photos waiting in the background is 2, which is calculated by subtracting the number of photos processed and the number of photos being processed from the current cumulative number of photos. Since the number of photos waiting in the background is less than 3, the above steps S104 to S117 can be executed to obtain the captured image triggered by click operation A3.

[0547] Finally, after processing the original image captured in response to click operation A3, the electronic device processes the original image captured by the camera in response to click operation A2. Since the number of photos processed at this point is 5 (the original images captured in response to click operation A1, click operation A4, click operation A6, click operation A5, and click operation A3 have been processed), and the number of photos being processed is 0, the number of photos waiting in the background is 1, which is calculated by subtracting the number of photos processed and the number of photos being processed from the current cumulative number of photos. Since the number of photos waiting in the background is less than 3, the above steps S104 to S117 can be executed to obtain the captured image triggered by click operation A2.

[0548] Example 2, as shown in Figure 25, after the user turns on the camera, when it is detected that the user long-presses the shooting button, in response to the long-press operation, the electronic device uses the camera to collect 20 original images, which is equivalent to instructing the camera to take 20 photos (for example, B1 to B20).

[0549] First, the single-frame algorithm is used to process the original image collected by the first photo B1. During the processing of the original image collected by the first photo B1, the camera is still taking photos from B2 to B12.

[0550] It should be understood that when taking the second photo B2, the electronic device uses the camera to collect an original image. At this time, the original image collected by the first photo B1 is still being processed, so the original image and related data collected this time are first put into the background and wait.

[0551] Similarly, when taking the third to twelfth photos B3 to B12, since the original image collected by the first photo B1 is still being processed, the original image and related data collected each time enter the background waiting.

[0552] Then, after the original image captured by the first shot B1 is processed, the 11 shots waiting in the background are processed first, starting with the original image captured by the twelfth shot B12, in order of time. In other words, the second shot processing should process the original image captured by the twelfth shot B12.

[0553] Therefore, the single-frame algorithm is then used to process the original image collected by the twelfth photograph B12. During the processing of the original image collected by the twelfth photograph B12, the camera is still taking photographs from B13 to B20.

[0554] It should be understood that when taking the thirteenth photo B13, the electronic device uses the camera to collect an original image. At this time, the original image collected by the twelfth photo B12 is still being processed, so the original image and related data collected this time are first put into the background and wait.

[0555] Similarly, when taking the fourteenth to twentieth photos B14 to B20, since the original image collected by the twelfth photo B12 is still being processed, the original image and related data collected each time enter the background waiting.

[0556] After the processing of the original image captured by the twelfth shot B12 is completed, the background waits for 17 shots in order of time, first processing the original image captured by the twentieth shot B20, then the original image captured by the nineteenth shot B19, and so on until the original image captured by the second shot B2 is processed. Each processing is performed using the single-frame algorithm.

[0557] The above is a detailed introduction to the image processing method provided in the embodiment of the present application. The following describes how a user enables the image processing method provided in the embodiment of the present application in conjunction with the display interface of an electronic device.

[0558] FIG26 is a schematic diagram of a display interface of an electronic device provided in an embodiment of the present application.

[0559] For example, in response to a user's click operation, when the electronic device 100 runs a camera application, the electronic device 100 displays a shooting interface as shown in (a) of Figure 26. A navigation bar is displayed above the interface, wherein an icon indicating "AI Photography" is displayed in the middle.

[0560] In response to the user clicking on "AI Photography", the electronic device 100 displays a prompt box as shown in (b) of Figure 26, which displays a prompt message that "AI Photography" is turned on. At this time, the electronic device 100 can enable the program related to the image processing method provided in the embodiment of the application when shooting.

[0561] It should be understood that the above is only an example of a user enabling the image processing method provided in an embodiment of the present application from the display interface of an electronic device. Of course, the image processing method provided in an embodiment of the present application can also be enabled in other ways, or the image processing method provided in an embodiment of the present application can be directly used by default during the shooting process. The embodiment of the present application does not impose any restrictions on this.

[0562] The image processing methods and related display interfaces provided in the embodiments of the present application are described in detail above with reference to Figures 1 to 26 ; the electronic devices, apparatuses, and chips provided in the embodiments of the present application are described in detail below with reference to Figures 27 to 30 . It should be understood that the electronic devices, apparatuses, and chips in the embodiments of the present application can perform the various image processing methods of the aforementioned embodiments of the present application. That is, the specific operating processes of the various products below can refer to the corresponding processes in the aforementioned method embodiments.

[0563] Figure 27 shows a hardware system of an electronic device applicable to the present application. The electronic device 100 can be used to implement the image processing method described in the above method embodiment.

[0564] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0565] It should be noted that the structure shown in FIG27 does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in FIG27, or the electronic device 100 may include a combination of some of the components shown in FIG27, or the electronic device 100 may include sub-components of some of the components shown in FIG27. The components shown in FIG27 may be implemented in hardware, software, or a combination of software and hardware.

[0566] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices.

[0567] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0568] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0569] In an embodiment of the present application, the processor 110 can execute the display of a first interface, the first interface including a preview image and a first control, the first control indicating a photo mode; when a first operation on a control is detected, in response to the first operation, an original image is captured once using the camera; the number of background waiting photos is determined; when the number of background waiting photos is less than or equal to M, the original image is processed using a first photo algorithm to obtain a corresponding captured image; when the number of background waiting photos is greater than M but less than or equal to N, the original image is processed using a second photo algorithm to obtain a corresponding captured image, and the captured image is saved.

[0570] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0571] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0572] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0573] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 is also used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance).

[0574] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0575] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0576] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0577] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0578] The camera 193 is used to capture still images or videos. It can be triggered to turn on via application instructions to implement the camera function, such as capturing images of any scene. The camera may include components such as an imaging lens, a filter, and an image sensor. Light emitted or reflected by an object enters the imaging lens, passes through the filter, and is ultimately focused on the image sensor. The imaging lens is primarily used to focus and image the light emitted or reflected by all objects within the camera's viewing angle (also referred to as the scene to be captured, the target scene, or the scene image the user expects to capture); the filter is primarily used to filter out excess light waves (e.g., light waves other than visible light, such as infrared); and the image sensor may be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The image sensor is primarily used to perform photoelectric conversion on the received light signal, converting it into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

[0579] Among them, the camera 193 can be located in the front of the electronic device 100 or in the back of the electronic device 100. The specific number and arrangement of the cameras can be set according to needs, and this application does not impose any restrictions.

[0580] Exemplarily, the electronic device 100 includes a front camera and a rear camera. For example, the front camera or the rear camera may include one or more cameras. Taking the electronic device 100 having four rear cameras as an example, when the electronic device 100 activates the four rear cameras for shooting, the image processing method provided in the embodiment of the present application can be used. Alternatively, the camera is set on an external accessory of the electronic device 100, and the external accessory is rotatably connected to the frame of the mobile phone, and the angle formed between the external accessory and the display screen 194 of the electronic device 100 is any angle between 0-360 degrees. For example, when the electronic device 100 takes a selfie, the external accessory drives the camera to rotate to a position facing the user. Of course, when the mobile phone has multiple cameras, only some of the cameras can be set on the external accessories, and the remaining cameras can be set on the electronic device 100 body. The embodiment of the present application does not impose any restrictions on this.

[0581] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0582] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0583] The internal memory 121 can also store the software code of the image processing method provided in the embodiment of the present application. When the processor 110 runs the software code, the process steps of the image processing method are executed to achieve "unlimited shooting".

[0584] The internal memory 121 may also store captured images.

[0585] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music can be stored in the external memory card.

[0586] Of course, the software code of the image processing method provided in the embodiment of the present application can also be stored in an external memory, and the processor 110 can run the software code through the external memory interface 120 to execute the process steps of the image processing method to obtain a higher-definition image. The image captured by the electronic device 100 can also be stored in an external memory.

[0587] It should be understood that the user can specify whether to store the image in the internal memory 121 or the external memory. For example, when the electronic device 100 is connected to the external memory, if the electronic device 100 captures a frame of image, a prompt message may pop up to prompt the user to store the image in the external memory or the internal memory. Of course, other designation methods are possible, and the present embodiment does not impose any restrictions on this. Alternatively, when the electronic device 100 detects that the memory capacity of the internal memory 121 is less than a preset amount, it may automatically store the image in the external memory.

[0588] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0589] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0590] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can also be used for image stabilization.

[0591] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0592] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0593] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0594] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0595] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect when the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0596] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0597] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0598] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0599] The touch sensor 180K is also called a "touch-sensitive device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0600] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0601] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0602] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or touch vibration feedback. For example, touch operations on different applications (such as taking photos, playing audio, etc.) can correspond to different vibration feedback effects.

[0603] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0604] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting or removing the SIM card into or from the SIM card interface 195 .

[0605] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0606] The hardware system of the electronic device 100 is described in detail above. The software system of the electronic device 100 is introduced below.

[0607] Figure 28 is a schematic diagram of the software system of the electronic device provided in an embodiment of the present application.

[0608] As shown in FIG. 28 , the system architecture may include an application layer (APP) 210 , an application framework layer 220 , a hardware abstract layer (HAL) 230 , a driver layer 240 , and a hardware layer 250 .

[0609] The application layer 210 may include a camera application or other applications. Other applications include but are not limited to: camera, gallery, and other applications.

[0610] The application layer 210 is at the top of the entire framework and is responsible for interacting directly with the user. Once it receives a direct or indirect request from the user, such as taking a photo, it will send the request to the application framework layer 220 through the interface and wait for the application framework layer 220 to return the processing result, which includes image data and camera parameters, etc.; then the application layer 210 will feed back the result to the user.

[0611] The application framework layer 220 may provide an application programming interface (API) and a programming framework to the application programs of the application layer; the application framework layer may include some predefined functions.

[0612] For example, the application framework layer 220 may include a camera access interface; the camera access interface may include camera management and camera equipment; wherein, camera management may be used to provide an access interface for managing the camera; and camera equipment may be used to provide an interface for accessing the camera.

[0613] The hardware abstraction layer 230 is used to abstract hardware. For example, the hardware abstraction layer can include a camera hardware abstraction layer and other hardware device abstraction layers; the camera hardware abstraction layer can include camera device 1, camera device 2, etc.; the camera hardware abstraction layer can be connected to the camera algorithm library, and the camera hardware abstraction layer can call the algorithms in the camera algorithm library.

[0614] In this application, the perception engine used to perform various detections can be set in the hardware abstraction layer.

[0615] The driver layer 240 is used to provide drivers for different hardware devices. For example, the driver layer may include a camera device driver, a digital signal processor driver, and a graphics processor driver.

[0616] The hardware layer 250 may include multiple image sensors, multiple image signal processors, digital signal processors, graphics processors, and other hardware devices.

[0617] For example, the hardware layer 250 includes sensors and image signal processors. The sensors may include sensor 1, sensor 2, a depth sensor (time of flight, TOF), a multispectral sensor, etc. The image signal processors may include image signal processor 1, image signal processor 2, etc.

[0618] In this application, by calling the hardware abstraction layer interface in the hardware abstraction layer 230, the connection between the application layer 210 and the application framework layer 220 above the hardware abstraction layer 230 and the driver layer 240 and the hardware layer 250 below can be realized, thereby realizing camera data transmission and function control.

[0619] Manufacturers can customize functionality within the camera hardware interface layer within the hardware abstraction layer 230. Compared to the hardware abstraction layer interface, the camera hardware interface layer is more efficient, flexible, and has lower latency. It also allows for more comprehensive access to the ISP and GPU for image processing. Images input to the hardware abstraction layer 230 can come from an image sensor or a stored image.

[0620] The scheduling layer in the hardware abstraction layer 230 includes a general functional interface for implementing management and control.

[0621] The camera service layer in the hardware abstraction layer 230 is used to access the ISP and other hardware interfaces.

[0622] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0623] The camera application in the application layer can be displayed on the screen of the electronic device 100 in the form of an icon. When the camera application icon is clicked by the user to trigger it, the electronic device 100 starts running the camera application. When the camera application is running on the electronic device 100, the camera application calls the interface corresponding to the camera application in the application framework layer 210, then starts the camera driver by calling the hardware abstraction layer 230, turns on the camera 193 on the electronic device 100, and captures one or more frames of initial images through the image sensor. The image sensor can then process the captured image within the image sensor or transmit it to one or more image signal processors for processing, and then save the processed captured image and / or transmit it to the display screen for display.

[0624] The following describes an image processing device 300 for implementing the above-mentioned image processing method provided in an embodiment of the present application. FIG29 is a schematic diagram of the image processing device 300 provided in an embodiment of the present application.

[0625] As shown in FIG. 29 , the image processing apparatus 300 includes a display unit 310 , an acquisition unit 320 , and a processing unit 330 .

[0626] The display unit 310 is used to display a first interface, which includes a preview image and a first control, and the first control indicates a photo taking mode.

[0627] The acquisition unit 320 is configured to detect a first operation performed by a user on a first control.

[0628] The processing unit 330 is configured to capture an original image using a camera in response to the first operation.

[0629] The processing unit 330 is also used to determine the number of background waiting times for taking photos; when the number of background waiting times for taking photos is less than or equal to M, the first photo algorithm is used to process the original image to obtain a corresponding photographed image; when the number of background waiting times for taking photos is greater than M but less than or equal to N, the second photo algorithm is used to process the original image to obtain a corresponding photographed image and save the photographed image.

[0630] It should be noted that the image processing device 300 is implemented in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0631] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the aforementioned functionality. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0632] Therefore, the units of each example described in the embodiments of this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0633] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium runs on the image processing device 300, the image processing device 300 executes the image processing method shown above.

[0634] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium (e.g., a solid state disk (SSD)).

[0635] The embodiment of the present application further provides a computer program product including computer instructions, which, when executed on the image processing apparatus 300, enables the image processing apparatus 300 to execute the aforementioned image processing method.

[0636] FIG30 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip shown in FIG30 can be a general-purpose processor or a dedicated processor. The chip includes a processor 401. The processor 401 is used to support the image processing device 300 in executing the technical solution shown above.

[0637] Optionally, the chip further includes a transceiver 402 , which is configured to accept control from the processor 401 and to support the image processing apparatus 300 in executing the aforementioned technical solution.

[0638] Optionally, the chip shown in FIG30 may further include: a storage medium 403 .

[0639] It should be noted that the chip shown in Figure 30 can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.

[0640] The electronic device, image processing device 300, computer storage medium, computer program product, and chip provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0641] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application. Based on the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, certain steps in each embodiment of the above detection method may be unnecessary, or certain new steps may be added. Or a combination of any two or any multiple embodiments described above. Such modifications, changes, or combined solutions also fall within the scope of the embodiments of the present application.

[0642] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced with each other. For the sake of brevity, they will not be repeated here.

[0643] It should also be understood that the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0644] It should also be understood that in the embodiments of the present application, "pre-setting" and "pre-definition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device (for example, including an electronic device), and the present application does not limit its specific implementation method.

[0645] It should also be understood that the division of the modes, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various modes, categories, situations and embodiments can be combined without contradiction.

[0646] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0647] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: Applied to an electronic device including a camera, the method includes: The electronic device displays a first interface, the first interface including a preview image and a first control, and the first control indicates a photo taking mode; When a first operation on the first control is detected, in response to the first operation, the electronic device uses the camera to capture an original image once; Determine the number of backend waiting photos, where the number of backend waiting photos is equal to the current cumulative number of photos minus the number of photos that have been processed and the number of photos being processed; When the number of times the background waits for taking photos is less than or equal to M, the original image is processed using a first photo-taking algorithm to obtain a corresponding photographed image; When the number of times the background waits for taking photos is greater than M but less than or equal to N, the original image is processed using a second photo-taking algorithm to obtain a corresponding captured image; Saving the captured image; Wherein, M<N, M and N are both positive integers, and the processing time of the second photographing algorithm is less than the processing time of the first photographing algorithm.

2. The image processing method according to claim 1, wherein: The method further comprises: When the number of times the background waits for taking photos is greater than N, the original image is processed using a third photo algorithm to obtain a corresponding captured image, and the processing time of the third photo algorithm is less than the processing time of the second photo algorithm.

3. The image processing method according to claim 1 or 2, characterized in that: The method further comprises: When a second operation on the first control is detected, in response to the second operation, the electronic device uses the camera to capture multiple original images; The original image collected each time is processed using a fourth photographing algorithm to obtain corresponding photographed images respectively; the processing time of the fourth photographing algorithm is less than the processing time of the second photographing algorithm.

4. The image processing method according to any one of claims 1 to 3, characterized in that: Before processing the original image using the first photographing algorithm to obtain a corresponding photographed image, the method further includes: Detecting the scene to be captured and obtaining detection results, wherein the detection includes at least one of AI scene recognition, motion detection, face detection, face attribute detection, illumination detection, and dynamic range detection; The first photographing algorithm is determined according to the detection result.

5. The image processing method according to claim 4, wherein: Detecting the scene to be photographed and obtaining a detection result, the method includes: Performing front-end processing on the original image, wherein the front-end processing is used to correct the original image; Perform detection on the original image after front-end processing to obtain the detection results.

6. The image processing method according to claim 5, characterized in that The front-end processing includes a first front-end processing and a second front-end processing; The first front-end processing includes: at least one of bad pixel correction, RAW domain noise reduction, black level correction, lens shading correction, and automatic white balance; The second front-end processing includes at least one of color interpolation, color correction, global tone mapping processing, and gamma correction.

7. The image processing method according to any one of claims 4 to 6, characterized in that: When the number of times the background waits for taking photos is less than or equal to M, the original image is processed using a first photo-taking algorithm to obtain a corresponding photographed image, including: When the number of times the background waits for taking photos is less than or equal to M, determining whether the preset subject in the scene to be photographed is static according to the detection result; When the scene is static, determining whether the illumination in the scene to be captured is greater than a first illumination threshold; When the illumination value is less than or equal to the first illumination threshold, determining that the first night scene algorithm is the first photographing algorithm, and using the first night scene algorithm to process the P frame in the original image to obtain a corresponding photographed image; When the illumination intensity is greater than the first illumination threshold, determining whether the dynamic range of the scene to be captured is greater than a first dynamic range threshold; When the dynamic range is less than or equal to the first dynamic range threshold, determining a first MFNR algorithm as the first photographing algorithm, and processing the P frame in the original image using the first MFNR algorithm to obtain a corresponding photographed image; When it is greater than the first dynamic range threshold, the first HDR algorithm is determined to be the first photographing algorithm, and the first HDR algorithm is used to process the P frame in the original image to obtain a corresponding photographed image, where P is an integer greater than 1.

8. The image processing method according to claim 7, wherein: Processing the P frame in the original image using the first night scene algorithm to obtain a corresponding captured image includes: Performing a third front-end processing on the P frame in the original image to obtain a first intermediate image of the P frame; Using the first night scene algorithm, processing the P frames of the first intermediate image to obtain a frame of the second intermediate image; Perform first back-end processing on one frame of the second intermediate image to obtain a corresponding captured image.

9. The image processing method according to claim 7 or 8, characterized in that: Processing the P frame in the original image using the first MFNR algorithm to obtain a corresponding captured image includes: performing a fourth front-end processing on the P frame in the original image to obtain a third intermediate image of the P frame; Processing the third intermediate image of the P frame using the first MFNR algorithm to obtain a fourth intermediate image; Performing second back-end processing on one frame of the fourth intermediate image to obtain a corresponding captured image.

10. The image processing method according to any one of claims 7 to 9, characterized in that: Processing the P frame in the original image using the first HDR algorithm to obtain a corresponding captured image includes: performing a fifth front-end processing on the P frame in the original image to obtain a fifth intermediate image of the P frame; Using the first HDR algorithm, processing the fifth intermediate image of the P frame to obtain a sixth intermediate image; Perform third back-end processing on one frame of the sixth intermediate image to obtain a corresponding captured image.

11. The image processing method according to any one of claims 7 to 10, characterized in that: The method further comprises: If the subject is not static, determining whether the moving speed of the preset subject is greater than a preset speed threshold; When the speed is greater than the preset speed threshold, determining whether the illumination in the scene to be captured is greater than a second illumination threshold; When the value is less than or equal to the second illumination threshold, determining that the second night scene algorithm is the first photographing algorithm, and using the second night scene algorithm to process the Q frame in the original image to obtain a corresponding photographed image; When the illumination intensity is greater than the second illumination threshold, determining whether the dynamic range of the scene to be captured is greater than a second dynamic range threshold; When the dynamic range is less than or equal to the second dynamic range threshold, determining that the second MFNR algorithm is the first photographing algorithm, and using the second MFNR algorithm to process the Q frame in the original image to obtain a corresponding photographed image; When it is greater than the second dynamic range threshold, the second HDR algorithm is determined to be the first photographing algorithm, and the second HDR algorithm is used to process Q frames in the original image to obtain a corresponding photographed image, where Q is an integer greater than or equal to 1, and Q<P.

12. The image processing method according to claim 11, wherein: Processing the Q frame in the original image using the second night scene algorithm to obtain a corresponding captured image includes: performing a sixth front-end processing on the Q frame in the original image to obtain a seventh intermediate image of the Q frame; Using the second night scene algorithm, processing the Q frames of the seventh intermediate image to obtain one frame of the eighth intermediate image; Perform fourth back-end processing on one frame of the eighth intermediate image to obtain a corresponding captured image.

13. The image processing method according to claim 11 or 12, characterized in that: Processing the Q frame in the original image using the second MFNR algorithm to obtain a corresponding captured image includes: performing a seventh front-end processing on the Q frame in the original image to obtain a ninth intermediate image of the Q frame; Processing the ninth intermediate image of Q frames using the second MFNR algorithm to obtain a tenth intermediate image of one frame; Perform fifth back-end processing on one frame of the tenth intermediate image to obtain a corresponding captured image.

14. The image processing method according to any one of claims 11 to 13, characterized in that: Processing the Q frame in the original image using the second HDR algorithm to obtain a corresponding captured image includes: performing an eighth front-end processing on the Q frame in the original image to obtain an eleventh intermediate image of the Q frame; Using the second HDR algorithm, processing the Q frames of the eleventh intermediate image to obtain a frame of the twelfth intermediate image; Perform sixth back-end processing on one frame of the twelfth intermediate image to obtain a corresponding captured image.

15. The image processing method according to any one of claims 11 to 14, characterized in that: The method further comprises: When the speed is less than or equal to the preset speed threshold, the single-frame algorithm is determined to be the first photographing algorithm, and the single-frame algorithm is used to process one frame of the original image to obtain a corresponding photographed image.

16. The image processing method according to claim 15, characterized in that: Using the single-frame algorithm to process one frame of the original image to obtain a corresponding captured image, including: performing the ninth front-end processing on one frame of the original image to obtain a thirteenth intermediate image; The seventh back-end processing is performed on one frame of the thirteenth intermediate image to obtain a corresponding captured image.

17. The image processing method according to any one of claims 4 to 16, characterized in that: When the number of times the background waits for taking photos is greater than M but less than or equal to N, the original image is processed using a second photo-taking algorithm to obtain a corresponding photographed image, including: When the number of times the background waiting for taking photos is greater than M but less than or equal to N, determining whether the illumination in the scene to be photographed is greater than a third illumination threshold according to the detection result; When the illumination value is less than or equal to the third illumination threshold, determining the third night scene algorithm as the second photographing algorithm, and using the third night scene algorithm to process the R frame in the original image to obtain a corresponding photographed image; When the illumination intensity is greater than the third illumination threshold, determining whether the dynamic range of the scene to be captured is greater than the third dynamic range threshold; When the value is less than or equal to the third dynamic range threshold, determining a third MFNR algorithm as the second photographing algorithm, and processing the R frame in the original image using the third MFNR algorithm to obtain a corresponding photographed image; When it is greater than the third dynamic range threshold, the third HDR algorithm is determined to be the second photographing algorithm, and the third HDR algorithm is used to process the R frame in the original image to obtain a corresponding captured image, where R is an integer greater than or equal to 1, and R≤Q<P.

18. The image processing method according to claim 17, wherein: Processing the R frame in the original image using the third night scene algorithm to obtain a corresponding captured image includes: performing a tenth front-end processing on the R frame in the original image to obtain a fourteenth intermediate image of the R frame; Using the third night scene algorithm, processing the fourteenth intermediate image of the R frame to obtain a fifteenth intermediate image of the R frame; Perform the eighth back-end processing on one frame of the fifteenth intermediate image to obtain a corresponding captured image.

19. The image processing method according to claim 17 or 18, characterized in that: Processing the R frame in the original image using the third MFNR algorithm to obtain a corresponding captured image includes: performing an eleventh front-end processing on the R frame in the original image to obtain a sixteenth intermediate image of the R frame; Processing the sixteenth intermediate image of the R frame using the third MFNR algorithm to obtain a seventeenth intermediate image of the R frame; The ninth back-end processing is performed on the seventeenth intermediate image of one frame to obtain a corresponding captured image.

20. The image processing method according to any one of claims 17 to 19, characterized in that: Processing the R frame in the original image using a third HDR algorithm to obtain a corresponding captured image includes: performing a twelfth front-end processing on the R frame in the original image to obtain an eighteenth intermediate image of the R frame; Using the third HDR algorithm, processing the eighteenth intermediate image of the R frame to obtain a nineteenth intermediate image; Perform tenth back-end processing on one frame of the nineteenth intermediate image to obtain a corresponding captured image.

21. The image processing method according to any one of claims 4 to 17, characterized in that: When the number of times the background waits for taking photos is greater than N, the original image is processed using a third photo taking algorithm to obtain a corresponding photographed image, including: When the number of times the background waits for taking photos is greater than N, the single-frame algorithm is determined to be the third photo-taking algorithm, and the single-frame algorithm is used to process one frame of the original image to obtain a corresponding captured image.

22. The image processing method according to claim 21, characterized in that: The third front-end processing to the twelfth front-end processing are respectively: at least one of bad pixel correction, RAW domain noise reduction, black level correction, lens shading correction, and automatic white balance.

23. The image processing method according to claim 21 or 22, characterized in that: The first back-end processing to the tenth back-end processing respectively include at least one of distortion correction, YUV domain noise reduction, brightness adjustment, color processing, and image scaling.

24. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device performs the image processing method according to any one of claims 1 to 23.

25. A chip, characterized in that: include: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes the image processing method according to any one of claims 1 to 23.

26. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to perform the image processing method according to any one of claims 1 to 23.

Images