Image processing method and electronic equipment

Abstract

The embodiment of the invention provides an image processing method and electronic equipment. The method comprises the steps that when a photographing interface is opened, environment detection is conducted on a photographing scene, and according to an environment detection result, image processing of N stages and an image processing algorithm of each stage are determined to be conducted on an image. When the scene state detection result is a meaningless scene, the image processing of the N stages comprises the first stage of image processing; when the scene state detection result is a weak texture scene, the image processing of the N stages comprises first-stage image processing and second-stage image processing. When the scene detection result is a strong texture scene, the image processing of the N stages comprises first-stage image processing, second-stage image processing and third-stage image processing. The calculation amount of the image processing algorithm in the later stage is greater than that of the image processing algorithm in the previous stage. And after image processing in each stage is completed, storing an image processing result into a memory space allowing the third-party application to access, and storing and displaying the image processing result in an image library.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and electronic device. Background Art

[0002] With the popularity of electronic devices with camera functions in our lives, taking photos with electronic devices has become a daily behavior.

[0003] When people take photos, they're primarily concerned with image quality and performance. Increasing algorithm complexity can improve image quality, but this inevitably leads to performance issues like longer capture times and slow image loading. Conversely, prioritizing performance by pruning and downgrading the algorithm maintains speed but significantly degrades image quality. To address both these requirements, a new approach is urgently needed. Summary of the Invention

[0004] Some implementation methods of the present application provide an image processing method. The present application is introduced from multiple aspects below. The implementation methods and beneficial effects of the following multiple aspects can be referenced to each other.

[0005] In a first aspect, an image processing method is provided, characterized in that it is applied to an electronic device, and the method includes:

[0006] Displaying a first interface, wherein the first interface includes a first control;

[0007] When a first operation on the first control is detected, capturing an image in response to the first operation;

[0008] Performing a first-stage image processing on the acquired image to generate a first image;

[0009] storing the first image in a gallery application;

[0010] When a system state of the electronic device satisfies a first condition, performing a second-stage image processing on the first image to generate a second image; the system state includes at least one of power level, temperature, and memory usage; and the algorithm complexity of the second-stage image processing is greater than the algorithm complexity of the first-stage image processing;

[0011] The second image is stored in the gallery application, and the first image is deleted from the gallery application.

[0012] Optionally, the first operation is a click operation. The first operation may also include a voice instruction operation or other operations that instruct the electronic device to take a photo.

[0013] In a possible implementation of the first aspect, the method further includes: performing environmental detection on the shooting scene;

[0014] According to the result of the environmental detection, N stages of image processing for the image and an image processing algorithm for each stage are determined, where N is an integer greater than 0.

[0015] In this implementation, different scenes can be determined through environmental detection. Then, different algorithms are selected based on the different scenes, so as to adaptively improve the quality and effect of the corresponding captured images in each scene.

[0016] In a possible implementation of the first aspect, the method further includes: the environment detection includes scene state detection;

[0017] When the result of the scene state detection is a meaningless scene, the N stages of image processing include the first stage of image processing;

[0018] When the result of the scene state detection is a weak texture scene, the N stages of image processing include the first stage image processing and the second stage image processing.

[0019] In this implementation, by distinguishing meaningless scenes from weak texture scenes, the number of image processing stages required for subsequent image processing is determined, so that the image processing flow is adapted to the image processing requirements itself, thereby improving the efficiency of image processing.

[0020] In a possible implementation of the first aspect, the method further includes: the environment detection includes illumination detection and dynamic range detection;

[0021] When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, the first stage image processing uses the A1 algorithm and the second stage image processing uses the A2 algorithm;

[0022] When the illuminance is less than the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the first stage image processing uses the B1 algorithm, and the second stage image processing uses the B2 algorithm;

[0023] When the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is less than the dynamic range threshold, the first stage image processing uses the C1 algorithm and the second stage image processing uses the C2 algorithm;

[0024] When the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the first stage image processing uses the D1 algorithm and the second stage image processing uses the D2 algorithm.

[0025] This implementation distinguishes between images with different illumination levels and dynamic ranges, and uses different image processing algorithms within the image processing flow, making the processing more tailored to the image itself and improving the quality of the resulting image. When processing images that include long-exposure, normal-exposure, and short-exposure images, the dynamic range and image detail can be adjusted, resulting in a more realistic image presentation and relatively better image quality.

[0026] In a possible implementation of the first aspect, the method further includes:

[0027] The meaningless scene refers to a shooting scene in which the pixel gradient is less than a first gradient threshold, and the weak texture scene refers to a shooting scene in which the pixel gradient is greater than the first gradient threshold and less than a second gradient threshold.

[0028] In this implementation, the AI ​​scene detection information algorithm determines the category of objects in the shooting scene through semantic recognition, while the traditional scene detection algorithm determines the pixel gradient in the shooting scene. Meaningless scenes and weak-texture scenes are divided based on these two dimensions. When the AI ​​scene detection algorithm cannot determine the category of objects in the image and the calculated pixel gradient is lower than the first gradient threshold, the shooting scene is considered to be a meaningless scene, such as an image blocked by the lens; when the AI ​​scene detection algorithm can determine the category of objects in the image and the calculated pixel gradient is higher than the first gradient threshold and lower than the second gradient threshold, the shooting scene is considered to be a weak-texture scene, such as an image without significant texture such as a wall or solid color.

[0029] In a possible implementation of the first aspect, the method also includes: the A1 algorithm includes a first noise texture algorithm, the A2 algorithm includes a second noise texture algorithm; the PSNR of the image processed by the second noise texture algorithm is higher than the PSNR of the image processed by the first noise texture algorithm; the B1 algorithm includes a third noise texture algorithm, the B2 algorithm includes a fourth noise texture algorithm; the PSNR of the image processed by the fourth noise texture algorithm is higher than the PSNR of the image processed by the third noise texture algorithm; the C1 algorithm includes a fifth noise texture algorithm, the C2 algorithm includes a sixth noise texture algorithm; the PSNR of the image processed by the sixth noise texture algorithm is higher than the PSNR of the image processed by the fifth noise texture algorithm; the D1 algorithm includes a seventh noise texture algorithm, the D2 algorithm includes an eighth noise texture algorithm; the PSNR of the image processed by the eighth noise texture algorithm is higher than the PSNR of the image processed by the seventh noise texture algorithm.

[0030] In this implementation, the second noise texture algorithm includes a gradually increasing number of processing steps compared to the first noise texture algorithm, or in other words, the complexity of the included processing steps gradually increases, and the computational effort gradually increases, thereby gradually increasing the processing time. This allows images with different processing requirements to enter different image processing stages and consume different image processing times. This allows images to achieve better image quality after image processing while also reducing the processing time consumed. The same applies to the relationship between the fourth noise texture algorithm and the third noise texture algorithm, the sixth noise texture algorithm and the fifth noise texture algorithm, and the eighth noise texture algorithm and the seventh noise texture algorithm.

[0031] In a possible implementation of the first aspect, the method further includes: the A1 algorithm includes a first brightness and color algorithm, and the A2 algorithm includes the first brightness and color algorithm; the B1 algorithm includes a second brightness and color algorithm, and the B2 algorithm includes the second brightness and color algorithm; the C1 algorithm includes a third brightness and color algorithm, and the C2 algorithm includes the third brightness and color algorithm; the D1 algorithm includes a fourth brightness and color algorithm, and the D2 algorithm includes the fourth brightness and color algorithm.

[0032] In this implementation, the brightness and color processing parameters for the A1 and A2 algorithms, the B1 and B2 algorithms, the C1 and C2 algorithms, and the D1 and D2 algorithms are consistent. This is intended to prevent users from experiencing significant visual differences when obtaining image results at each stage. This is because, after taking multiple consecutive photos, users generally don't immediately zoom in to view image details; their perception of photos primarily comes from their brightness, color, and composition. Therefore, within the limited time between when a user takes a photo and when they first view it, the brightness and color processing parameters for the different stages of image processing remain consistent, resolving the issue of users experiencing significant differences in image quality resulting from processing.

[0033] In a possible implementation of the first aspect, the method further includes:

[0034] The first condition includes: the power level is greater than a first power level threshold, the temperature is within a first temperature range, and the memory usage is less than a first memory threshold.

[0035] In this implementation scenario, system status detection can determine the current system load. Based on this load, the system then decides whether to proceed to the next stage of image processing. This adaptively improves the efficiency of image algorithm processing under specific system conditions, allowing users to quickly access captured images.

[0036] In a possible implementation of the first aspect, the method further includes:

[0037] When the system state of the electronic device satisfies a second condition, performing a third-stage image processing on the second image to generate a third image;

[0038] The system status includes at least one of power, temperature, and memory usage;

[0039] The second condition is that when the power level is greater than a second power threshold, the temperature is within a second temperature range, and the memory usage is less than a second memory threshold, the computational complexity of the image processing in the third stage is greater than the computational complexity of the image processing in the second stage;

[0040] The third image is stored in the gallery application, and the second image in the gallery application is deleted.

[0041] Optionally, only two power thresholds are set, namely the first power threshold and the second power threshold. The power size can be divided into three intervals through the two power thresholds. When the power meets one of the intervals, it is determined whether to enter the next stage of image algorithm processing and use the algorithm corresponding to the interval to process, so as to obtain the corresponding captured image. In addition, a memory threshold can also be set to divide the memory size into two intervals; or, three, four or more memory thresholds can be set to subdivide the memory size into multiple intervals. The specific settings and modifications can be made as needed. The embodiment of the present application does not impose any restrictions. The same applies to the temperature threshold and memory threshold.

[0042] In a possible implementation of the first aspect, the method further includes: characterized in that the environment detection includes scene state detection; when the scene detection result is a strong texture scene, the N stages of image processing include the first stage image processing, the second stage image processing and the third stage image processing.

[0043] In this implementation, similar to distinguishing meaningless scenes from weak-texture scenes, strong-texture scenes are distinguished, and the number of image processing stages required for subsequent image processing is determined, so that the image processing flow is adapted to the image processing requirements itself and the efficiency of image processing is improved.

[0044] In a possible implementation of the first aspect, the method also includes: the environmental detection includes illuminance detection and dynamic range detection; when the illuminance is less than the illuminance threshold and the dynamic range value is less than the dynamic range threshold, the third stage image processing uses the A3 algorithm; when the illuminance is less than the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the third stage image processing uses the B3 algorithm; when the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is less than the dynamic range threshold, the third stage image processing uses the C3 algorithm; when the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the third stage image processing uses the D3 algorithm.

[0045] In this implementation, images with different illumination and dynamic ranges are distinguished, and different image processing algorithms are used in the image processing flow, so that the image processing is more adapted to the image itself and the image quality of the image processing result is improved.

[0046] In a possible implementation manner of the first aspect, the method further includes: the strong texture scene refers to a shooting scene in which a pixel gradient is greater than a first gradient threshold.

[0047] In this implementation, similar to meaningless scenes and weakly textured scenes, the AI ​​scene detection information algorithm determines the category of objects in the shooting scene through semantic recognition, while the traditional scene detection algorithm determines the pixel gradient in the shooting scene. Strongly textured scenes are divided based on these two dimensions. When the AI ​​scene detection algorithm can determine the category of objects in the image and the calculated pixel gradient is higher than the second gradient threshold, the shooting scene is considered to be a strongly textured scene, such as images with rich textures such as green plants, buildings, portraits, and text.

[0048] In a possible implementation of the first aspect, the method also includes: the A3 algorithm includes a ninth noise texture algorithm, and the PSNR of the image processed by the ninth noise texture algorithm is higher than the PSNR of the image processed by the second noise texture algorithm; the B3 algorithm includes a tenth noise texture algorithm, and the PSNR of the image processed by the tenth noise texture algorithm is higher than the PSNR of the image processed by the fourth noise texture algorithm; the C3 algorithm includes an eleventh noise texture algorithm, and the PSNR of the image processed by the eleventh noise texture algorithm is higher than the PSNR of the image processed by the sixth noise texture algorithm; the D3 algorithm includes a twelfth noise texture algorithm, and the PSNR of the image processed by the twelfth noise texture algorithm is higher than the PSNR of the image processed by the eighth noise texture algorithm.

[0049] In this implementation, the ninth noise texture algorithm includes a gradually increasing number of processing steps compared to the second noise texture algorithm. In other words, the complexity of the included processing steps gradually increases, and the computational effort gradually increases, resulting in a gradual increase in processing time. This allows images with different processing requirements to enter different image processing stages and consume different amounts of image processing time. This allows images to achieve better image quality after image processing while also minimizing the time spent on image processing. The same applies to the relationship between the tenth noise texture algorithm and the fourth noise texture algorithm, the eleventh noise texture algorithm and the sixth noise texture algorithm, and the twelfth noise texture algorithm and the eighth noise texture algorithm.

[0050] In a possible implementation of the first aspect, the method further includes: the A3 algorithm includes the first brightness and color algorithm; the B3 algorithm includes the second brightness and color algorithm; the C3 algorithm includes the third brightness and color algorithm; and the D3 algorithm includes the fourth brightness and color algorithm.

[0051] In this implementation, the brightness and color processing parameters of the A3 and A2 algorithms, the B3 and B2 algorithms, the C3 and C2 algorithms, and the D3 and D2 algorithms are consistent. This is intended to prevent users from experiencing significant visual differences when obtaining image results at each stage. This is because, after taking multiple consecutive photos, users generally don't immediately zoom in to view image details; their perception of photos primarily comes from their brightness, color, and composition. Therefore, within the limited time between when a user takes a photo and when they first view it, the brightness and color processing parameters for the different stages of image processing remain consistent, resolving the issue of users experiencing significant differences in image quality resulting from processing.

[0052] In a possible implementation of the first aspect, the first interface refers to a photographing interface, and the first control refers to a control for instructing photographing.

[0053] In a second aspect, an electronic device is provided, comprising one or more processors and a memory;

[0054] The memory is coupled to the one or more processors, and the memory is used to store computer program code, wherein the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method provided in the above first aspect or any possible implementation of the first aspect.

[0055] In a third aspect, a chip is provided, which is applied to an electronic device, and the chip includes one or more processors, and the processors are used to call computer instructions to enable the electronic device to execute the method provided in the first aspect or any possible implementation of the first aspect.

[0056] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes the method provided in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a schematic diagram of an application scenario provided by the embodiment of the present application

[0058] Figure 2 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0059] Figure 3 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0060] Figure 4 This is a schematic diagram of an image processing process provided by an embodiment of the present application;

[0061] Figure 5 This is a flow chart of a method for image processing in stages provided in an embodiment of the present application;

[0062] Figure 6 This is a flow chart of an environmental detection process provided by an embodiment of the present application;

[0063] Figure 7 is a schematic diagram of an image processing flow at different stages provided by an embodiment of the present application;

[0064] Figure 8 This is a schematic diagram of the number of image processing stages under different system states provided by an embodiment of the present application;

[0065] Figure 9 This is a schematic diagram of an algorithm used in each stage under different environmental conditions provided by an embodiment of the present application;

[0066] Figure 10 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0067] Figure 11 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solution in this application will be described below with reference to the accompanying drawings.

[0069] In the description of the embodiments of this application, unless otherwise specified, " / " represents or. For example, A / B can represent A or B. "And / or" in this article is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "plurality" means two or more than two.

[0070] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0071] With the popularity of electronic devices with camera functions in our lives, taking photos with electronic devices has become a daily behavior.

[0072] When most electronic devices perform image processing, in order to achieve better photo quality, the image often needs to be processed by multiple algorithm modules. The current main algorithm processing categories include brightness, color processing, noise, and texture processing. When performing brightness and color processing on an image, it is not necessary to process every pixel of the image, the amount of processing data is small, and the processing speed is generally in the millisecond level. When performing noise and texture processing on an image, every pixel needs to be processed, the amount of processing data is large, and the processing time is long, ranging from 1-3 seconds to more than 10 seconds.

[0073] During the image processing process of an electronic device, when the sensor captures multiple images, the electronic device first performs algorithmic processing on the first image and then outputs the processed image. It then performs algorithmic processing on the next image and then outputs the processed image. If the user triggers multiple photo commands in succession, and the interval between triggering photos is short, the sensor captures multiple images in a short period of time. Based on the above image processing logic, the electronic device will not be able to respond to the processing needs of subsequent images in a timely manner due to the long processing time of each image, resulting in a gradual increase in the number of images waiting to be processed in the background. As the number of waiting photos increases, the amount of data to be processed accumulated in the background also increases, continuously expanding the memory usage. When there is insufficient memory to support single-photo image processing, the image processing process is forced to terminate, and the device cannot output the processed image and display it in the gallery. This makes it impossible for users to quickly access and view the latest captured images, resulting in a poor user experience.

[0074] To address this issue, the primary solution currently employed is to categorize multiple algorithms into multiple levels based on processing time. This allows adaptive selection of algorithms from different levels based on the current memory usage. For example, when memory usage is low, algorithms with longer processing times are selected, while algorithms with shorter processing times are selected as memory usage increases. This reduces the pressure on electronic devices to increase memory usage, improves photo processing efficiency, and allows for timely response to user photo requests, enabling continuous and rapid photo taking.

[0075] While the above solution effectively reduces the image processing time for this capture, the resulting image quality varies due to the varying complexity of the algorithms used. For example, after capturing and processing image 1, if the memory usage has not yet reached a preset threshold, the full AIRAW algorithm and YUV domain algorithm are used for image processing. Shortly thereafter, image 2 is captured and processed. Because image 1 may still be processing, the memory usage exceeds the preset threshold, and image 2 is processed using the YUV domain algorithm. The user ultimately receives processed image 1 and processed image 2, which ultimately produce different visual effects due to the different algorithms, resulting in inconsistent image quality on the device. For the user, the image quality of the photos captured before and after the first image is significantly different in the gallery, resulting in a poor user experience. The more complex algorithm (referred to as the second stage of image processing) generates the second image, which is then displayed in the gallery app, and the first image is deleted. This ensures that the image processing results are promptly returned to the user while ensuring the consistency and superior quality of the final image processing results displayed in the gallery. Achieve a balance between photo performance and image quality.

[0076] It is understandable that the above solution may also include third stage, fourth stage, etc. image processing, and the embodiment of the present application does not limit this.

[0077] The photographing method provided in the embodiments of the present application can be applied to various electronic devices.

[0078] In some embodiments of the present application, the electronic device may be various camera devices such as sports cameras and digital cameras, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc., or may be other devices or apparatuses capable of displaying images. The embodiments of the present application do not impose any restrictions on the specific type of electronic device.

[0079] The following description uses a mobile phone as an example electronic device. The mobile phone may include a display screen and one or more cameras. In a given shooting scenario, the mobile phone uses one or more cameras to capture images with different characteristics. The display screen is used to display the processed images.

[0080] Based on this, combined Figure 1 The application scenarios provided by the embodiments of this application are introduced.

[0081] like Figure 1 As shown in (a) of FIG, the electronic device 100 is installed with a camera application. In addition, the electronic device 100 may also be installed with multiple applications, and the present application embodiment does not impose any restrictions on this. For example, in response to a user's click operation on the camera application, when the electronic device 100 runs the camera application, the electronic device 100 displays the following Figure 1 The shooting interface shown in (b).

[0082] The shooting interface includes multiple shooting modes of the camera application, such as large aperture mode 41, night scene mode 42, portrait mode 43, photo mode 44, video mode 45, etc. The shooting interface also includes a first control, which is a shooting key 50. The shooting key 50 is used to trigger the camera to take a photo. When the user clicks the shooting key 50 once, the camera takes a photo.

[0083] like Figure 1 As shown in (b) of FIG. , the shooting interface also includes a viewfinder 60, which can be used to display a preview image before shooting in real time. The shooting interface also displays a second control, which is a zoom option 61. The user can select the currently desired zoom factor in zoom option 61, for example, 0.5x, 2x, or 50x.

[0084] Combine Figure 1As shown in (a) and (b), when the user turns on the camera and the current shooting mode defaults to the photo mode, after the user points the camera on the mobile phone at the shooting scene, the viewfinder 60 can display the preview image before taking the photo in real time; then, in response to the user clicking the shooting key 50, the electronic device can call the image processing method provided in this application to perform phased algorithm module processing on the image.

[0085] like Figure 1 As shown in (b) of FIG. 4 , when the user selects portrait mode 43, the process is similar to the above-mentioned photo mode 44. After the user points the camera on the mobile phone at the scene to be photographed, the viewfinder 60 can display a preview image before the photo is taken in real time. Then, in response to the user clicking the capture button 50, the electronic device can invoke the image processing method provided in this application to perform phased algorithm module processing on the image.

[0086] Figure 2 A hardware system of an electronic device suitable for the present application is shown.

[0087] The electronic device 100 can be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, etc. The embodiment of the present application does not impose any restrictions on the specific type of the electronic device 100.

[0088] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0089] Among them, the sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0090] It should be noted that Figure 2 The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 2 More or fewer components than those shown, or the electronic device 100 may include Figure 2 Combinations of some of the components shown, or alternatively, the electronic device 100 may include Figure 2 Subassemblies of some of the components shown. Figure 2 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0091] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices. The controller may generate an operation control signal based on the instruction opcode and timing signal to complete the control of instruction fetching and execution.

[0092] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0093] Figure 2The connection relationship between the modules shown is only for illustrative purposes and does not limit the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of the multiple connection modes in the above embodiments.

[0094] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0095] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 is also used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance).

[0096] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0097] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0098] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0099] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0100] The camera 193 is used to capture still images or videos. It can be triggered to turn on via application instructions to implement the camera function, such as capturing images of any scene. The camera may include components such as an imaging lens, a filter, and an image sensor. Light emitted or reflected by an object enters the imaging lens, passes through the filter, and ultimately converges on the image sensor. The imaging lens is primarily used to focus and image the light emitted or reflected by all objects in the photographic field of view (also referred to as the shooting scene, target scene, or the scene image that the user expects to capture); the filter is primarily used to filter out excess light waves (for example, light waves other than visible light, such as infrared); and the image sensor may be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The image sensor is primarily used to perform photoelectric conversion on the received light signal, converting it into an electrical signal, and then transmitting the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

[0101] Among them, the camera 193 can be located in the front of the electronic device 100 or in the back of the electronic device 100. The specific number and arrangement of the cameras can be set according to needs, and this application does not impose any restrictions.

[0102] Exemplarily, the electronic device 100 includes a front camera and a rear camera. For example, the front camera or the rear camera may include one or more cameras. Taking the electronic device 100 having four rear cameras as an example, when the electronic device 100 activates the four rear cameras to take pictures, the photo-taking method provided in the embodiment of the present application can be used. Alternatively, the camera is set on an external accessory of the electronic device 100, and the external accessory is rotatably connected to the frame of the mobile phone, and the angle formed between the external accessory and the display screen 194 of the electronic device 100 is any angle between 0-360 degrees. For example, when the electronic device 100 takes a selfie, the external accessory drives the camera to rotate to a position facing the user. Of course, when the mobile phone has multiple cameras, only some of the cameras can be set on the external accessories, and the remaining cameras can be set on the electronic device 100 body. The embodiment of the present application does not impose any restrictions on this.

[0103] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0104] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0105] The internal memory 121 can also store the software code of the photographing method provided in the embodiment of the present application. When the processor 110 runs the software code, the process steps of the photographing method are executed to achieve fast and continuous photographing.

[0106] The internal memory 121 may also store captured images.

[0107] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music can be stored in the external memory card.

[0108] Of course, the software code of the photographing method provided in the embodiment of the present application can also be stored in an external memory, and the processor 110 can run the software code through the external memory interface 120 to execute the process steps of the photographing method to obtain multiple frames of captured images. The captured images obtained by the electronic device 100 can also be stored in an external memory.

[0109] It should be understood that the user can specify whether to store the image in the internal memory 121 or the external memory. For example, when the electronic device 100 is connected to the external memory, if the electronic device 100 captures a frame of image, a prompt message may pop up to prompt the user to store the image in the external memory or the internal memory. Of course, other designation methods are possible, and the present embodiment does not impose any restrictions on this. Alternatively, when the electronic device 100 detects that the memory capacity of the internal memory 121 is less than a preset amount, it may automatically store the image in the external memory.

[0110] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0111] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0112] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or touch vibration feedback. For example, touch operations on different applications (such as taking photos, playing audio, etc.) can correspond to different vibration feedback effects.

[0113] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0114] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting or removing the SIM card into or from the SIM card interface 195 .

[0115] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0116] The hardware system of the electronic device 100 is described in detail above. The software system of the electronic device 100 is introduced below.

[0117] Figure 3 Schematic diagram of the software system of the electronic device provided in an embodiment of the present application.

[0118] like Figure 3 As shown, the system architecture may include an application layer (APP) 210 , an application framework layer 220 , a hardware abstract layer (HAL) 230 , a driver layer 240 and a hardware layer 250 .

[0119] The application layer 210 may include a camera application or other applications. Other applications include but are not limited to: camera, gallery, and other applications.

[0120] The application layer 210 is at the top of the entire framework and is responsible for interacting directly with the user. Once it receives a direct or indirect request from the user, such as taking a photo, it will send the request to the application framework layer 220 through the interface and wait for the application framework layer 220 to return the processing result, for example, the result includes image data and camera parameters; then the application layer 210 will feed back the result to the user.

[0121] The application framework layer 220 may provide an application programming interface (API) and a programming framework to the application programs of the application layer; the application framework layer may include some predefined functions.

[0122] For example, the application framework layer 220 may include a camera access interface; the camera access interface may include camera management and camera equipment; wherein, camera management may be used to provide an access interface for managing the camera; and camera equipment may be used to provide an interface for accessing the camera.

[0123] The hardware abstraction layer 230 is used to abstract hardware. For example, the hardware abstraction layer may include a camera hardware abstraction layer and other hardware device abstraction layers. The camera hardware abstraction layer may include camera device 1, camera device 2, and so on. Camera device 1 and camera device 2 are abstract camera hardware, not physical camera devices. The camera hardware abstraction layer can be connected to the camera algorithm library, and the camera hardware abstraction layer can call the algorithms in the camera algorithm library.

[0124] In this application, the perception engine used to perform various detections can be set in the hardware abstraction layer.

[0125] The driver layer 240 is used to provide drivers for different hardware devices. For example, the driver layer may include a camera device driver, a digital signal processor driver, and a graphics processor driver.

[0126] The hardware layer 250 may include multiple image sensors, multiple image signal processors, digital signal processors, graphics processors, and other hardware devices.

[0127] For example, the hardware layer 250 includes sensors and image signal processors. The sensors may include sensor 1, sensor 2, a depth sensor (time of flight, TOF), a multispectral sensor, etc. The image signal processors may include image signal processor 1, image signal processor 2, etc.

[0128] In this application, by calling the hardware abstraction layer interface in the hardware abstraction layer 230, the connection between the application layer 210 and the application framework layer 220 above the hardware abstraction layer 230 and the driver layer 240 and the hardware layer 250 below can be realized, thereby realizing camera data transmission and function control.

[0129] In the camera hardware interface layer within the hardware abstraction layer 230, manufacturers can customize functionality based on their needs. Compared to the hardware abstraction layer interface, the camera hardware interface layer is more efficient, flexible, and has lower latency. It also allows for more comprehensive access to the ISP and GPU for image processing. The image input to the hardware abstraction layer 230 can come from an image sensor or a stored image.

[0130] The scheduling layer in the hardware abstraction layer 230 includes a general functional interface for implementing management and control.

[0131] The camera service layer in the hardware abstraction layer 230 is used to access the ISP and other hardware interfaces.

[0132] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0133] The camera application in the application layer can be displayed as an icon on the screen of the electronic device 100. When the camera application icon is clicked by the user to trigger it, the electronic device 100 starts running the camera application. When the camera application is running on the electronic device 100, the camera application calls the corresponding interface of the camera application in the application framework layer 210, then starts the camera driver by calling the hardware abstraction layer 230, turns on the camera 193 on the electronic device 100, and simultaneously the camera algorithm library starts loading the photography method used in the embodiment of the present application.

[0134] For example, the camera application of the application layer 210 can generate a shooting instruction in response to the user's photo-taking or recording operation, and transmit the shooting instruction to the hardware layer 250 through the application framework layer 220, the hardware abstraction layer 230, and the driver layer 240. After the hardware layer 250 sensor receives the shooting instruction, after obtaining one or more frames of images through the image sensor, the obtained images can be processed by the image signal processor (ISP) and returned to the hardware abstraction layer 230. In the hardware abstraction layer 230, one or more algorithms in the camera algorithm library can be used to perform one or more stages of image processing on the image to obtain a processed image. The processed image is passed to the gallery application of the application layer 210 through the application framework layer 220 for storage. Subsequently, the gallery can display the captured image on the display screen in response to user clicks and other operations.

[0135] The process of performing one or more stages of image processing on an image using one or more algorithms in the camera algorithm library will be described in detail in subsequent embodiments.

[0136] Figure 4 This is a flow chart of an image processing method provided by the present application, which is applied to electronic devices including cameras, such as mobile phones.

[0137] like Figure 4 As shown, in response to the user clicking the photo control of the camera application, the electronic device controls the camera to capture an image, and the image sensor in the camera obtains the image captured by the lens.

[0138] The image can be in RAW format. Unlike common JPEG and PNG image formats, RAW images are uncompressed or processed and contain the raw data captured by the camera sensor. To view and process RAW images, you typically need to convert them to a common image format, such as RGB.

[0139] The captured RAW image is input to the ISP module of the hardware layer 250 for image processing. The ISP includes an IFE module and a BPS module. For example, the BPS module and the IPE module each represent a hardware processing unit within the chip, which stores various image processing algorithms.

[0140] For example, the IFE module performs color correction, downsampling, demosaicing and statistical 3A data processing for preview and video image data. This technical solution mainly focuses on image processing in photo mode, and the functions of the IFE module are not detailed here.

[0141] For example, the BPS module preprocesses the image data using multiple algorithm modules, including default pixel correction (DPC), black level correction (BLC), demosaicing, auto white balance (AWB), and color correction matrix (CCM).

[0142] Among them, bad pixel correction refers to: solving the defects in the array formed by the light collection points on the sensor, or the incorrect brightness values ​​in the process of converting the light signal, by taking the average of other surrounding pixels in the brightness value range.

[0143] Black level correction means that due to the presence of dark current in the image sensor, the pixels also have a certain output voltage when there is no light, and pixels at different positions may correspond to different output voltages. Therefore, it is necessary to correct the output voltage corresponding to the pixels when there is no light (i.e., black).

[0144] De-mosaicing means that since each pixel in the RAW domain image only corresponds to the color information of one channel, the information of surrounding pixels can be used to estimate other colors. For example, through linear interpolation, the color information of the other two channels missing for each pixel can be determined, thereby restoring all channel information of all pixels in the image.

[0145] Automatic white balance means: in order to eliminate the influence of light source on image sensor imaging, simulate the color constancy of human vision, and ensure that the white seen in any scene is truly white, it is necessary to correct the color temperature and automatically adjust the white balance to the appropriate position.

[0146] Color correction means that since the image captured by the camera may not match the expected color, it is necessary to correct the color. Since the automatic white balance has already calibrated the white color, color correction can be used to calibrate other colors besides white.

[0147] After BPS preprocessing, the image is then fed into the hardware abstraction layer 230's image algorithm library for further processing. This internal algorithm module further enhances image clarity, texture detail, tonality, color, and sharpness. The algorithm modules involved are primarily categorized as the AIRAW algorithm module and the YUV domain algorithm module.

[0148] The AIRAW algorithm module is an end-to-end algorithm module from the RAW domain to the YUV domain, which encapsulates many image processing algorithm sub-modules, including noise reduction (NR), tone mapping (TM), and color space conversion (CSC).

[0149] For example, noise reduction refers to Raw domain noise reduction. Raw domain noise reduction is used to reduce noise in images. The noise in the image will affect the user's visual experience, and noise reduction can improve the image quality to a certain extent. Tone mapping refers to mapping all pixels in the image with a space-invariant mapping function. First, the average brightness of the scene should be calculated based on the current scene, and then a suitable brightness domain should be selected based on this average brightness, and then the entire scene should be mapped to this brightness domain to obtain the correct result. Color space conversion: convert the image from the RGB domain to the YUV domain. In this way, the image obtained after processing in the previous step can be converted from the RGB domain to the YUV domain to reduce the amount of data stored and transmitted subsequently and save bandwidth.

[0150] The YUV domain algorithm module is an algorithm module that supports YUV format image processing in the YUV domain, including: Noise Reduction for Luma (NR Luma), Noise Reduction for Chroma (NR Chroma), Edge Enhancement, etc.

[0151] For example, YUV is an image encoding method. YUV color encoding uses brightness and chroma to specify the color of a pixel. Y represents brightness (luminance), while U and V represent chroma. Chroma defines two aspects of color: hue and saturation. Each pixel contains Y, U, and V components. However, its Y and UV components can be separated. Without the UV component, the complete image can still be displayed, but in black and white.

[0152] Luminance noise reduction: Denoise is performed on both luminance (Luma) and chroma (Chroma) in the YUV domain. Luminance noise in the Y domain represents luminance, while the UV component represents color, which is chroma noise in the UV domain. Luminance noise reduction ignores color. Denoise is performed based on a luminance noise map analysis.

[0153] Color noise reduction: Targets the noise of the UV component in the V domain, specifically the color fluctuations between pixels and the noise of the chroma part.

[0154] Edge enhancement: The previous noise reduction inevitably removes some image details, resulting in image blur. To restore image details and minimize image loss, it is necessary to enhance them without reintroducing noise. This is where edge enhancement comes in.

[0155] Furthermore, the image output by the image algorithm library can be input into the JPEG module for JPEG encoding, which then outputs a JPEG format image. Because images are typically stored and displayed in JPEG format on electronic devices, the YUV-format image obtained through YUV domain algorithm processing needs to be JPEG-encoded to obtain a JPEG format image.

[0156] Finally, the JPEG format image output by the JREG encoder can be stored in the gallery. Subsequently, the JPEG format image can be displayed in the gallery in response to a user's operation such as clicking on the gallery application.

[0157] For example, Figure 4 The illustrated modules do not constitute a specific limitation on the image processing method of this application. Figure 4 The image processing modules shown are only functional modules. The image processing method of the present application may include more or fewer modules than shown in the figure, or combine some modules, split some modules, or arrange the modules differently. The modules shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0158] from Figure 4 It can be seen that for an image captured by a camera, it needs to go through multiple modules and multiple algorithm processing to obtain a processed image with good image quality. In order to meet the higher requirements for image quality today, Figure 4 On top of this, the AIRAW algorithm module typically adds optimized image processing results from algorithmic submodules. For example, the AI ​​HDR fusion algorithm, which provides refined denoising while preserving more image detail, typically takes 400ms; and the AIGC super-resolution algorithm, which improves image clarity while creating a more natural and detailed visual effect, typically takes 5000ms.

[0159] While the introduction of the aforementioned algorithm yields higher-quality processed images, it also results in an overly lengthy image processing process, failing to meet users' needs for quick photo capture. In embodiments of the present application, the number of image processing stages required for an image in a given shooting scene, as well as the algorithms used in each stage, can be determined based on the scene's brightness and dynamic range information. For example, the corresponding image processing algorithm is determined based on the scene's brightness and dynamic range information. The number of image processing stages required is then determined based on the image's scene state information. Images with complex texture details require more stages of image processing, each using a specific algorithm; images with simple texture details require fewer stages of image processing. The image undergoes a first stage of image processing to produce a first image, which is then output and displayed in the gallery application, allowing users to promptly access the results of the photo. When the system load meets certain conditions, the image undergoes a second stage of image processing to produce a second image, which is then displayed in a thumbnail box and in the gallery application, and the first image is deleted. The image processing algorithm in this next stage is more complex and time-consuming than the previous stage's image processing algorithm, resulting in a higher-quality image. This solution enables users to obtain the latest captured images in a short time while also ensuring that the final image processing results have good image quality.

[0160] Figure 5 This is a flow chart of a method for processing an image in stages provided by this application. This method is applied to electronic devices including cameras, such as mobile phones. Figure 5 As shown, the steps of the method may include the following S11 to S23.

[0161] S11. The electronic device displays a first interface, where the first interface includes a first control.

[0162] Alternatively, as Figure 1 As shown in (b), the first interface can be a photo mode

[44] of the shooting interface, and the first control is used to indicate the shooting key 50 in the shooting interface.

[0163] Alternatively, as Figure 1 As shown in (b), the first interface may be a portrait mode

[44] of a shooting interface, and the first control is used to indicate a shooting key 50 in the shooting interface.

[0164] Of course, the first interface may also be other interfaces, and the first control may correspondingly be a control on other interfaces for instructing taking photos. This embodiment of the present application does not impose any restrictions on this.

[0165] S12: Performing environmental detection on the shooting scene, and determining an image processing method according to the environmental detection result.

[0166] The "shooting scene" refers to the scene that the user desires to capture. If a user uses the camera of an electronic device to point at a scene containing an object, the scene containing the object is considered the shooting scene. It should be understood that the shooting scene does not refer to a specific scene, but rather to the scene that the camera is pointing at in real time.

[0167] Among them, environmental detection includes at least one of illumination value (LV) detection, dynamic range detection and scene detection. Of course, environmental detection can also include other detection items, which can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0168] For example, Figure 6 A schematic diagram of a process flow of environmental detection provided in an embodiment of the present application. Figure 6 As shown, the above S12 may include the following S131 to S144.

[0169] S131 , performing illumination detection and dynamic range detection on the captured image.

[0170] It should be understood that illumination detection refers to estimating the ambient brightness in the shooting scene. Illumination refers to the degree to which an object is illuminated, specifically refers to the energy of visible light received per unit area of ​​the object, referred to as illumination, and the unit is lux. Streets at night, dimly lit rooms, etc. are generally low illumination. In contrast, playgrounds under the sun, stages under spotlights, etc. can be called medium and high illumination. If the illumination of the shooting scene is relatively low, when using an electronic device to shoot the shooting scene, it will be blurry and the effect will be poor. If the illumination of the shooting scene is relatively high, when using an electronic device to shoot the shooting scene, it will be clearer and the shooting effect will be good.

[0171] Here, for example, a photoelectric sensor can be set in the electronic device to detect the illumination of the shooting scene. Alternatively, the illumination can be calculated using the following formula based on the camera's exposure parameters, such as exposure time, sensitivity, aperture, and other parameters, and the corresponding values ​​received:

[0172] Among them, Exposure is the exposure time, Aperture is the aperture size, ISO is the sensitivity, and Luma is the average value of Y in the XYZ color space.

[0173] When the exposure parameters are the same, a higher response value indicates a higher illumination of the shooting scene, and thus a larger calculated illumination value.

[0174] It should be understood that dynamic range detection refers to the detection of the dynamic range value of the captured scene. The dynamic range value is used to represent the brightness range of the pixels in the image, that is, the number of grayscale levels between the "brightest" pixel and the "darkest" pixel in the image. The larger the dynamic range value of an image, the richer the brightness levels it can represent and the more realistic the visual effect of the image. The expression of the dynamic range value can be:

[0175] Where dynamic range is the dynamic range value, bright is the brightness of the "brightest" pixel, and dark is the brightness of the "darkest" pixel. The unit of dynamic range is stop.

[0176] Here, based on the detected dynamic range value, it can be determined whether the captured scene is a low dynamic range (LDR) scene or a high dynamic range (HDR) scene. A low dynamic range scene refers to a scene with either low or high ambient light intensities, resulting in a narrow dynamic range; a high dynamic range scene refers to a scene with low light intensities in some areas and high light in others, resulting in a wide dynamic range.

[0177] S132: Determine whether the illumination is less than an illumination threshold.

[0178] Among them, the illumination threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0179] S133: Determine whether the detected dynamic range value is less than a dynamic range threshold.

[0180] The dynamic range threshold can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0181] S134 : When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, or when the shooting scene is a low-brightness, low-dynamic-range scene, determine to use the A1 algorithm in the first stage image processing.

[0182] S135 : When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, or when the shooting scene is a low-brightness, high-dynamic-range scene, determine to use the B1 algorithm in the first stage of image processing.

[0183] S136 : When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, or when the shooting scene is a high-brightness, low-dynamic-range scene, determine to use the C1 algorithm in the first-stage image processing.

[0184] S137 : When the illumination is greater than or equal to the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, or when the shooting scene is a high-brightness, high-dynamic-range scene, determine to use the D1 algorithm in the first stage of image processing.

[0185] It should be understood that A1, B1, C1 and D1 are used to refer to the names of algorithms. A1 algorithm, B1 algorithm, C1 algorithm and D1 algorithm can indicate the same algorithm or different algorithms. They can be set and modified as needed. The embodiments of this application do not impose any restrictions on this.

[0186] In other words, illumination detection and dynamic range detection divide the captured scene into four categories: low-light, low-dynamic-range scenes, low-light, high-dynamic-range scenes, high-light, low-dynamic-range scenes, high-light, and high-dynamic-range scenes. This determines the image processing algorithm used in each image processing stage. For example, if the illumination value is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, the A1 algorithm is used in the first image processing stage. The A2 algorithm is used in the second image processing stage, and the A3 algorithm is used in the third image processing stage.

[0187] The following uses low-light, low dynamic range scenes as an example to illustrate the process of taking photos and performing further scene detection:

[0188] S138 detects the scene status of the shooting scene

[0189] It should be understood that scene state detection refers to the combination of traditional scene detection algorithms and AI scene detection information algorithms. AI scene detection algorithms may include semantic recognition algorithms that classify all pixels in the image, assigning pixels of the same category to the same label, and then use the model to calculate the category of objects contained in the image. Traditional scene detection algorithms may include filtering operations to filter out the high-frequency parts of the captured photo, and then calculate the pixel gradient of the high-frequency part. The calculation formula for pixel gradient is as follows:

[0190] f(x) represents the pixel value at a certain position on the image, f(x-△x) represents the pixel value of a neighboring pixel, and f'(x) is the image gradient value at that position.

[0191] When the AI ​​scene detection algorithm cannot determine the category of objects in the image and the calculated pixel gradient is lower than the first gradient threshold, the shooting scene is considered to be a meaningless scene, such as an image blocked by the lens; when the AI ​​scene detection algorithm can determine the category of objects in the image and the calculated pixel gradient is higher than the first gradient threshold and lower than the second gradient threshold, the shooting scene is considered to be a weak-texture scene, such as images without significant textures such as walls or solid colors; when the AI ​​scene detection algorithm can determine the category of objects in the image and the calculated pixel gradient is higher than the second gradient threshold, the shooting scene is considered to be a strong-texture scene, such as images with rich textures such as green plants, buildings, portraits, and text.

[0192] It should be understood that the method of distinguishing meaningless scenes, weak texture scenes, and strong texture scenes in the above scene detection is only an example, and the present application embodiment does not impose any limitation on this.

[0193] At S138 , an environmental detection result of the captured scene is obtained. Based on the detection result, it is determined that N stages of image processing are required, where N is an integer greater than 0. The logic for determining the number of image processing stages based on the environmental detection results is described below, using three examples where the environmental detection results are a meaningless scene, a weakly textured scene, and a strongly textured scene.

[0194] When the detection result is a meaningless scenario:

[0195] S139, end image processing. It should be understood that in a scenario without practical significance, the image processing flow ends after the first stage of algorithm processing.

[0196] When the detection result is a weak texture scene:

[0197] S140: Determine to use the A2 algorithm in the second stage of image processing.

[0198] S141, end image processing. It should be understood that in a weak texture scene, the image processing flow ends after the first and second stage algorithm processing.

[0199] When the detection result is a strong texture scene:

[0200] S142: Determine to use the A3 algorithm in the third stage of image processing.

[0201] S143, end image processing. It should be understood that in a strong texture scene, the image processing flow ends after the first stage, the second stage algorithm, and the third stage algorithm processing.

[0202] It should be understood that A2 and A3 are used to refer to the names of algorithms, which are used to refer to the image processing algorithms of each stage in the staged image processing process. Compared with the A3 algorithm, the processing steps of the A2 algorithm are gradually reduced, or the complexity of the processing steps included is gradually reduced, so that the processing time is gradually reduced. Compared with the A2 algorithm, the processing steps of the A1 algorithm are gradually reduced, or the complexity of the processing steps included is gradually reduced, so that the processing time is gradually reduced. Assuming that the processing time of the A1 algorithm is T1, the processing time of the A2 algorithm is T2, and the processing time of the A3 algorithm is T3, the difference between T3-T2 and the difference between T2-T1 can be the same or different, and can be set and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0203] When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, or when the scene is low-light and high-dynamic-range, the subsequent image processing logic is the same as for low-light and low-dynamic-range scenes. The B2 algorithm is used in the second stage of image processing, and the B3 algorithm is used in the third stage of image processing.

[0204] It should be understood that B2 and B3 are used to refer to the names of algorithms, which are used to refer to the image processing algorithms of each stage in the staged image processing process. Compared with the B3 algorithm, the B2 algorithm has gradually fewer processing steps, or the complexity of the processing steps included is gradually reduced, so that the processing time is gradually reduced. Compared with the B2 algorithm, the B1 algorithm has gradually fewer processing steps, or the complexity of the processing steps included is gradually reduced, so that the processing time is gradually reduced. Assuming that the processing time of the B1 algorithm is T1, the processing time of the B2 algorithm is T2, and the processing time of the B3 algorithm is T3, the difference between T3-T2 and the difference between T2-T1 can be the same or different, and can be set and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0205] When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, or in other words, when the scene is high-brightness and low dynamic range, the subsequent image processing logic is the same as for low-brightness and low dynamic range scenes. The C2 algorithm is used in the second stage of image processing, and the C3 algorithm is used in the third stage of image processing.

[0206] It should be understood that, compared to the C3 algorithm, the C2 algorithm has a gradually reduced number of processing steps, or in other words, the complexity of the processing steps included is gradually reduced, thereby gradually reducing the processing time. Compared to the C2 algorithm, the C1 algorithm has a gradually reduced number of processing steps, or in other words, the complexity of the processing steps included is gradually reduced, thereby gradually reducing the processing time. Assuming that the processing time of the C1 algorithm is T1, the processing time of the C2 algorithm is T2, and the processing time of the C3 algorithm is T3, then the difference between T3-T2 and the difference between T2-T1 can be the same or different, and can be specifically set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0207] When the illumination is greater than or equal to the illumination threshold and the dynamic range is greater than or equal to the dynamic range threshold, or in other words, when the scene is bright and has a high dynamic range, the subsequent image processing logic is the same as for low-light and low dynamic range scenes. The D2 algorithm is used in the second stage of image processing, and the D3 algorithm is used in the third stage of image processing.

[0208] It should be understood that, compared to the D3 algorithm, the D2 algorithm has a gradually reduced number of processing steps, or in other words, the complexity of the processing steps included is gradually reduced, thereby gradually reducing the processing time. Compared to the D2 algorithm, the D1 algorithm has a gradually reduced number of processing steps, or in other words, the complexity of the processing steps included is gradually reduced, thereby gradually reducing the processing time. Assuming that the processing time of the D1 algorithm is T1, the processing time of the D2 algorithm is T2, and the processing time of the D3 algorithm is T3, then the difference between T3-T2 and the difference between T2-T1 can be the same or different, and can be specifically set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0209] It should be noted that the above sequence of S131 to S147 is only one example. When performing environmental detection, it is also possible to first perform illuminance detection on the shooting scene to determine whether the detected illuminance is less than the illuminance threshold, and then perform dynamic range detection on the shooting scene to determine whether the detected dynamic range value is less than the dynamic range threshold. Alternatively, the order of the two steps can be reversed. The specific execution process can be adjusted as needed. In addition, other steps can be added or some steps can be deleted on this basis. The embodiments of the present application do not impose any restrictions on this.

[0210] After the above-mentioned environmental detection steps, the electronic device can determine that the image will undergo N stages of image processing, and simultaneously determine the image processing algorithm used in each stage, where N is an integer greater than 0.

[0211] S13: When a first operation of the user on the first control is detected, the electronic device uses a camera to capture an image in response to the first operation.

[0212] For example, Figure 2 As shown, the user issues a start instruction, which can transmit the shooting instruction to the hardware layer 250 through the application layer 210, application framework layer 220, hardware abstraction layer 230, and driver layer 240. After receiving the start instruction, the image sensor obtains the real-time image captured by the lens.

[0213] Optionally, the first operation may be a click operation on the first control, or may be a voice instruction operation or other operation to instruct the electronic device to take a photo, and the embodiment of the present application does not impose any limitation on this.

[0214] The click operation refers to the behavior of a user touching the first control for a short time and then leaving the control.

[0215] When an electronic device includes multiple cameras, one or more cameras may be used to capture an image in response to a single click operation, and each camera may capture one or more frames of images. In other words, one or more frames of images may be captured each time, and the specific number of frames may be set and modified as needed, and this embodiment of the present application does not impose any limitation thereto.

[0216] It should also be understood that the image may be an image in the RAW domain, but when the camera is a black and white camera, the image may be a grayscale image; when the camera is a multispectral camera, the image may be a multispectral image including multiple color channel signals. The format or characteristics of the image changes with the camera, and the embodiments of the present application do not impose any restrictions on this.

[0217] In addition, the images collected at one time may also include at least one of a long-exposure image, a normal-exposure image, and a short-exposure image.

[0218] Among them, the long-exposure image refers to the image obtained after a longer exposure time during shooting, and the short-exposure image refers to the image obtained after a shorter exposure time during shooting. It should be understood that "long" and "short" are relative to the "normal" exposure time. When the exposure time corresponding to the normally exposed image is different, the long exposure and short exposure will also change accordingly.

[0219] When the multiple-frame images include long-exposure images, and / or normal-exposure images, and / or short-exposure images, the number of frames of the long-exposure images, the number of frames of the normal-exposure images, and the number of frames of the short-exposure images can be selected and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0220] After the sensor acquires the image, the image may be transmitted to the image signal processor first. The image signal processor determines the image processing flow of the image based on the previously acquired environmental detection information.

[0221] Exemplarily, when the illumination detection and dynamic range detection results are low brightness and low dynamics, and the environmental detection result is a scene with no practical significance, image processing is terminated after S14-S16 is performed on the image; when the illumination detection and dynamic range detection results are low brightness and low dynamics, and the environmental detection result is a weak texture scene, image processing is terminated after S14, S17-S21 is performed on the image; when the illumination detection and dynamic range detection results are low brightness and low dynamics, and the environmental detection result is a strong texture scene, image processing is terminated after S14, S17-S20, S22-S25 are performed on the image.

[0222] S14: Perform the first stage of image processing on the captured image. Figure 7 This is a schematic diagram of an image processing process at different stages provided in an embodiment of the present application.

[0223] The specific method of performing the first stage of image processing on the image is as follows Figure 7 As shown, the above S14 may include the following S151 to S153.

[0224] S151. Perform BPS preprocessing on the image.

[0225] The BPS module in the ISP receives images from the image sensor. Images captured by the camera are in RAW format. Unlike common JPEG and PNG image formats, RAW images are uncompressed or unprocessed, containing the raw data captured by the camera's image sensor. To view and process RAW images, they typically need to be converted to a common image format, such as RGB. BPS preprocessing involves converting the RAW image to RGB, decompressing it, and correcting bad pixels.

[0226] In addition, the preprocessing may also include simple algorithm processing such as noise reduction or color correction in the RAW domain, but the computational complexity of the algorithm does not exceed a certain threshold, and this embodiment does not impose any restriction on this.

[0227] S152: Perform brightness and color algorithm processing on the image.

[0228] After obtaining the pre-processed image output by the BPS module, the image is input into the algorithm submodule related to the first-stage algorithm processing preset in the camera algorithm library.

[0229] Exemplary brightness and color algorithms may include the AIDLC algorithm, which adjusts local contrast and color. The AILCE algorithm, by adjusting lighting style and color saturation, can adaptively adjust the contrast and color of different semantic categories, such as the sky, buildings, greenery, and portraits, achieving the same effect as professional landscape / portrait retouchers.

[0230] At the same time, in terms of parameter settings, the AILCE algorithm has three adjustable enhancement parameters: brightness (including contrast), hue, and saturation. The value range is a floating point number [0,2]. When it is 0, no enhancement is performed; when it is 1, the enhancement amplitude is medium (close to the actual P-image label effect); when it is 2, the enhancement amplitude is doubled.

[0231] S153: Perform a first noise texture algorithm process on the image.

[0232] Exemplarily, the first noise texture algorithm may include a multi-frame denoising algorithm

[0233] Traditional multi-frame denoising algorithm: Reduces image noise. When the ambient illumination is dark (such as below 500 lux), the image output by the sensor will have large noise, and the image details and textures will be submerged in the noise. The denoising algorithm is used to reduce noise and highlight the details and textures.

[0234] The first noise texture algorithm is generally set to take more than 200ms, and the algorithm complexity for a 12MP image is 20GFlops.

[0235] It should be understood that algorithmic complexity refers to the complexity of the algorithm, which refers to the time and space (memory) resources required by the program that executes the algorithm at runtime. Complexity analysis mainly considers two levels: time complexity and space complexity.

[0236] The difference in PSNR (peak signal-to-noise ratio) of images processed by the algorithm can reflect the difference in algorithm complexity of the image processing algorithm.

[0237] PSNR (Peak Signal-to-Noise Ratio): Image quality evaluation indicator. Given a clean image I and a noisy image K of size M*N, the mean square error (MSE) is defined as:

[0238]

[0239] Then PSNR is defined as:

[0240] in is the maximum possible pixel value of the image. If each pixel is represented by 8 bits of binary 2, then

[0241]

[0242] It is 255. Generally, if the pixel value is represented by B bits of binary, then MAXI=2B-1.

[0243] Generally speaking, for uint8 data, the maximum pixel value is 255; for floating-point data, the maximum pixel value is 1.

[0244] The above is the calculation method for grayscale images. If it is a color image, there are usually three methods to calculate:

[0245] Calculate the PSNR of the three RGB channels separately and then take the average; calculate the MSE of the three RGB channels and then divide it by 3; convert the image to YCbCr format, and then calculate only the PSNR of the Y component, which is the brightness component.

[0246] The PSNR (peak signal-to-noise ratio) of the pre-processed image is generally 20dB, and the PSNR of the image obtained after the first stage of image processing is 25dB.

[0247] It should be understood that the first noise texture algorithm can also include a traditional long-short frame HDR fusion algorithm. After algorithmic processing, the image can increase its dynamic range by 2-4 stops. For every N stops added, the dynamic range of the corresponding image is 2n stops. Testing using a DSC Xyla-26 dynamic range test chart shows that the dynamic range of the image before processing is 12 stops, while the dynamic range of the processed image is 16 stops.

[0248] It should be understood that the image processing algorithms at each stage are only described as examples and can be specifically configured and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0249] S15: Generate a first image.

[0250] It should be understood that after obtaining the image processed by the brightness, color algorithm sub-module and the noise, texture algorithm sub-module, because the image is usually stored and displayed in JPEG format on the electronic device, the image is input into the JPEG module, the image is JPEG encoded, and the first image is generated.

[0251] S16: Save the first image.

[0252] It should be understood that for images that do not require the second stage of image processing, the image processing process ends after the above image processing steps, and the final processing result is the first image. The first image is stored in the ROM space and is stored in the image library or displayed on the device.

[0253] S17: Generate a first image and save it to the first ROM space, compress the multiple RAW images and corresponding metadata data of the current photo and save them to the second ROM space.

[0254] It should be understood that for images requiring second-stage image processing, a first image is generated after the above-mentioned S14 image processing step is completed and saved to the first ROM space. On the device, this is stored in the gallery or displayed. The first ROM space is the ROM space that supports access by all applications. During the image processing process, the user can also obtain the first image from the gallery or share the first image with other third-party applications.

[0255] Compress multiple RAW image frames and metadata needed for subsequent image processing and store them in ROM. The second ROM space is accessible only to system applications. The number of RAW image frames and the specific metadata are determined based on the needs of subsequent image processing.

[0256] It should be understood that metadata data may include: 3a (auto exposure control AE, auto focus control AF, auto white balance control AWB) information, image information (such as image width and height), exposure parameters (aperture size, shutter speed, and ISO value), etc. Metadata data can be used to assist in post-photographing processing of RAW images and / or YUV images (for example, including at least one of beauty processing, filter processing, rotation processing, watermark processing, blur processing, HDR processing, and multi-frame processing).

[0257] In one embodiment, the metadata data may further include sensitivity (ISO) information, which may assist in adjusting the brightness of the RAW image and / or YUV image, thereby implementing post-photography processing related to brightness adjustment.

[0258] S18. Determine whether the system status in the first stage is appropriate.

[0259] The device's system status affects the efficiency of its various functions, including image processing efficiency. System status can be represented by one or more indicators, such as the device's temperature, battery level, and memory.

[0260] When the device temperature is too high or too low, the increase in the amount of data processed and the complexity of the processing algorithm will increase the hardware load of the device. At the same time, the processor will try to prevent its own temperature from being abnormal and reduce the processing speed of various functions. At this time, the efficiency of image processing is low, which does not meet the user's needs for fast photography. When the battery is too low, in order to protect the battery life and extend the battery life, the mobile phone will automatically adjust and reduce some performance. At this time, some applications with higher performance requirements may become stuck. At this time, the next stage of more complex image algorithm processing will not only fail to complete the image processing work, but will also cause damage to the mobile phone's battery. When the memory usage is too large, continuing to perform image processing with high data volume requirements and high algorithm complexity, the available memory cannot meet the demand, which increases the pressure on memory increment, reduces the image processing efficiency, and cannot meet the demand for fast image output. The memory is RAM space, that is, running memory.

[0261] Therefore, by segmenting the system based on temperature, power consumption, and memory usage, and performing the next stage of image processing, which takes longer and is more complex, under appropriate temperature, power consumption, and memory usage, it helps improve processing efficiency and meet the demand for rapid image output. For example, the first stage of system status judgment can include three parts:

[0262] First, determine whether the power level is greater than a first power level threshold. If yes, proceed to the next step of determination. If no, postpone proceeding to the next step of determination and repeat the above power level determination after a period of time.

[0263] In one embodiment, Figure 8 As shown in (a), the first power threshold can be set to 10%. If the power level is greater than 10%, the next step of judgment is entered. If the power level is less than 10%, the next step of judgment is temporarily postponed and the above power level judgment is repeated after a period of time.

[0264] Then, it is determined whether the temperature is within the first temperature range. If the result is yes, the next step of determination is performed. If the result is no, the next step of determination is temporarily postponed and the above temperature determination is repeated after a period of time.

[0265] In one embodiment, Figure 8 As shown in (b), because both low and high temperatures affect the operating efficiency of the device, the first temperature range can be set to a temperature greater than or equal to -5°C and less than or equal to 43°C. If the temperature is greater than or equal to -5°C and less than or equal to 43°C, the next step of judgment is performed. If the temperature is less than -5°C and greater than 43°C, the next step of judgment is postponed and the above temperature judgment is repeated after a period of time.

[0266] Finally, it is determined whether the memory usage is greater than the first memory threshold. If so, the captured image is processed in the second stage. If not, the image processing is temporarily suspended and the temperature determination is repeated after a period of time.

[0267] In one embodiment, Figure 8 As shown in (c), the first memory threshold can be set to 80%. If the memory usage is less than 80%, the captured image is subjected to the second stage of image processing. If the memory usage is greater than 80%, the image processing is temporarily suspended and the above memory usage judgment is repeated after a period of time.

[0268] In another embodiment, the order of the steps for determining the power level, temperature, and memory usage can be set and modified as needed. Furthermore, the power level, temperature, and memory usage can be determined serially in a single thread or in parallel in multiple threads. This embodiment of the present application does not impose any restrictions on this.

[0269] In another embodiment, when the camera preview interface is in the foreground, the system status is judged to decide whether to enter the next stage of image processing. If the camera preview causes the system load to be high, the next stage of image processing is temporarily postponed, and the system status judgment is repeated after a period of time. During the period before the system makes the next judgment, if the user clicks the gallery button to enter the gallery, it is generally believed that there are no other functions with high system load running in the background at this time. Therefore, the device will actively trigger the next power, temperature and system occupancy detection, and will not be limited by the waiting time between the two judgments. Its purpose is to speed up the image processing process based on the timely response to the user's click on the gallery and believe that he has the need to view photos, and to provide the user with the latest results of the image processing as timely as possible.

[0270] S19: Perform the second stage of image processing on the captured image. When the power, temperature, and memory usage in the system state all meet the conditions, the second stage of image processing is entered.

[0271] Figure 7 This is a schematic diagram of an image processing process at different stages provided in an embodiment of the present application.

[0272] The specific method of performing the second stage image processing on the first image is as follows: Figure 7 As shown, the above S19 may include the following S161 to S163.

[0273] S161 . Perform BPS preprocessing on the first image.

[0274] For example, the first image obtained after the first stage of image processing can be stored in the first ROM space. When the second stage of image processing is required, the first image can be obtained from the first ROM space, and the metadata required for the second stage of image processing can be obtained from the second ROM space.

[0275] It should be understood that the metadata here is obtained by the terminal after decompression from the second ROM space according to the requirements of the second stage algorithm processing. The metadata data may include: 3a (automatic exposure control AE, automatic focus control AF, automatic white balance control AWB) information, image information (such as image width and height), exposure parameters (aperture size, shutter speed and sensitivity aperture value), etc. Metadata data can be used to assist in the post-photographing processing of RAW images and / or YUV images (for example, including at least one of beauty processing, filter processing, rotation processing, watermark processing, blur processing, HDR processing, and multi-frame processing).

[0276] In one embodiment, the metadata data may further include sensitivity (ISO) information, which may assist in adjusting the brightness of the RAW image and / or YUV image, thereby implementing post-photography processing related to brightness adjustment.

[0277] The first image acquired is in JPRG format. For subsequent viewing and processing, it is usually necessary to convert it to a common image format, such as RGB. BPS preprocessing involves converting the image from JPRG format to a format suitable for image processing and decompressing it.

[0278] S162: Perform brightness and color algorithm processing on the first image.

[0279] After obtaining the pre-processed image output by the BPS module, the image is input into the algorithm submodule related to the second-stage algorithm processing preset in the camera algorithm library.

[0280] For example, the pre-processed image can be adjusted for local contrast and color using the AIDLC algorithm. The AILCE algorithm can adaptively adjust the contrast and color of different semantic categories, such as the sky, buildings, greenery, and portraits, by adjusting the lighting style and color saturation. This allows the contrast and color of different semantic categories to achieve the effect of a professional landscape or portrait retoucher.

[0281] At the same time, in terms of parameter settings, the AILCE algorithm has three adjustable enhancement parameters: brightness (including contrast), hue, and saturation. The value range is a floating point number [0,2]. When it is 0, no enhancement is performed; when it is 1, the enhancement amplitude is medium (close to the actual P-image label effect); when it is 2, the enhancement amplitude is doubled.

[0282] The brightness and color processing parameters for the image here are consistent with those used in S152. This is intended to prevent significant visual differences between the image results obtained at each stage. This is because after taking multiple consecutive photos, users generally don't immediately zoom in to examine the details; their perception of the photos primarily comes from their brightness, color, and composition. Therefore, during the limited time between when a user takes a photo and when they first view it, the brightness and color processing parameters for each stage of image processing remain consistent, resolving the issue of significant differences in image quality resulting from processing, which can negatively impact user experience.

[0283] S163: Perform a second noise texture algorithm process on the first image.

[0284] Exemplarily, the second noise texture algorithm may include an AI multi-frame denoising algorithm. This AI multi-frame denoising algorithm is used to reduce image noise and offers better noise removal and detail preservation capabilities than traditional multi-frame denoising algorithms. The second noise texture algorithm typically takes longer than 400ms, and the algorithm complexity for a 12MP image is 2.4 teraflops.

[0285] It should be understood that the difference in PSNR (Peak Signal-to-Noise Ratio) between images processed by this algorithm reflects the difference in algorithmic complexity. The PSNR achieved by the first noise texture algorithm in the first stage of image processing is 25dB. The PSNR achieved by the first and second noise texture algorithms in the first and second stages of image processing is 30dB.

[0286] Exemplarily, the second noise texture algorithm may also include an AI HDR fusion algorithm. After algorithmic processing, the image's dynamic range can be increased by 2-8 stops, with each increase of N stops resulting in a dynamic range of 2n stops. Testing using a DSC Xyla-26 dynamic range test chart shows that the dynamic range of the image before processing was 12 stops, while the dynamic range of the processed image was 20 stops.

[0287] It should be understood that the image processing algorithms at each stage are only described as examples and can be specifically configured and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0288] It should be understood that the second noise texture algorithm includes more processing steps than the first noise texture algorithm, or the complexity of the processing steps is gradually enhanced, and the amount of calculation is gradually increased, so that the processing time is gradually increased.

[0289] The specific settings and modifications can be made as needed, and the embodiments of the present application do not impose any restrictions on this.

[0290] S20: Generate a second image and delete the first image.

[0291] It should be understood that after obtaining the image processed by the brightness, color algorithm submodule and the noise, texture algorithm submodule, because the storage and display of the image on the electronic device are usually in JPEG format, the image can be input into the JPEG module, the image is JPEG encoded, and a second image is generated. At the same time, the first image stored in the first ROM space in S17 is deleted. Of course, the image processed in the second stage can also be converted to other formats (such as PNG, TIFF, etc.) or not converted, and the embodiment of the present application does not impose any restrictions on this.

[0292] S21. Save the second image.

[0293] It should be understood that for images that do not need to continue the third stage of image processing, the image processing process ends after the above image processing steps. The second image is stored in the first ROM space, which is stored in the image library or displayed on the device.

[0294] S22: Generate and save the second image, and delete the first image

[0295] It should be understood that for images that require third-stage image processing, after completing the above-mentioned S14, S17-S19 image processing steps, a second image is encoded using a JPEG encoder and saved to the first ROM space. On the device, it is saved in the gallery or displayed. The first ROM space is a ROM space that supports access by all applications. During the image processing process, the user can also obtain the second image from the gallery or share the second image with other third-party applications. At the same time, the first image stored in the first ROM space in S17 is deleted.

[0296] S23: Determine whether the system status in the second phase is appropriate.

[0297] Similar to S18, the temperature, power consumption and memory usage are segmented, and the next stage of image processing, which takes longer and is more complex, is carried out under appropriate temperature, power consumption and memory usage conditions, which helps to improve processing efficiency and meet the demand for fast image output.

[0298] Exemplarily, the second stage system status determination may include three parts:

[0299] First, determine whether the power level is greater than the second power level threshold. If the result is yes, proceed to the next step of determination. If the result is no, postpone proceeding to the next step of determination and repeat the above power level determination after a period of time.

[0300] In one embodiment, Figure 8As shown in (a), the second power threshold can be set to 50%. If the power level is greater than 50%, the next step of judgment is entered. If the power level is less than 50%, the next step of judgment is temporarily postponed and the above power judgment is repeated after a period of time.

[0301] Then, it is determined whether the temperature is within the second temperature range. If the result is yes, the next step of determination is entered. If the result is no, the next step of determination is temporarily postponed and the above temperature determination is repeated after a period of time.

[0302] In one embodiment, Figure 8 As shown in (b), because both low and high temperatures affect the operating efficiency of the device, the second temperature range can be set to a temperature greater than or equal to 10°C and less than or equal to 40°C. If the temperature is greater than or equal to -10°C and less than or equal to 40°C, the next step of judgment is performed. If the temperature is less than 10°C and greater than 40°C, the next step of judgment is postponed and the above temperature judgment is repeated after a period of time.

[0303] Finally, it is determined whether the memory usage is greater than a second memory threshold. If so, the captured image is processed in the second stage. If not, the image processing is temporarily suspended and the temperature determination is repeated after a period of time.

[0304] In one embodiment, Figure 8 As shown in (c), the second memory threshold can be set to 80%. If the memory usage is less than 80%, the captured image is subjected to the second stage of image processing. If the memory usage is greater than 80%, the image processing is temporarily suspended and the above memory usage judgment is repeated after a period of time.

[0305] In another embodiment, the order of the steps of power determination, temperature determination, and memory usage determination can be set and modified as needed. At the same time, the power determination, temperature determination, and memory usage determination can be performed serially in a single thread or in parallel in multiple threads. The present embodiment does not impose any restrictions on this. It should be understood that in the above-mentioned S18 first-stage system status determination and S22 second-stage system status determination, only two memory thresholds are set, namely the first power threshold and the second power threshold. Through these two power thresholds, the power level can be divided into three intervals. When the power level meets one of the intervals, it is determined whether to enter the next stage of image algorithm processing and use the algorithm corresponding to the interval to perform processing, thereby obtaining the corresponding captured image. In addition, a memory threshold can also be set to divide the memory size into two intervals; or three, four or more memory thresholds can be set to subdivide the memory size into multiple intervals. The specific setting and modification can be made as needed, and the present embodiment does not impose any restrictions. The same applies to the temperature threshold and memory threshold.

[0306] S23: Perform the third stage of image processing on the captured image. When the power, temperature, and memory usage in the system state all meet the conditions, the third stage of image processing is entered.

[0307] Figure 7 This is a schematic diagram of an image processing process at different stages provided in an embodiment of the present application.

[0308] The specific method of performing the third stage image processing on the second image is as follows: Figure 7 As shown, the above S24 may include the following S171 to S173.

[0309] S171 . Perform BPS preprocessing on the second image.

[0310] Get the second image from the first ROM space, and the metadata required for the third stage image processing from the second ROM space.

[0311] It should be understood that the metadata here is obtained by the terminal after decompression from the ROM space according to the requirements of the third stage algorithm processing. The metadata data may include: 3a (automatic exposure control AE, automatic focus control AF, automatic white balance control AWB) information, image information (such as image width and height), exposure parameters (aperture size, shutter speed and sensitivity aperture value), etc. Metadata data can be used to assist in the post-photographing processing of RAW images and / or YUV images (for example, including at least one of beauty processing, filter processing, rotation processing, watermark processing, blur processing, HDR processing, and multi-frame processing).

[0312] In one embodiment, the metadata data may further include sensitivity (ISO) information, which may assist in adjusting the brightness of the RAW image and / or YUV image, thereby implementing post-photography processing related to brightness adjustment.

[0313] Compared with the second stage, the amount of metadata in the third stage is larger than that in the second stage.

[0314] The second image is acquired in JPEG format. To view and process RAW images later, we usually need to convert them to a common image format, such as RGB. BPS preprocessing involves converting the image from JPEG format to the image format required for image processing and decompressing it.

[0315] S172: Perform brightness and color algorithm processing on the second image.

[0316] For example, the pre-processed image can be adjusted for local contrast and color using the AIDLC algorithm. The AILCE algorithm can adaptively adjust the contrast and color of different semantic categories, such as the sky, buildings, greenery, and portraits, by adjusting the lighting style and color saturation. This allows the contrast and color of different semantic categories to achieve the effect of a professional landscape or portrait retoucher.

[0317] At the same time, in terms of parameter settings, the AILCE algorithm has three adjustable enhancement parameters: brightness (including contrast), hue, and saturation. The value range is a floating point number [0,2]. When it is 0, no enhancement is performed; when it is 1, the enhancement amplitude is medium (close to the actual P-image label effect); when it is 2, the enhancement amplitude is doubled.

[0318] The image brightness and color processing parameters here are consistent with those used in S163. Similar to S163, this is intended to ensure that the user experiences minimal visual differences between the image results at each stage. This is because, after taking multiple consecutive photos, users generally don't immediately zoom in to examine the details; their perception of the photos primarily comes from their brightness, color, and composition. Therefore, during the limited time between when a photo is taken and when the user first views it, the brightness and color processing parameters for each stage of image processing remain consistent, resolving the issue of significant differences in image quality resulting from processing, which can negatively impact user experience.

[0319] S173 , performing a third noise texture algorithm process on the second image.

[0320] Exemplarily, the third noise texture algorithm may include an AI multi-frame denoising / fusion algorithm model. This model is used to reduce image noise. Compared to the AI ​​multi-frame denoising algorithm, it can more intelligently remove noise while preserving details based on different shooting scenes and subjects. The third noise texture algorithm typically takes more than 1000ms, and the algorithm complexity for a 12MP image is 13 teraflops.

[0321] It should be understood that the difference in PSNR (Peak Signal-to-Noise Ratio) between images processed by this algorithm reflects the difference in algorithmic complexity. The PSNR achieved after the first and second stages of the noise texture algorithm is 30dB, while the PSNR achieved after the first, second, and third stages of the noise texture algorithm is over 33dB.

[0322] For example, the third noise texture algorithm may also include a Quad / Binning large and small image fusion algorithm, a Deblur algorithm, and an AIGC super-resolution algorithm. Compared to the second stage, this algorithm can significantly improve detail texture while optimizing noise.

[0323] It should be understood that the image processing algorithms at each stage are only described as examples and can be specifically configured and modified as needed. The embodiments of the present application do not impose any restrictions on this.

[0324] It should be understood that the third noise texture algorithm includes gradually more processing steps than the second noise texture algorithm, or the complexity of the processing steps included is gradually enhanced, and the amount of calculation is gradually increased, so that the processing time is gradually increased. It can be set and modified as needed, and the embodiments of the present application do not impose any restrictions on this.

[0325] S25 , generating a third image, deleting the second image, and deleting multiple frames of RAW images and metadata data related to the first image and the second image.

[0326] It should be understood that after obtaining the image processed by the brightness and color algorithm submodule and the noise and texture algorithm submodule, since images are typically stored and displayed on electronic devices in JPEG format, the image is input to the JPEG module for JPEG encoding to generate the third image. Simultaneously, the second image stored in the first ROM space in S22 is deleted.

[0327] S26: Save the third image.

[0328] It should be understood that after the above image processing steps, the image processing flow ends. The third image is stored in the first ROM space, and is stored in the image library or displayed on the device.

[0329] It should be understood that in the above S14 to S25, only three stages of image processing are set. The algorithm complexity of each stage is different. As the stages increase, the complexity of the algorithm increases, thereby gradually increasing the processing time. The N stages of image processing are determined based on the environmental state of the shooting scene and the system state, where N is an integer greater than 0. In addition, a fourth stage, a fifth stage, etc. can also be set. The specific settings and modifications can be made as needed, and the embodiments of the present application do not impose any restrictions.

[0330] Figure 9 This table provides examples of the present application, illustrating the algorithms used at each stage under different environmental conditions. Different preset algorithms are used at each stage of image processing under different brightness levels and dynamic ranges. Based on the scene detection results, whether they indicate a meaningless scene, a weakly textured scene, or a strongly textured scene, the image processing process is completed, with the final image output determined by the number of stages required.

[0331] Combine Figure 1 As shown, when the user opens the camera application, the phone displays a shooting interface, which includes a shooting button.

[0332] At this point, the phone can perform environmental detection for the shooting scene, such as illumination detection, dynamic range detection, and scene detection. Based on the environmental detection results, it can also determine whether the detected illumination is less than the illumination threshold, whether the detected dynamic range value is less than the dynamic range threshold, and what kind of scene the shooting scene is.

[0333] When the mobile phone detects a click operation by the user, in response to the click operation, the mobile phone uses the camera to capture an image.

[0334] When the illumination is less than the illumination threshold, the dynamic range value is less than the dynamic range threshold, or when the shooting scene is in low light and low dynamic range, such as Figure 9 As shown, the first-stage A1 algorithm may include the MFHDR algorithm from ISP processing. If the captured scene is meaningless, image processing ends and the final image is generated. Otherwise, the next stage of image processing is performed. The second-stage A2 algorithm may include the Ellip or Elliplite algorithm from the AIRAW algorithm suite. If the captured scene is weakly textured, image processing ends and the final image is generated. Otherwise, the next stage of image processing is performed. The third-stage A3 algorithm may include the RAW-domain Ellip or Elliplite algorithm as well as optimization algorithms tailored to different scenarios. For example, AIGC is used for landscapes; facer, ref-face, open / closed eye optimization, and expression optimization are used for portraits; TextSR is used for text images; and deblurring is performed using deblur, RefSRC, SISR, and QRFusion. After processing the image using the RAW-domain Ellip or Elliplite algorithm, the image can be directly output as a JPEG image or further optimized using an optimization algorithm determined by the image scene information.

[0335] When the illumination is less than the illumination threshold and the dynamic range value is greater than the dynamic range threshold, or when the shooting scene is a low-light, high dynamic range scene, the first stage B1 algorithm may include the MFHDR algorithm in ISP processing or the HDR algorithm in YUV domain processing. When the shooting scene is a meaningless scene, the image processing is terminated and the final image is generated. Otherwise, the next stage of image processing is performed. The second stage A2 algorithm may include the Ellip-nh algorithm or Elliplite-nh algorithm in the AI ​​RAW algorithm set. When the shooting scene is a weak-texture scene, the image processing is terminated and the final image is generated. Otherwise, the next stage of image processing is performed. The third stage B3 algorithm may include the Ellip-nh or Elliplite-nh algorithm in the RAW domain and optimization algorithms determined according to different scenarios, such as: AIGC is used for landscape photos; facer, ref-face, open and closed eye algorithm optimization, expression optimization, etc. are used for portrait photos; TextSR is used for text images; deblurring uses deblur, RefSRC, SISR, and QRFusion. After the image is processed by the RAW domain Ellip-nh or Elliplite-nh algorithm, it can be directly output as a JPEG image, or it can continue to be processed by an optimization algorithm determined according to the image scene information to further optimize the image.

[0336] When the illumination is greater than the illumination threshold and the dynamic range is less than the dynamic range threshold, or when the scene is bright and low dynamic range, the first stage C1 algorithm can be the MFHDR algorithm in the ISP processing. If the scene is meaningless, image processing ends and the final image is generated. Otherwise, the next stage of image processing is performed. The second stage C2 algorithm can be HDRPLite from the AI ​​RAW algorithm set. If the scene is weakly textured, image processing ends and the final image is generated. Otherwise, the next stage of image processing is performed. The third stage C3 algorithm can include HDRPLite in the Raw domain and optimization algorithms determined by different scenarios. For example, AIGC is used for landscape photos; facer, ref-face, open and closed eye optimization, and expression optimization are used for portrait photos; TextSR is used for text images; and deblurring is performed using deblur, RefSRC, SISR, and QRFusion. After processing the image in the Raw domain HDRPLite algorithm, the output can be a JPEG image directly or further optimized using an optimization algorithm determined by the image scene information.

[0337] When the illuminance is greater than the illuminance threshold and the dynamic range value is greater than the dynamic range threshold, or when the shooting scene is a high-brightness, high-dynamic-range scene, the D1 algorithm in the first stage may include the MFHDR algorithm in SP processing or the HDR in YUV domain processing. When the shooting scene is a scene without practical meaning, the image processing is terminated and the final image is generated, otherwise the next stage of image processing is performed; the D2 algorithm in the second stage may include MEF in the AI ​​RAW algorithm set. When the shooting scene is a weak-texture scene, the image processing is terminated and the final image is generated, otherwise the next stage of image processing is performed; the D3 algorithm in the third stage may include the MEF algorithm in the RAW domain and the optimization algorithm determined according to different scenes, such as: the AIGC algorithm is used for landscape photos; the facer algorithm, ref-face algorithm, open and closed eye algorithm optimization algorithm, expression optimization algorithm, etc. are used for portrait photos; the TextSR algorithm is used for text images; the deblurring algorithm, RefSRC algorithm, SISR algorithm or QRFusion algorithm are used for deblurring. After the image is processed by the RAW domain MEF algorithm, it can be directly output as a JPEG image, or it can continue to be processed by an optimization algorithm determined according to the image scene information to further optimize the image.

[0338] Figure 9 An example is given for the algorithm in the above process. This example is only used as an image processing process that may exist in the staged image processing process. Other steps can be added or some steps can be deleted on this basis. The embodiments of the present application do not impose any restrictions on this.

[0339] The following describes an image processing device 300 provided in an embodiment of the present application for implementing the above-mentioned photographing method. Figure 10 3 is a schematic diagram of an image processing device 300 provided in an embodiment of the present application.

[0340] like Figure 10 As shown, the image processing apparatus 300 includes a display unit 310 , an acquisition unit 320 and a processing unit 330 .

[0341] The display unit 310 is used to display a first interface, which includes a first control.

[0342] The acquisition unit 320 is configured to detect a first operation performed by a user on a first control.

[0343] The processing unit 330 is configured to capture an image once in response to the first operation.

[0344] The processing unit 330 is also used to perform environmental detection by determining the illumination and dynamic range of the captured scene. When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, the A1 algorithm is used for the first stage of image processing. When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the B1 algorithm is used for the first stage of image processing. When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, the C1 algorithm is used for the first stage of image processing. When the illumination is greater than or equal to the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the D1 algorithm is used for the first stage of image processing. By determining the scene state of the captured scene, if it is determined to be a meaningless scene, the processing result is output after the first stage of image processing. If it is determined to be a weak texture scene, the processing result is output after the first and second stages of image processing. If it is determined to be a strong texture scene, the processing result is output after the first, second, and third stages of image processing.

[0345] It should be noted that the image processing device 300 is implemented in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0346] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the aforementioned functionality. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0347] Therefore, the units of each example described in the embodiments of this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0348] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium runs on the image processing device 300, the image processing device 300 executes the photographing method shown above.

[0349] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0350] The embodiment of the present application further provides a computer program product including computer instructions, which, when executed on the image processing device 300, enables the image processing device 300 to execute the aforementioned photographing method.

[0351] Figure 11 A schematic diagram of the structure of a chip provided in an embodiment of the present application. Figure 11 The chip shown can be a general-purpose processor or a dedicated processor. The chip includes a processor 401. The processor 401 is used to support the image processing device 300 to execute the technical solution shown above.

[0352] Optionally, the chip further includes a transceiver 402 , which is configured to accept control from the processor 401 and to support the image processing apparatus 300 in executing the aforementioned technical solution.

[0353] Optional, Figure 11 The chip shown may further include: a storage medium 403 .

[0354] It should be noted that Figure 11 The chip shown can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.

[0355] The electronic device, image processing device 300, computer storage medium, computer program product, and chip provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0356] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application. Based on the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, certain steps in each embodiment of the above detection method may be unnecessary, or certain new steps may be added. Or a combination of any two or any multiple embodiments described above. Such modifications, changes, or combined solutions also fall within the scope of the embodiments of the present application.

[0357] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced with each other. For the sake of brevity, they will not be repeated here.

[0358] It should also be understood that the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0359] It should also be understood that in the embodiments of the present application, "pre-setting" and "pre-definition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device (for example, including an electronic device), and the present application does not limit its specific implementation method.

[0360] It should also be understood that the division of the modes, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various modes, categories, situations and embodiments can be combined without contradiction.

[0361] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0362] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: Applied to electronic equipment, the method includes: Displaying a first interface, wherein the first interface includes a first control; When a first operation on the first control is detected, capturing an image in response to the first operation; Performing a first-stage image processing on the acquired image to generate a first image; storing the first image in a gallery application; When a system state of the electronic device satisfies a first condition, performing a second-stage image processing on the first image to generate a second image; the system state includes at least one of power level, temperature, and memory usage; and the algorithm complexity of the second-stage image processing is greater than the algorithm complexity of the first-stage image processing; The second image is stored in the gallery application, and the first image is deleted from the gallery application.

2. The image processing method according to claim 1, wherein: The method further comprises: When the first interface is displayed, an environmental detection is performed on the shooting scene; According to the result of the environmental detection, N stages of image processing for the image and an image processing algorithm for each stage are determined, where N is an integer greater than 0.

3. The image processing method according to claim 2, wherein: The environmental detection includes scene state detection; When the result of the scene state detection is a meaningless scene, the N stages of image processing include the first stage of image processing; When the result of the scene state detection is a weak texture scene, the N stages of image processing include the first stage image processing and the second stage image processing.

4. The image processing method according to claim 2, wherein: The environmental detection includes illumination detection and dynamic range detection; When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, the first stage image processing uses the A1 algorithm and the second stage image processing uses the A2 algorithm; When the illuminance is less than the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the first stage image processing uses the B1 algorithm, and the second stage image processing uses the B2 algorithm; When the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is less than the dynamic range threshold, the first stage image processing uses the C1 algorithm and the second stage image processing uses the C2 algorithm; When the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the first stage image processing uses the D1 algorithm and the second stage image processing uses the D2 algorithm.

5. The image processing method according to claim 3, wherein: The meaningless scene refers to a shooting scene in which the pixel gradient is less than a first gradient threshold, and the weak texture scene refers to a shooting scene in which the pixel gradient is greater than the first gradient threshold and less than a second gradient threshold.

6. The image processing method according to claim 4, wherein: The A1 algorithm includes a first noise texture algorithm, and the A2 algorithm includes a second noise texture algorithm; the PSNR of an image processed by the second noise texture algorithm is higher than the PSNR of an image processed by the first noise texture algorithm; The B1 algorithm includes a third noise texture algorithm, and the B2 algorithm includes a fourth noise texture algorithm; the PSNR of the image processed by the fourth noise texture algorithm is higher than the PSNR of the image processed by the third noise texture algorithm; The C1 algorithm includes a fifth noise texture algorithm, and the C2 algorithm includes a sixth noise texture algorithm; the PSNR of the image processed by the sixth noise texture algorithm is higher than the PSNR of the image processed by the fifth noise texture algorithm; The D1 algorithm includes a seventh noise texture algorithm, and the D2 algorithm includes an eighth noise texture algorithm; the PSNR of an image processed by the eighth noise texture algorithm is higher than the PSNR of an image processed by the seventh noise texture algorithm.

7. The image processing method according to claim 4, wherein: The A1 algorithm includes a first brightness and color algorithm, and the A2 algorithm includes the first brightness and color algorithm; The B1 algorithm includes a second brightness and color algorithm, and the B2 algorithm includes the second brightness and color algorithm; The C1 algorithm includes a third brightness and color algorithm, and the C2 algorithm includes the third brightness and color algorithm; The D1 algorithm includes a fourth brightness and color algorithm, and the D2 algorithm includes the fourth brightness and color algorithm.

8. The image processing method according to claim 1, wherein: The first condition includes: The power level is greater than a first power level threshold, the temperature is within a first temperature range, and the memory usage is less than a first memory threshold.

9. The image processing method according to claim 6, wherein: The method further comprises: When the system state of the electronic device satisfies a second condition, performing a third-stage image processing on the second image to generate a third image; The system status includes at least one of power, temperature, and memory usage; The second condition is that when the power level is greater than a second power threshold, the temperature is within a second temperature range, and the memory usage is less than a second memory threshold, the computational complexity of the image processing in the third stage is greater than the computational complexity of the image processing in the second stage; The third image is stored in the gallery application, and the second image in the gallery application is deleted.

10. The image processing method according to claim 9, wherein: The environmental detection includes scene state detection; When the scene detection result is a strong texture scene, the N stages of image processing include the first stage image processing, the second stage image processing and the third stage image processing.

11. The image processing method according to claim 9, wherein: The environmental detection includes illumination detection and dynamic range detection; When the illumination is less than the illumination threshold and the dynamic range value is less than the dynamic range threshold, the third stage image processing uses the A3 algorithm; When the illumination is less than the illumination threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the third stage image processing uses the B3 algorithm; When the illumination is greater than or equal to the illumination threshold and the dynamic range value is less than the dynamic range threshold, the third stage image processing uses the C3 algorithm; When the illuminance is greater than or equal to the illuminance threshold and the dynamic range value is greater than or equal to the dynamic range threshold, the third stage image processing uses the D3 algorithm.

12. The image processing method according to claim 10, wherein: The strong texture scene refers to a shooting scene in which the pixel gradient is greater than a first gradient threshold.

13. The image processing method according to claim 11, wherein: The A3 algorithm includes a ninth noise texture algorithm, and the PSNR of an image processed by the ninth noise texture algorithm is higher than the PSNR of an image processed by the second noise texture algorithm; The B3 algorithm includes a tenth noise texture algorithm, and the PSNR of the image processed by the tenth noise texture algorithm is higher than the PSNR of the image processed by the fourth noise texture algorithm; The C3 algorithm includes an eleventh noise texture algorithm, and the PSNR of the image processed by the eleventh noise texture algorithm is higher than the PSNR of the image processed by the sixth noise texture algorithm; The D3 algorithm includes a twelfth noise texture algorithm, and the PSNR of an image processed by the twelfth noise texture algorithm is higher than the PSNR of an image processed by the eighth noise texture algorithm.

14. The image processing method according to claim 11, wherein: The A3 algorithm includes a first brightness and color algorithm; The B3 algorithm includes a second brightness and color algorithm; The C3 algorithm includes a third brightness and color algorithm; The D3 algorithm includes a fourth brightness and color algorithm.

15. The image processing method according to any one of claims 1 to 13, characterized in that: The first interface refers to a photo taking interface, and the first control refers to a control for instructing photo taking.

16. An electronic device, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 15.

17. A chip, characterized in that: The chip is applied to an electronic device, and the chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 15 .

Images