A dark image processing method, electronic equipment and server

Abstract

A dark light image processing method, electronic equipment and server are used to improve the image quality of dark light images. In the method, the electronic equipment can collect a first dark light image in a dark light environment; wherein the dark light environment can be detected when the light brightness is less than a preset threshold. Then, the electronic equipment sends the first dark light image and the first category label of the first dark light image to the server. Correspondingly, the server enhances the image of the first dark light image through an AI model to obtain a first image; wherein the input parameters of the AI model can be based on the first blur result obtained by blurring the first dark light image, the first category label, and the first dark light image. Finally, the electronic equipment receives the first image from the server and displays it. The method can improve the quality of dark light images taken by the electronic equipment.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method for processing low-light images, an electronic device, and a server. Background Technology

[0002] As the performance of cameras on electronic devices gradually improves, users are becoming increasingly accustomed to using cameras to capture beautiful scenery or wonderful things anytime, anywhere in their daily lives. However, due to hardware limitations such as the typically small sensor size and physical size constraints of lens apertures used in cameras on electronic devices, problems such as low image quality and underexposure still exist in low-light environments, affecting the user's shooting experience. Summary of the Invention

[0003] This application provides a low-light image processing method, electronic device, and server to provide a technical solution that can improve the image quality of low-light images, thereby enhancing the user's shooting experience.

[0004] Firstly, a method for processing low-light images is provided. In this method, an electronic device captures a first low-light image, wherein the light intensity of the first low-light image at the time of capture is less than a preset threshold; the electronic device sends the first low-light image and a first category label for the first low-light image to a server; the server blurs the first low-light image to obtain a first blurring result, the first blurring result indicating areas that do not require image enhancement; the server obtains input parameters for an artificial intelligence (AI) model based on the first blurring result, the first category label, and the first low-light image, and performs image enhancement through the AI ​​model to obtain a first image; the server sends the first image to the electronic device; and the electronic device displays the first image.

[0005] The first category label is obtained by the electronic device through image recognition of the first low-light image. For example, the electronic device may display a preview of the first image in a camera application, or it may store and display the first image in a photo album application, or display the first image in other applications that can access the photo album, etc.

[0006] This method, by employing an edge-cloud integrated system architecture, can fully utilize the preliminary processing capabilities of electronic devices and the powerful computing resources of the cloud, thereby improving the processing effect of low-light images and optimizing their quality. Furthermore, by using the first blurring result, the first category label, and the first low-light image as input parameters for the AI ​​model, the layering and naturalness of the first image can be enhanced, highlighting the subject in the low-light image.

[0007] In one possible implementation, the first dark image is obtained by the electronic device after processing a RAW format dark image.

[0008] In this implementation, the image enhancement method used in this application does not need to be limited to RAW format dark light images. It can enhance the first dark light image through the training and learning of AI models, and can improve the effect and naturalness of image enhancement.

[0009] In one possible implementation, the method further includes: the electronic device processing the RAW format low-light image using a convolutional neural network. For example, the electronic device may use a lightweight convolutional neural network or a standard convolutional neural network to process the RAW format low-light image.

[0010] In this implementation, the preliminary processing capability of the electronic device can be used to obtain a first low-light image that has undergone preliminary processing, which can improve the image enhancement effect and naturalness of the obtained first image.

[0011] In one possible implementation, the image enhancement via the AI ​​model includes: performing a first image enhancement on a first region using the AI ​​model, the first region being determined based on the first category label; and performing a second image enhancement on a second region using the AI ​​model, the second region being determined based on the first blurring result; wherein the degree of the first image enhancement is stronger than the degree of the second image enhancement.

[0012] In this implementation, by dividing the first dark-light image into target and non-target regions and performing image enhancement to different degrees, the sense of depth of the first image can be improved, making the subject in the dark-light image stand out more.

[0013] In one possible implementation, the method further includes: the server generating a prompt word based on the first category label. The server obtaining input parameters for the AI ​​model based on the first category label includes: the server obtaining the input parameters based on the prompt.

[0014] In this implementation, the server injects semantics into the AI ​​model based on the first category label, which can better guide the AI ​​model on how to perform image enhancement through prompt words, thereby improving the efficiency and targeting of image enhancement.

[0015] In one possible implementation, obtaining the input parameters of the AI ​​model based on the first bokeh result, the first category label, and the first low-light image includes: generating a mask based on the first bokeh result; encoding the first low-light image using an image encoder to generate a first latent space; encoding the prompt using a text encoder to generate a text embedding; and using the mask, the first latent space, and the text embedding as input parameters of the AI ​​model.

[0016] In this implementation, by generating prompt words based on the first category label and inputting them into the AI ​​model, the server can perform targeted processing based on the first category label obtained from the electronic device. For example, it can perform first image enhancement on the target area corresponding to the first category label, thereby achieving depth optimization of the target area. Furthermore, by inputting the first blurring result into the AI ​​model, the server can perform a second image enhancement process based on the first blurring result, which is weaker than the first image enhancement process, thus improving the layering of the obtained second image.

[0017] In one possible implementation, the step of enhancing the image using the AI ​​model to obtain the first image includes: enhancing the image using the AI ​​model to obtain a second latent space; and decoding the second latent space using an image decoder to obtain the first image.

[0018] In this implementation, by decoding the second latent space output by the AI ​​model, the first image after image enhancement can be restored, thereby improving the optimization effect on the first low-light image.

[0019] In one possible implementation, the image enhancement includes at least one of the following: deblurring, super-resolution processing, image sharpening, bokeh optimization, brightness optimization, or contrast optimization.

[0020] In this implementation, deblurring can be used to eliminate blur in the first low-light image, improving its clarity; super-resolution processing can be used to increase the resolution of the first low-light image, improving its image quality; image sharpening can be used to improve the clarity and detail of the subject in the first low-light image, the subject being determined based on a first category label; blurring optimization can be used to enhance the layering in the low-light image, making the subject stand out more; brightness optimization and contrast optimization can be used to improve the image quality of the low-light image. Therefore, through image enhancement, a first image with better image quality can be obtained.

[0021] In one possible implementation, before the electronic device sends the first low-light image and a first category label for the first low-light image to the server, the method further includes: the electronic device determining that the first category label belongs to a category label that the AI ​​model can process.

[0022] In this implementation, by filtering through electronic devices, the processing of some scenarios with poor server image enhancement effects can be reduced, thus reducing resource waste.

[0023] Secondly, an image processing method is also provided, applied to an electronic device. The method includes: capturing a first low-light image, wherein the light intensity of the first low-light image at the time of capture is less than a preset threshold; blurring the first low-light image to obtain a first blurring result, the first blurring result indicating areas that do not require image enhancement; obtaining input parameters for an AI model based on the first blurring result, a first category label of the first low-light image, and the first low-light image; performing image enhancement through the AI ​​model to obtain a first image; and displaying the first image. The first category label is obtained by the electronic device through image recognition of the first low-light image.

[0024] For example, the electronic device may display a preview of the first image in a camera application, or it may store and display the first image in a photo album application, or display the first image in other applications that can access the photo album, etc.

[0025] In this implementation, by using the first blurring result, the first category label, and the first dark light image as input parameters for the AI ​​model, the layering and naturalness of the first image can be improved, highlighting the subject in the dark light image.

[0026] In one possible implementation, the first dark image is obtained by the electronic device after processing the original RAW format dark image.

[0027] In one possible implementation, the method further includes processing the RAW format low-light image using a convolutional neural network. For example, an electronic device may use a lightweight convolutional neural network or a standard convolutional neural network to process the RAW format low-light image.

[0028] In one possible implementation, the image enhancement via the AI ​​model includes: performing a first image enhancement on a first region using the AI ​​model, the first region being determined based on the first category label; and performing a second image enhancement on a second region using the AI ​​model, the second region being determined based on the first blurring result; wherein the degree of the first image enhancement is stronger than the degree of the second image enhancement.

[0029] In one possible implementation, the method further includes: generating a prompt word based on the first category label; and obtaining input parameters of the AI ​​model based on the first category label, including: obtaining the input parameters based on the prompt.

[0030] In one possible implementation, obtaining the input parameters of the AI ​​model based on the first bokeh result, the first category label, and the first low-light image includes: generating a mask based on the first bokeh result; encoding the first low-light image using an image encoder to generate a first latent space; encoding the prompt using a text encoder to generate a text embedding; and using the mask, the first latent space, and the text embedding as input parameters of the AI ​​model.

[0031] In one possible implementation, the step of enhancing the image using the AI ​​model to obtain the first image includes: enhancing the image using the AI ​​model to obtain a second latent space; and decoding the second latent space using an image decoder to obtain the first image.

[0032] In one possible implementation, the image enhancement includes at least one of the following:

[0033] Deblurring, super-resolution processing, image sharpening, bokeh optimization, brightness optimization, or contrast optimization.

[0034] Thirdly, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the steps performed by the electronic device as described in the first or second aspect above.

[0035] Fourthly, an electronic device is also provided, comprising a module / unit for performing the method corresponding to any possible implementation of the first or second aspect described above. These modules / units may be implemented in hardware or by hardware executing corresponding software.

[0036] Fifthly, a server is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps performed by the server as described in the first aspect above.

[0037] Sixthly, a server is also provided, including modules / units for executing the methods corresponding to any of the designs in the first aspect above. These modules / units can be implemented in hardware or by executing corresponding software in hardware.

[0038] In a seventh aspect, a chip is also provided, including a processor and an interface; the processor is configured to read instructions via the interface to execute the method described in the first or second aspect above.

[0039] Eighthly, a chip system is also provided, the chip system including a processing circuit and a storage medium, the storage medium storing instructions; when the instructions are executed by the processing circuit, they implement the method as described in the first or second aspect above.

[0040] Ninthly, a communication system is also provided, comprising:

[0041] An electronic device for performing the steps performed by the electronic device in any of the possible implementations of the first aspect described above;

[0042] A server is used to perform the steps executed by the server in any of the possible implementations of the method described in the first aspect above.

[0043] In a tenth aspect, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program that, when executed by at least one processor, implements the method as described in the first or second aspect above.

[0044] Eleventhly, a computer program product is also provided, comprising a computer program that, when run on a computer, enables the computer to perform the methods described in the first or second aspect above.

[0045] The beneficial effects of the design in any of the second to eleventh aspects mentioned above can be referred to the beneficial effects of the corresponding implementation in the first aspect mentioned above, and this application will not elaborate on them one by one. Attached Figure Description

[0046] Figure 1 A schematic diagram of the hardware structure of a possible electronic device is shown;

[0047] Figure 2A software system architecture block diagram of an electronic device provided in this application embodiment;

[0048] Figure 3 A comparative diagram illustrating a low-light image processing method provided in an embodiment of this application;

[0049] Figure 4 Another comparative diagram illustrating a low-light image processing method provided in an embodiment of this application;

[0050] Figure 5 A schematic diagram of the system architecture for a low-light image processing method provided in an embodiment of this application;

[0051] Figure 6 An example diagram of a low-light image 600 provided in an embodiment of this application;

[0052] Figure 7 A flowchart illustrating a low-light image processing method provided in an embodiment of this application;

[0053] Figure 8 This is a schematic diagram illustrating information changes on server 500 provided in an embodiment of this application.

[0054] Figure 9 This is a schematic diagram of an interface for a low-light image processing method provided in an embodiment of this application. Detailed Implementation

[0055] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0056] The embodiments of this application involve at least one, including one or more; where "multiple" means two or more. Furthermore, it should be understood that in the description of this specification, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order. For example, the first image and the second image do not represent their relative importance or order, but are merely for descriptive distinction. In the embodiments of this application, "and / or" merely describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0057] The directional terms mentioned in the embodiments of this application, such as "up", "down", "left", "right", "inner", and "outer", are only for reference to the directions in the accompanying drawings. Therefore, the directional terms used are for better and clearer explanation and understanding of the embodiments of this application, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0058] References to "one embodiment," "in some examples," or "some embodiments" as described in this specification mean that one or more embodiments of this specification include a particular feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in some examples," "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0059] Low-light images are images taken in extremely poor lighting conditions. Currently, these images typically suffer from low brightness, high noise, and blurred details. Processing low-light images is an important research direction in computer vision and image processing. For example, photos taken by electronic devices such as mobile phones at night in low light are usually low-light images.

[0060] Due to hardware limitations, low-light images captured by electronic devices often suffer from poor clarity and quality. For example, the camera sensors integrated into mobile phones are generally small due to size constraints, resulting in limited visible light capture in low-light environments. This often leads to lower contrast, resulting in dark, blurry photos that fail to meet user expectations. Compared to professional cameras, mobile phone camera sensors are much smaller, severely impacting image quality in low-light conditions. Furthermore, the aperture size of mobile phone camera lenses is also limited by physical dimensions, unlike DSLR cameras which can increase light intake in low light by widening the aperture. Therefore, underexposure is particularly pronounced in low-light environments with mobile phone cameras.

[0061] In addition, in order to improve exposure in low-light environments, electronic devices usually increase the camera's ISO value, but a high ISO value will lead to increased image noise, further reducing the quality of the photo.

[0062] Besides image quality issues caused by hardware limitations, even with current post-processing algorithms, it's impossible to meet users' demands for high-quality photos. In some cases, the poor quality of unprocessed raw image data (RAW) captured by mobile phones in low-light environments not only affects user experience but also poses challenges to subsequent image signal processing (ISP) processes. For example, detection based on low-quality RAW data leads to a significant drop in performance for tasks such as segmentation and recognition, ultimately resulting in unsatisfactory image quality. In other cases, while existing software algorithms can improve low-light photography to some extent, their effectiveness is often limited and cannot meet users' needs for high-quality photos.

[0063] In view of this, embodiments of this application provide a method for processing low-light images. This method provides an AI model for image enhancement of a first low-light image, and uses a first blurring result obtained by blurring the first low-light image, a first category label of the first low-light image, and the first low-light image itself as input parameters to the AI ​​model. This allows the AI ​​model to enhance the first low-light image, resulting in a first image. Therefore, this method not only optimizes the processing of the first low-light image obtained in a low-light environment, but also improves the accuracy of enhancement of the target region corresponding to the first category label in the first low-light image, and reduces enhancement of non-target regions in the first low-light image. For example, non-target regions can be the remaining regions in the first low-light image other than the target region corresponding to the first category label, such as background regions.

[0064] The AI ​​model for image enhancement of the first low-light image can be trained on a large amount of training data. For example, it can be an AI model based on controlled image generation technology, a diffusion-based AI model, or an AI model based on generative adversarial networks (GANs).

[0065] In addition, the first category label can be obtained by image recognition of the first dark light image, and may include, but is not limited to: green plants, grass, flowers, pets, portraits, buildings, ground, text, etc.

[0066] In one possible scenario, the method provided in this application embodiment can be applied to a low-light image processing system including an electronic device and a server. The electronic device can be used to acquire a first low-light image, and the server can be used to enhance the first low-light image acquired by the electronic device to obtain an enhanced low-light image, which can also be understood as the first image, and then return it to the electronic device for display or storage. The specific processes of the electronic device and the server can be found in the detailed description below, and will not be repeated here.

[0067] In another possible scenario, the method provided in this application embodiment can also be applied to an electronic device. For example, in this application embodiment, a trained AI model for image enhancement of a first low-light image can be deployed on an electronic device, thereby enabling the electronic device to perform image enhancement on the first low-light image using the AI ​​model after acquiring the first low-light image.

[0068] Optionally, the electronic device in this application embodiment can be a device with shooting capabilities. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA) and other portable electronic devices; or it can be a wearable device such as a watch or bracelet; or it can be a virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, etc. In short, this application embodiment does not limit the specific type of electronic device.

[0069] The following describes the electronic device according to the embodiments of this application. The electronic device may be the electronic device described above.

[0070] Figure 1 A schematic diagram of the hardware structure of a possible electronic device is shown. The electronic device 100 includes: a radio frequency (RF) circuit 110A, a wireless-fidelity (Wi-Fi) module 110B, a power supply 120, a processor 130, a memory 140, an input unit 150, a display unit 160, a camera 170, and a sensor module 180, among other components. Those skilled in the art will understand that... Figure 1 The hardware structure of the electronic device 100 shown in the figure does not constitute a limitation on the electronic device 100. The electronic device 100 provided in the embodiments of this application may include more or fewer components than shown, may combine two or more components, or may have different component configurations. Figure 1 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0071] The following is combined Figure 1 The various components of the electronic device 100 are described in detail below:

[0072] The RF circuit 110A can be used for receiving and transmitting data during communication or a call. Specifically, after receiving downlink data from the base station, the RF circuit 110A sends it to the processor 130 for processing; additionally, it sends uplink data to be transmitted to the base station. Typically, the RF circuit 110A includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc.

[0073] Furthermore, the RF circuit 110A can also communicate with other devices via a wireless communication network. The wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0074] The electronic device 100 can also perform communication services and interact with other electronic devices. Therefore, the electronic device 100 needs to have data transmission capabilities, meaning that the electronic device 100 needs to include a communication module. Although Figure 1 The RF circuit 110A, the Wi-Fi module 110B, and the communication interface are shown. Figure 1 The electronic device 100 may contain at least one of the aforementioned components or other communication modules (such as a Bluetooth module) for data transmission.

[0075] For example, when the electronic device 100 is a mobile phone, the electronic device 100 may include the RF circuit 110A, and may also include the Wi-Fi module 110B, or may include a Bluetooth module ( Figure 1 (Not shown in the image); when the electronic device 100 is a computer, the electronic device 100 may include a communication interface, and may also include the Wi-Fi module 110B, or may include a Bluetooth module (not shown in the image); Figure 1 (Not shown in the image); when the electronic device 100 is a tablet computer, the electronic device 100 may include the Wi-Fi module, or may include a Bluetooth module (not shown in the image); Figure 1 (Not shown in the image).

[0076] Wi-Fi technology is a short-range wireless transmission technology. The electronic device 100 can connect to an access point (AP) via the Wi-Fi module 110B, thereby enabling access to the data network. The Wi-Fi module 110B can be used for receiving and sending data during communication.

[0077] The memory 140 can be used to store software programs and modules. The processor 130 executes various functional applications and data processing of the electronic device 100 by running the software programs and modules stored in the memory 140. Optionally, the memory 140 may mainly include a program storage area and a data storage area. The program storage area may store the operating system (mainly including the software programs or modules corresponding to the kernel layer, system layer, application framework layer, and application layer).

[0078] Furthermore, the memory 140 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. In this embodiment, the memory 140 may store a first low-light image captured by a camera, or it may also store a first image obtained after image enhancement using the method provided in this embodiment.

[0079] The input unit 150 can be used to receive editing operations on various types of data objects, such as numbers or characters, input by the user, and to generate key signal inputs related to user settings and function control of the electronic device 100. Optionally, the input unit 150 may include a touch panel 151 and other input devices 152. The touch panel 151, also known as a touchscreen, can collect touch operations performed by the user on or near it (e.g., operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 151), and drive corresponding connection devices according to a pre-set program. In this embodiment, the touch panel 151 can collect various user operations, such as taking a photo.

[0080] Optionally, the other input device 152 may include, but is not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a mouse, and a joystick. For example, when the electronic device 100 is a laptop computer, the other input device 152 may be a mouse and / or a keyboard.

[0081] The display unit 160 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 100. The display unit 160 is the display system of the electronic device 100, used to present the interface and realize human-computer interaction. The display unit 160 may include a display panel 161. Optionally, the display panel 161 can be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. In this embodiment, the display unit 160 can be used to display a user interface, such as displaying a first image after image enhancement.

[0082] The processor 130 is the central controller of the electronic device 100. It connects various components via interfaces and lines, and executes software programs and / or modules stored in the memory 140, as well as calling data stored in the memory 140, to perform various functions and process data of the electronic device 100, thereby realizing multiple services based on the electronic device 100. In this embodiment, the processor 130 can be used to implement the low-light image processing method provided in this embodiment.

[0083] In some possible embodiments, the processor 130 can process the acquired RAW format low-light image to obtain a first low-light image; wherein the processing method can be, for example, the default processing method of the electronic device. It is understood that electronic devices such as mobile phones are currently configured by default with some processing models using convolutional neural networks. For example, considering the limitations of the processing power of electronic devices, a lightweight convolutional neural network processing model can be configured. Based on this, the AI ​​model provided in this application embodiment can perform further image enhancement processing on the first low-light image; wherein the first low-light image is obtained by the electronic device after processing the RAW format low-light image. In other embodiments, the processor 130 can also perform image recognition on the first low-light image to obtain a first category label; wherein the first category label may include, but is not limited to: green plants, grass, flowers, pets, portraits, buildings, ground, text, etc.

[0084] It should be noted that the first dark-light image can also be a RAW format dark-light image. This application does not limit this. For example, when deploying an AI model on an electronic device, the processor 130 can also use a RAW format dark-light image as the first dark-light image for image enhancement to obtain the first image.

[0085] In this embodiment, the processor 130 may also send the first low-light image and the first category label to the server via the mobile communication module 150 or the wireless communication module 160. Alternatively, the processor 130 may further enhance the first low-light image, which has been processed by the default processing method of the electronic device, based on an AI model for image enhancement of the first low-light image, to obtain the first image.

[0086] In this embodiment of the application, the processor 130 may also store and / or display the first image that has been image-enhanced from the server.

[0087] like Figure 1 As shown, the electronic device 100 includes a camera 170. The camera 170 may include one or more cameras, such as a rear camera, a front camera, etc. In this embodiment, the image captured by the camera 170 in a low-light environment is a first low-light image, which can be captured by either the rear camera or the front camera.

[0088] The electronic device 100 also includes a power supply 120 (such as a battery) for supplying power to various components. Optionally, the power supply 120 can be logically connected to the processor 130 through a power management system, thereby enabling the power management system to manage functions such as charging, discharging, and power consumption.

[0089] In this embodiment, the sensor module 180 may include at least an ambient light sensor 181. The ambient light sensor 181 can be used to detect the brightness of ambient light. When the brightness of the light at the time of shooting is detected to be less than a preset threshold, the image acquired at this time can be determined to be a first low-light image. Although Figure 1 As not shown in the diagram, the electronic device 100 may also include a variety of other sensors, which will not be described in detail here. These other sensors may include, but are not limited to, pressure sensors, barometric pressure sensors, accelerometers, distance sensors, fingerprint sensors, touch sensors, temperature sensors, etc.

[0090] The operating system (OS) involved in this application embodiment is the most basic system software running on the electronic device 100. The software system of the electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes an operating system adopting a layered architecture as an example to illustrate the software system architecture of the electronic device 100.

[0091] Figure 2 This is a software system architecture block diagram of an electronic device provided as an embodiment of this application. For example... Figure 2 As shown, the software system architecture of an electronic device can be a layered architecture. For example, the software can be divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into five layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the runtime and system libraries, the kernel layer, and the hardware layer.

[0092] The application layer can include a series of application packages. For example... Figure 2 As shown, the application layer can include camera, photo album, settings, call, music, Bluetooth, notes, and third-party applications. Third-party applications can include notes, videos, etc.

[0093] In one possible implementation, the application can be developed using Java, by calling the application programming interface (API) provided by the application framework layer. Developers can then interact with the underlying operating system layers (such as the hardware layer and kernel layer) to develop their own applications. This application framework layer primarily consists of a series of services and management systems within the operating system.

[0094] The application framework layer provides application programming interfaces and a programming framework for applications within the application layer. The application framework layer includes some predefined functions. For example... Figure 2 As shown, the application framework layer may include an activity manager, window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0095] The Activity Manager manages the lifecycle of each application and provides commonly used navigation and back functions, offering an interactive interface for all program windows.

[0096] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0097] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0098] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0099] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0100] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0101] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0102] The runtime includes the core libraries and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0103] The core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core libraries of the operating system. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0104] A system library can include multiple functional modules. For example: a surface manager, a media framework, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), etc.

[0105] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0106] The media framework supports playback and recording of various commonly used audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.164, MP3, AAC, and AMR; and also multiple image encoding formats, such as JPG and PNG.

[0107] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0108] A 2D graphics engine is a drawing engine for 2D drawing.

[0109] In some embodiments, a 3D graphics processing library can be used to draw 3D motion images, and a 2D graphics engine can be used to draw 2D motion images.

[0110] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0111] The hardware layer can include various sensors, such as ambient light sensors, gravity sensors, and touch sensors.

[0112] Typically, an electronic device 100 can run multiple applications simultaneously. In a simpler scenario, one application corresponds to one process; in a more complex scenario, one application can correspond to multiple processes. Each process has a unique process ID.

[0113] It should be understood that the hardware structure of electronic devices can be as follows: Figure 1 As shown, the software system architecture can be as follows: Figure 2 As shown, the software programs and / or modules corresponding to the software system architecture in the electronic device can be stored in the memory 140, and the processor 130 can run the software programs and applications stored in the memory 140 to execute the flow of a low-light image processing method provided in the embodiments of this application.

[0114] The following text is in conjunction with the accompanying drawings and... Figure 1 and Figure 2 The electronic device shown is a mobile phone, and the low-light image processing method provided in this application embodiment will be described in detail.

[0115] The embodiments of this application are applicable to electronic devices such as mobile phones or tablets, and can be specifically applied to scenarios where electronic devices capture low-light images through cameras.

[0116] For example, Figure 3 This is a comparative diagram illustrating a low-light image processing method provided in an embodiment of this application. For example... Figure 3As shown in interface 300, this is the first low-light image, which can also be understood as a low-light image obtained based on an electronic device. It can be a low-light image that has been processed by the electronic device using the default processing method for a RAW format low-light image, or it can be a RAW format low-light image itself. As can be seen from interface 300, due to the limitations of shooting in low-light environments, the image suffers from quality problems such as low brightness, unclear details, poor contrast, blurriness, and poor sharpness.

[0117] like Figure 3 As shown in interface 301, this is the first image, which is the low-light image obtained after image enhancement using the method provided in this embodiment. Compared to interface 300, interface 301, through the method provided in this embodiment, can improve the processing effect of low-light images, optimize the quality of low-light images, and improve the brightness and clarity of the images, as well as enhance the details and contrast of the images.

[0118] For example, Figure 4 Another comparative diagram illustrating a low-light image processing method provided in this application embodiment. Figure 4 The interface 300 is Figure 3 The interface 300 in the document will not be described again here.

[0119] like Figure 4 As shown in interface 401, this is the first image, which is the low-light image obtained after image enhancement using the method provided in this embodiment. Compared to interface 300, interface 401, through the method provided in this embodiment, can perform first image enhancement on a first region in the low-light image, where the first region can be, for example, a building in interface 401, or can be understood as a target region or region of interest (ROI); and can perform second image enhancement on a second region in the low-light image, where the second region can be, for example, other regions in interface 401 besides buildings, or can be understood as a background region, a non-target region, or a non-ROI.

[0120] The first image enhancement can be an image enhancement process, while the second image enhancement can be a process without image enhancement. This allows for image enhancement of the target area in the first low-light image, while leaving the non-target area unenhanced. This improves the processing effect of the low-light image, optimizes its quality, and results in a lower-quality low-light image during shooting.

[0121] like Figure 4As shown in interface 402, this is another first image, namely, a low-light image obtained after image enhancement using the method provided in this embodiment. Compared to interface 300, interface 402, through the method provided in this embodiment, can perform first image enhancement on a first region in the low-light image, where the first region can be, for example, a building in interface 402, or can be understood as a target region or region of interest (ROI); and can perform second image enhancement on a second region in the low-light image, where the second region can be, for example, other regions in interface 402 besides buildings, or can be understood as a background region, a non-target region, or a non-ROI.

[0122] The first image enhancement can be a more robust image enhancement, while the second image enhancement can be a less robust image enhancement. This allows for a higher degree of image enhancement on the target area of ​​the first low-light image, while a weaker degree of image enhancement is applied to the non-target area. This improves the processing effect of the low-light image, optimizes the quality of the resulting low-light image, and yields a lower-quality low-light image during shooting.

[0123] It should be noted that, in the embodiments of this application, the first image can be stored and displayed; or, the first dark-light image and the first image can be stored and displayed.

[0124] Furthermore, the low-light images at each stage in the embodiments of this application can be dynamic low-light images, and the methods provided in the embodiments of this application can also be applied to scenarios of shooting low-light videos in low-light environments. Dynamic low-light images can be understood as dynamic images including multiple frames of low-light images. For example, camera applications can provide a mode for shooting dynamic images, and dynamic low-light images can be understood as dynamic images captured in low-light environments. Taking low-light video as an example, in scenarios of shooting low-light video in low-light environments, image enhancement can be performed on each frame of the low-light image included in the first low-light video to obtain a second low-light video; or image enhancement can be performed on a specified frame in the first low-light video, and then image enhancement can be achieved on other frames in the first low-light video through motion estimation, thereby obtaining a second low-light video after image enhancement according to the embodiments of this application. It can be understood that the processing of dynamic low-light images can refer to the processing of low-light videos. In the embodiments of this application, static images of low-light images are used as an example for description.

[0125] Based on the foregoing Figure 4 and Figure 5 The present application describes the processing effects achievable using the method provided in the embodiments of this application. The following describes the implementation process of the low-light image processing method provided in this application, to illustrate how the method provided in this application can achieve the aforementioned results. Figure 3 and Figure 4 The processing effect shown can improve the processing effect of low-light images, optimize the quality of low-light images, and enhance the user's experience in shooting low-light images.

[0126] The method provided in this application embodiment can be implemented using an edge-cloud combined architecture, or it can be implemented on the edge side.

[0127] For example, Figure 5 This is a schematic diagram of the system architecture for a low-light image processing method provided in an embodiment of this application. Figure 5 This illustrates an edge-cloud hybrid architecture. From Figure 5 As can be seen, the system architecture may include, but is not limited to: 100 electronic devices and 500 servers. Among them,

[0128] (1) Electronic device 100, used to capture a first low-light image by camera 170.

[0129] For example, the electronic device 100 can determine that it has captured a first low-light image when the ambient light sensor 181 detects that the light intensity during the shooting is less than a preset threshold. For example, the preset threshold can be 40 or other values, and this application does not limit the specific value.

[0130] Optionally, the electronic device 100 can first acquire a RAW format low-light image through the camera 170, and then process the RAW format low-light image using the default processing method configured in the electronic device 100 to obtain a first low-light image. For example, when a mobile phone takes a picture, the stored photo and the interface displayed in the camera preview usually differ because electronic devices such as mobile phones 100 are generally equipped with some processing capabilities to perform preliminary optimization processing on the RAW format low-light image. For example, the default processing method can be processing using a convolutional neural network. Optionally, the default processing method can specifically be processing using a lightweight convolutional neural network, or it can be processing using a standard convolutional neural network.

[0131] Alternatively, the first low-light image can also be a RAW format low-light image. For example, when the method provided in this embodiment is implemented on the device side, that is, when the AI ​​model provided in this embodiment is deployed on the electronic device 100, the electronic device 100 can also directly perform image enhancement on the RAW format low-light image to obtain the first image.

[0132] In some possible examples, the electronic device 100 can also perform image recognition on the first low-light image to obtain a first category label. For example, the first category label may include, but is not limited to: green plants, grass, flowers, pets, portraits, buildings, ground, text, etc. Figure 3 or Figure 4 As shown, the first category label is "building" as an example.

[0133] Furthermore, the electronic device 100 can also perform filtering based on the first category label obtained. For example, when it is determined that the first category label belongs to a category that the AI ​​model deployed on the server can process, then interaction with the server 500 occurs, meaning the server 500 needs to perform image enhancement. Alternatively, when it is determined that the first category label does not belong to a category that the AI ​​model deployed on the server 500 can process, then no interaction with the server 500 is required, meaning the server 500 does not need to perform image enhancement. Category labels that are not processed by the AI ​​model deployed on the server 500 may include, for example, non-fixed category labels, causing the AI ​​model to be unable to establish a stable mapping relationship; and category labels with insufficient training data, resulting in low accuracy of the AI ​​model.

[0134] In this embodiment, the content of the information sent by the electronic device 100 to the server 500 is not limited. In one possible scenario, the electronic device 100 is also used to send a first low-light image to the server 500. In another possible scenario, the electronic device 100 is also used to send the first low-light image and a first category label of the first low-light image to the server 500.

[0135] It is understood that after receiving the first image from the server 500, the electronic device 100 can display or store it. For example, the electronic device 100 may display a preview of the first image in a camera application. Alternatively, the electronic device 100 may also display the first image in an application such as a photo album or other applications that access photo albums.

[0136] (2) Server 500 is used to receive a first low-light image sent from an electronic device. It can be understood that when server 500 does not receive a first category label from electronic device 100, it can perform image recognition on the first low-light image to obtain a first category label.

[0137] Alternatively, it can be used to receive a first low-light image and a first category label sent from an electronic device. It is understood that when server 500 receives the first category label from electronic device 100, it can apply the first category label from electronic device 100.

[0138] In one possible implementation, server 500 blurs the first low-light image to obtain a first blurring result, which is used to indicate areas where image enhancement is unnecessary. Image blurring is a technique that uses blurring to highlight a subject or achieve a specific effect in an image. It primarily works by reducing the sharpness of certain parts of the image, focusing the image's attention on a specific subject, thereby highlighting the subject.

[0139] For example, the server 500 can blur the first low-light image by performing blur detection on the first low-light image, such as an image captured in portrait mode. Figure 6 This is an example diagram of a low-light image 600 provided in an embodiment of this application. The low-light image 600 may include a portrait region 601 and a background region 602. Based on image recognition by the electronic device 100, a first category label of "portrait" can be obtained; and based on the blurring processing by the server 500, a first blurring result can be obtained as the background region 602. It can be understood that in portrait mode, the background region is an area that is not of interest. Thus, by obtaining the first blurring result, the server 500 can acquire non-target regions such as the background, thereby enabling weaker image enhancement or eliminating the need for image enhancement.

[0140] In another example, the server 500 can blur the first low-light image, or perform blurring processing on the first low-light image. For example, if the first low-light image is an image including a human figure captured in normal shooting mode, the server 500 can perform blurring processing on the image, thereby enabling different image enhancements to be applied to different areas of the first low-light image.

[0141] Based on the obtained first blurring result, first category label, and first low-light image, server 500 can perform image enhancement on the first low-light image to obtain a first image. Server 500 is also used to obtain input parameters for an AI model based on the first blurring result, first category label, and first low-light image, and to perform image enhancement through the AI ​​model to obtain the first image. The AI ​​model can be, for example, an AI model based on controllable image generation technology, a diffusion-based AI model, or a generative adversarial network-based AI model, etc.

[0142] The specific implementation method of image enhancement by server 500 will be described in detail in the process section below, and will not be repeated here.

[0143] It is understandable that server 500 is also used to send the first image to electronic device 100.

[0144] Furthermore, when using an electronic device to implement the method provided in this application embodiment, the functions of the electronic device 100 and the server 500 can be realized through the electronic device 100. Optionally, the AI ​​model deployed on the electronic device 100 can be a pre-trained AI model on the server 500. In this application embodiment, the AI ​​model deployed on the electronic device 100 can be the same as or different from the AI ​​model deployed on the server 500. For example, since the computing power of the electronic device 100 is limited, a lightweight AI model can be deployed, while the server 500 has strong computing power and therefore can deploy a standard AI model.

[0145] based on Figure 5 The system architecture combining edge and cloud is shown below. The following describes the flow of the method provided in the embodiments of this application. For example, Figure 7 This is a flowchart illustrating a low-light image processing method provided in an embodiment of this application. The process may include the following steps:

[0146] Step 701: Electronic device 100 captures a first low-light image, wherein the light intensity of the first low-light image is less than a preset threshold at the time of capture.

[0147] For example, electronic device 100 can capture a first low-light image using camera 170. Optionally, electronic device 100 can determine that the image captured by electronic device 100 belongs to the first low-light image under low-light conditions, such as nighttime, when the ambient light sensor 181 detects that the light intensity during shooting is less than a preset threshold. In another example, if electronic device 100 detects that the light intensity during shooting is greater than or equal to a preset threshold using ambient light sensor 181, it can determine that the captured image does not belong to the image captured under low-light conditions, i.e., a normal image, and the subsequent steps 702 to 706 can be omitted.

[0148] In one possible implementation, the first dark-light image is obtained by processing a RAW format dark-light image by the electronic device 100. Optionally, the electronic device 100 uses a convolutional neural network to process the RAW format dark-light image. The convolutional neural network can be a lightweight convolutional neural network or a standard convolutional neural network, depending on factors such as the computing power of the electronic device 100.

[0149] Step 702: Electronic device 100 sends a first low-light image and a first category label of the first low-light image to server 500.

[0150] In some possible embodiments, the first category label can be obtained by the electronic device 100 through image recognition of the first low-light image. For example, the first category label may include, but is not limited to: green plants, grass, flowers, pets, portraits, buildings, ground, text, etc. Figure 3 and Figure 4 As shown, the first category label is "building" as an example.

[0151] In some possible embodiments, the electronic device 100 can obtain a first category label and then perform filtering based on that label. When it is determined that the first category label belongs to a category that the AI ​​model deployed on the server can process, interaction with the server 500 occurs, i.e., the processing shown in steps 702 to 706 continues. When it is determined that the first category label does not belong to a category that the AI ​​model deployed on the server can process, no interaction with the server 500 is required, i.e., the processing shown in steps 702 to 706 continues. For example, category labels that are not processed by the AI ​​model deployed on the server 500 may include: non-fixed category labels, causing the AI ​​model to be unable to establish a stable mapping relationship; and category labels with insufficient training data, resulting in low accuracy of the AI ​​model. Thus, by filtering through the electronic device, processing of scenarios where the image enhancement effect of the server 500 is poor can be reduced, thus reducing resource waste.

[0152] In some other possible embodiments, the electronic device 100 may also directly send the first dark light image to the server 500. After receiving the first dark light image sent by the electronic device 100, the server 500 can perform image recognition on the first dark light image to obtain a first category label. In this way, the computing power requirements of the electronic device 100 can be reduced, and more processing can be performed by utilizing the powerful computing capabilities of the server 500.

[0153] Step 703: Server 500 blurs the first dark light image to obtain a first blurring result, which is used to indicate areas that do not require image enhancement.

[0154] In some possible embodiments, the server 500 blurs the first low-light image, which can be done by performing blur detection on the first low-light image. For example, the first low-light image can be an image captured in portrait mode. For instance, the first low-light image can be an image captured in portrait mode, and the first low-light image can be as follows... Figure 6 The dark-light image 600 is shown in the image. For example, the first blurring result can be as follows: Figure 6 The background area 602 is shown in the image.

[0155] In other possible embodiments, the server 500 can blur the first low-light image, or perform a blurring process on the first low-light image. For example, if the first low-light image is an image including a human figure captured in normal photography mode, the server 500 can blur this image, thereby enabling different regions of the first low-light image to be enhanced differently. For example, after blurring the first low-light image, the server 500 can obtain... Figure 6 The image shown is a low-light image.

[0156] Step 704: Server 500 obtains the input parameters of the AI ​​model based on the first blurring result, the first category label, and the first dark light image, and performs image enhancement through the AI ​​model to obtain the first image.

[0157] For example, Figure 8 This is a schematic diagram illustrating information changes on server 500 provided in an embodiment of this application. For example, server 500 can receive a first low-light image and a first category label from electronic device 100; and, through the processing in step 703, a first blurring result can be obtained based on the first low-light image. Exemplarily, server 500 obtaining input parameters for an artificial intelligence (AI) model based on the first blurring result, the first category label, and the first low-light image may include the following components:

[0158] (A) Generate a mask based on the first blurring result. For example, the mask can be understood as a binary image.

[0159] (B) Generate a prompt word based on the first category label. Then, encode the prompt using a text encoder to generate a text embedding. Text embedding is a technique that maps discrete text symbols (such as prompts) into continuous low-dimensional vectors. Its core goal is to capture semantic, syntactic, and contextual information through vectorized representation, thereby supporting the server's understanding and processing of natural language.

[0160] For example, a prompt is used to instruct the AI ​​model how to process the image. For instance, a prompt might include: deblurring the first low-light image, performing super-resolution processing on regions in the first low-light image that are indicated by building categories, and optimizing the blurring of regions in the first low-light image that are not indicated by building categories.

[0161] Another example is that a text encoder can be used to encode the first category label to generate a text embedding.

[0162] (C) Encode the first low-light image using an image encoder to generate a first latent space. The first latent space can be understood as a description of the encoding stage.

[0163] In statistics, machine learning, and deep learning, latent space refers to a hypothetical, typically unobservable, multidimensional space containing latent features or properties of data. Latent space plays a crucial role in various generative models, such as autoencoders (including image encoders and decoders), variational autoencoders (VAEs), and generative adversarial networks (GANs). Latent space can be viewed as a compressed representation of data, where each point or vector represents a latent pattern or feature in the dataset. In machine learning, models learn how to map input data into the latent space and how to reconstruct the original data from points in the latent space, facilitating image enhancement tasks.

[0164] In one possible embodiment, server 500 can use the mask obtained through (A), the text embedding obtained through (B), and the first latent space obtained through (C) as input parameters for the AI ​​model. It is understood that by generating prompt words based on the first category label and inputting them into the AI ​​model, server 500 can perform targeted processing based on the first category label obtained by electronic device 100, such as performing first image enhancement on the target region corresponding to the first category label, thereby achieving depth optimization of the target region; and by inputting the first blurring result into the AI ​​model, server 500 can perform a second image enhancement process based on the first blurring result, which is weaker than the first image enhancement process, thus improving the hierarchy of the obtained second image.

[0165] In another possible embodiment, server 500 may also use the text embedding obtained through (B) and the first latent space obtained through (C) as input parameters for the AI ​​model. It can be understood that the AI ​​model, through text embedding, can determine the target region corresponding to the first category label, and based on the target region, can also determine the non-target regions in the first low-light image, i.e., regions that do not require image enhancement.

[0166] In another possible embodiment, server 500 may also use the mask obtained through (A) and the first latent space obtained through (C) as input parameters for the AI ​​model. It can be understood that the AI ​​model can determine the non-target region corresponding to the first blurring result through the mask, and can also determine the target region in the first image based on the non-target region, or it can be understood as the target region corresponding to the first category label, that is, the region that needs to be enhanced.

[0167] After obtaining the input parameters of the AI ​​model based on any of the foregoing embodiments, the server 500 performs image enhancement using the AI ​​model. For example, a first image enhancement is performed on a first region determined based on a first category label using the AI ​​model; a second image enhancement is performed on a second region determined based on a first blurring result using the AI ​​model; wherein the degree of the first image enhancement is greater than the degree of the second image enhancement.

[0168] For example, the first region can be as follows Figure 4 The area shown in the building, the second area can be as follows Figure 4 The areas not shown in the diagram. For example... Figure 4 As shown in interface 401, the building area underwent first image enhancement, which can also be understood as image enhancement; the non-building area underwent second image enhancement, which can also be understood as no image enhancement. For example... Figure 4 As shown in interface 402, the building area has undergone a first image enhancement, which can be understood as a relatively strong image enhancement; the non-building area has undergone a second image enhancement, which can be understood as a relatively weak image enhancement.

[0169] Or, for example, the first region can be as follows: Figure 6 The portrait area shown in the image is 601. The second area can be as follows: Figure 6 The background area 602 is not shown as a human figure. It can be understood that a first image enhancement can be performed on the human figure area 601, and a second image enhancement can be performed on the background area 602.

[0170] In this embodiment, after image enhancement using an AI model, a second latent space is obtained; then, an image decoder is used to decode the second latent space to obtain a first image. The second latent space can be understood as a description of the decoding stage. It is understood that the AI ​​model is pre-trained based on training data and can perform multiple iterations of model inference based on input parameters, thereby obtaining the second latent space, which can be used to reconstruct the first image after image enhancement.

[0171] The AI ​​model can be a large AI model based on controllable image generation technology, or it can be other types of large models, such as diffusion-based AI models or generative adversarial networks.

[0172] For example, image enhancement in the embodiments of this application may include, but is not limited to, at least one of the following: deblurring, super-resolution processing, image sharpening, bokeh optimization, brightness optimization, or contrast optimization. Specifically, deblurring can be used to eliminate blurring in the first low-light image, improving its clarity; super-resolution processing can be used to increase the resolution of the first low-light image, improving its image quality; image sharpening can be used to improve the clarity and detail of the subject in the first low-light image, where the subject can be determined based on a first category label; bokeh optimization can be used to enhance the layering in the low-light image, making the subject stand out more; and brightness optimization or contrast optimization can be used to improve the image quality of the low-light image.

[0173] Step 705: Server 500 sends the first image to electronic device 100.

[0174] Step 706: Electronic device 100 displays the first image.

[0175] In one possible scenario, the electronic device 100 can display a preview of the first image in a camera application. In another possible scenario, the electronic device 100 can also store and display the first image in a photo album application. In other possible scenarios, the electronic device can also display the first image in other applications that can access the photo album, etc.

[0176] For example, Figure 9 This is a schematic diagram of an interface for a low-light image processing method provided in an embodiment of this application. Figure 9 As shown in interface 900A, this is the camera preview interface. From interface 900A, it can be seen that the following can be displayed: Figure 4 The first dark-light image shown in interface 300.

[0177] In response to a click on the shooting control in interface 900A, the mobile phone can display interface 900B. As shown in interface 900B, it is still a camera preview interface. The difference from interface 900A is that interface 900B can display a preview control 901, which can display a low-light image captured based on interface 900A. Before processing the first low-light image using the method provided in this application embodiment, the preview image displayed in preview control 901 can be a preview of the first low-light image. After processing the first low-light image using the method provided in this application embodiment, the preview image displayed in preview control 901 can be a preview of the first image.

[0178] For example, in response to a click on the preview control 901, the mobile phone can display interface 900C. As shown in interface 900C, this can be a large image viewing interface for the photo album. It can be seen from interface 900C that at this time, the following can be displayed: Figure 4The first image shown in interface 401.

[0179] Additionally, the electronic device can also store the first low-light image. It is understood that the photo album can include both the first low-light image and the first image.

[0180] The method provided in this application, by adopting an edge-cloud combined system architecture, can fully utilize the preliminary processing capabilities of electronic devices and the powerful computing resources of the cloud, thereby improving the processing effect of low-light images and optimizing their quality. Furthermore, by using the first blurring result, the first category label, and the first low-light image as input parameters for the AI ​​model, the layering and naturalness of the first image can be enhanced, highlighting the subject in the low-light image.

[0181] Based on the above embodiments, this application also provides an electronic device, which includes multiple functional modules; the multiple functional modules interact to realize the functions performed by the electronic device in the methods described in the embodiments of this application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations. For example, [the following is an example of implementation details]. Figure 7 Steps 701, 702, 705, and 706 are performed by the electronic device in the illustrated embodiment.

[0182] Based on the above embodiments, this application also provides an electronic device, which includes at least one processor and at least one memory, wherein the at least one memory stores computer program instructions. When the electronic device is running, the at least one processor executes the functions performed by the electronic device in the various methods described in the embodiments of this application. For example, when executing... Figure 7 Steps 701, 702, 705, and 706 are performed by the electronic device in the illustrated embodiment.

[0183] Based on the above embodiments, this application also provides a server, which includes multiple functional modules; the multiple functional modules interact to implement the functions performed by the server in the methods described in the embodiments of this application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on specific implementations. For example, execution... Figure 7 Steps 702 to 705 are performed by the server in the illustrated embodiment.

[0184] Based on the above embodiments, this application also provides a server, which includes at least one processor and at least one memory, wherein the at least one memory stores computer program instructions. When the server is running, the at least one processor executes the functions performed by the server in the various methods described in the embodiments of this application. For example, ... Figure 7 Steps 702 to 705 are performed by the server in the illustrated embodiment.

[0185] Based on the above embodiments, this application also provides a low-light image processing system. This system may include the electronic device and server described above.

[0186] Based on the above embodiments, this application also provides a computer program product, which includes a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the methods described in the embodiments of this application.

[0187] Based on the above embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the methods described in the embodiments of this application.

[0188] Based on the above embodiments, this application also provides a chip for reading computer programs stored in a memory to implement the methods described in the embodiments of this application.

[0189] Based on the above embodiments, this application provides a chip system including a processor for supporting a computer device in implementing the methods described in the embodiments of this application. In one possible design, the chip system further includes a memory for storing necessary programs and data of the computer device. This chip system may be composed of chips or may include chips and other discrete devices. Those skilled in the art will understand that the embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0192] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0193] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of protection of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for processing low-light images, characterized in that, The method includes: An electronic device captures a first low-light image, wherein the light intensity of the first low-light image at the time of capture is less than a preset threshold. The electronic device sends the first low-light image and a first category label of the first low-light image to the server; The server blurs the first dark-light image to obtain a first blurring result, which is used to indicate areas that do not require image enhancement. The server obtains the input parameters of the artificial intelligence (AI) model based on the first blurring result, the first category label, and the first low-light image, and performs image enhancement through the AI ​​model to obtain the first image; The server sends the first image to the electronic device; The electronic device displays the first image.

2. The method according to claim 1, characterized in that, The first low-light image is obtained by the electronic device after processing the original RAW format low-light image.

3. The method according to claim 2, characterized in that, The method further includes: The electronic device uses a convolutional neural network to process the RAW format low-light image.

4. The method according to any one of claims 1 to 3, characterized in that, The image enhancement using the AI ​​model includes: The first image enhancement is performed on the first region using the AI ​​model, where the first region is determined based on the first category label; The AI ​​model is used to perform a second image enhancement on the second region, which is determined based on the first blurring result; The degree of enhancement of the first image is stronger than the degree of enhancement of the second image.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The server generates a prompt word based on the first category label; The server obtains the input parameters of the AI ​​model based on the first category label, including: The server obtains the input parameters based on the prompt.

6. The method according to claim 5, characterized in that, The input parameters for obtaining the AI ​​model based on the first blurring result, the first category label, and the first low-light image include: Based on the first blurring result, generate a mask; The first dark-light image is encoded using an image encoder to generate a first latent space; The prompt is encoded using a text encoder to generate a text embedding; The mask, the first latent space, and the embedded text are used as input parameters for the AI ​​model.

7. The method according to any one of claims 1 to 6, characterized in that, The process of enhancing the image using the AI ​​model to obtain the first image includes: Image enhancement is performed using the AI ​​model to obtain the second latent space; The second latent space is decoded using an image decoder to obtain the first image.

8. The method according to any one of claims 1 to 7, characterized in that, The image enhancement includes at least one of the following: Deblurring, super-resolution processing, image sharpening, bokeh optimization, brightness optimization, or contrast optimization.

9. An electronic device, characterized in that, It includes at least one processor coupled to at least one memory, the at least one processor being configured to read a computer program stored in the at least one memory to execute the method performed by the electronic device as described in any one of claims 1 to 8.

10. A server, characterized in that, It includes at least one processor coupled to at least one memory, the at least one processor being configured to read a computer program stored in the at least one memory to execute the method executed by the server as described in any one of claims 1 to 8.

11. A low-light image processing system, characterized in that, Includes the electronic device as described in claim 9 and the server as described in claim 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 8.

13. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 8.

Images