Image processing method, electronic device, and computer-readable storage medium

Abstract

The application relates to the field of image processing, in particular to an image processing method, an electronic device and a computer readable storage medium. The method comprises the following steps: displaying a first interface on a display screen of the electronic device, the first interface being a shooting interface of a camera application, the first interface comprising a first control, and the first control being used for triggering shooting; in response to a first operation of a user on the first control, acquiring an original image collected by an image sensor of a camera in the electronic device; performing denoising processing on the original image to obtain a first image after denoising; acquiring a first noise image used for representing image details according to the original image; performing superposition processing on the first image and the first noise image to obtain a second image after processing; and displaying a second interface on the display screen, the second interface comprising the second image. Through the above method, the image processing effect is effectively improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image processing method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] During the process of a camera capturing an image, the image sensor inevitably receives some noise signals. To improve image quality, noise reduction processing is usually performed on the captured image to filter out the noise. However, since noise and signals are difficult to distinguish, the noise-reduced image is prone to loss of detail. Summary of the Invention

[0003] This application provides an image processing method, an electronic device, and a computer-readable storage medium that solves the problem of missing details in images after denoising.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, an image processing method is provided, applied to an electronic device, the method comprising:

[0006] The electronic device displays a first interface, which is a shooting interface for a camera application. The first interface includes a first control, which is used to trigger shooting.

[0007] In response to the user's first operation on the first control, the original image captured by the image sensor of the camera in the electronic device is obtained;

[0008] The original image is denoised to obtain a denoised first image;

[0009] A first noise image representing image details is obtained from the original image;

[0010] The first image and the first noisy image are superimposed to obtain the processed second image;

[0011] The second interface is displayed on the screen, and the second interface includes the second image.

[0012] In this embodiment, noise information representing image details is obtained from the original image and superimposed onto the denoised image, thereby compensating for the lack of details in the denoised image and effectively improving the image processing effect.

[0013] The original image can be in RAW format.

[0014] In one implementation of the first aspect, obtaining a first noise image representing image details from the original image includes:

[0015] The difference image is obtained by subtracting the first image from the original image.

[0016] The first noise image is obtained by acquiring the channel data used to represent the brightness value in the difference image.

[0017] The image subtraction process involves subtracting the pixel value of a second pixel from the pixel value of a first pixel. The first pixel is a pixel in the original image, and the second pixel is a pixel in the first image that corresponds to the first pixel.

[0018] The image subtraction process involves subtracting the pixel value of the second pixel from the pixel value of the first pixel. The first pixel is a pixel in the original image, and the second pixel is a pixel in the first image that corresponds to the first pixel.

[0019] In some implementations, the first image and the first noise image are RGB images, and correspondingly, the difference image is also an RGB image. The difference image can be converted into a YUV image, and the Y channel data can be obtained from the YUV image to obtain the first noise image.

[0020] YUV is a color encoding method. In a YUV image, the Y channel represents the brightness of the image, while the U and V channels represent the color.

[0021] In this embodiment, the first noisy image includes only the channel data of the brightness value and does not include the data of the color channel. Accordingly, after subsequent superposition processing, the image details are increased without changing the color of the image, which helps to improve the image processing effect.

[0022] In one embodiment, image subtraction processing may include:

[0023] The first image is de-mosaiced to obtain the processed first image;

[0024] The original image is de-mosaiced to obtain the processed original image;

[0025] The difference image is obtained by subtracting the processed first image from the processed original image.

[0026] In one implementation of the first aspect, the step of superimposing the first image and the first noise image to obtain the processed second image includes:

[0027] A second noise image and a third noise image are obtained based on the first noise image, wherein the image frequency of the second noise image is higher than the image frequency of the third noise image;

[0028] Based on the brightness values ​​of pixels in the first image, noise fusion processing is performed on the second noise image and the third noise image to obtain a fourth noise image;

[0029] The second image is obtained by superimposing the fourth noise image and the first image.

[0030] In some implementations, different convolution kernels can be used to perform Gaussian filtering on the first noisy image to obtain the second and third noisy images.

[0031] It is understood that the second noise image corresponds to high-frequency noise, and the third noise image corresponds to low-frequency noise. High-frequency noise typically contains fewer image details, while low-frequency noise typically contains more image details. In this embodiment, by acquiring noise images of different frequencies, appropriate noise images can be selected for superposition during subsequent overlay processes, which helps to improve the flexibility and effectiveness of image processing.

[0032] In one implementation of the first aspect, the step of performing noise fusion processing on the second noise image and the third noise image based on the brightness values ​​of pixels in the first image to obtain a fourth noise image includes:

[0033] If the brightness value of the third pixel is less than the first threshold, the brightness value of the fourth pixel is determined according to the first weight, the first brightness value, and the second brightness value.

[0034] Wherein, the third pixel is a pixel in the first image, the fourth pixel is a pixel in the fourth noisy image corresponding to the third pixel, the first brightness value is the brightness value in the third noisy image corresponding to the third pixel, and the second brightness value is the brightness value in the second noisy image corresponding to the third pixel.

[0035] In one implementation of the first aspect, the step of performing noise fusion processing on the second noise image and the third noise image based on the brightness values ​​of pixels in the first image to obtain a fourth noise image includes:

[0036] If the brightness value of the third pixel is greater than the second threshold, then the brightness value of the fourth pixel is determined based on the second weight, the first brightness value, and the second brightness value.

[0037] In one implementation of the first aspect, the step of performing noise fusion processing on the second noise image and the third noise image based on the brightness values ​​of pixels in the first image to obtain a fourth noise image includes:

[0038] If the first brightness value of the third pixel is greater than the first threshold and less than the second threshold, then the third weight is determined based on the first weight, the second weight and the first brightness value.

[0039] The brightness value of the fourth pixel is determined based on the first brightness value, the second brightness value, and the third weight.

[0040] In one implementation of the first aspect, the step of superimposing the fourth noise image and the first image to obtain the processed second image includes:

[0041] The superposition weight is obtained based on the edge information in the first image;

[0042] The fourth noise image and the first image are superimposed according to the superposition weight to obtain the processed second image.

[0043] In one example, when the brightness value of the third pixel is less than the first threshold, the brightness value of the fourth pixel can be calculated using the formula noise = α1 × noise1 + (1 - α1)noise2, where noise represents the brightness value of the fourth pixel, noise1 represents the first brightness value, noise2 represents the second brightness value, and α1 represents the first weight. When the brightness value of the third pixel is greater than the second threshold, the brightness value of the fourth pixel can be calculated using the formula noise = α2 × noise1 + (1 - α2)noise2, where α2 represents the second weight. When the brightness value of the third pixel is greater than the first threshold and less than the second threshold, the brightness value of the fourth pixel can be calculated using the formula noise = α3 × noise1 + (1 - α3)noise2, where α3 represents the third weight.

[0044] The third weight can be calculated using the formula: Where t3 represents the brightness value of the third pixel, t1 represents the first threshold, and t2 represents the second threshold.

[0045] In this embodiment, two thresholds are set, and the middle range of the two thresholds corresponds to the transition region of image brightness. Within this transition region, the weights of the second and third noise images are smoothly adjusted, making the details in the superimposed fourth noise image smoother. This effectively reduces the problem of abrupt changes in details after noise superposition, thereby improving the image processing effect.

[0046] In one implementation of the first aspect, obtaining the overlay weight based on the edge information in the first image includes:

[0047] Detect edge information in the first image to obtain the edge image of the first image;

[0048] The overlay weights are obtained based on the edge image.

[0049] In some examples, an image detection model can be used to detect edge information in the first image. This image detection model can be a neural network model or other algorithmic model capable of image edge detection.

[0050] In one implementation of the first aspect, detecting edge information in the first image to obtain an edge image of the first image includes:

[0051] The channel data representing brightness values ​​in the first image are downsampled to obtain a sampled image;

[0052] The sampled image is subjected to a first blurring process to obtain a first blurred image;

[0053] The first blurred image is subjected to edge detection processing to obtain the edge image.

[0054] In this embodiment, downsampling is used to improve the efficiency of subsequent data processing.

[0055] In one implementation of the first aspect, obtaining the overlay weights based on the edge image includes:

[0056] The edge image is subjected to a second blurring process to obtain the blurred edge image;

[0057] The superposition weight is obtained based on the blurred edge image.

[0058] In this embodiment of the application, the second blurring process can smooth the uneven parts of the edge image, which can effectively reduce the abruptness of image edge details in the superimposed image and improve the image processing effect.

[0059] In one implementation of the first aspect, the superposition weight includes the weight corresponding to each pixel in the fourth noisy image;

[0060] The step of obtaining the superposition weight based on the blurred edge image includes:

[0061] If the pixel value of the fifth pixel is the first value, then the weight of the sixth pixel is set to the fourth weight.

[0062] Wherein, the fifth pixel is a pixel in the blurred edge image, the sixth pixel is a pixel in the fourth noise image corresponding to the fifth pixel, and the first value is the minimum brightness value in the blurred edge image.

[0063] In one implementation of the first aspect, obtaining the superposition weight based on the blurred edge image includes:

[0064] If the pixel value of the fifth pixel is the second value, then the weight of the sixth pixel is set to the fifth weight.

[0065] Wherein, the second value is the maximum brightness value in the blurred edge image, and the fourth weight is less than the fifth weight.

[0066] In one implementation of the first aspect, obtaining the superposition weight based on the blurred edge image includes:

[0067] If the pixel value of the fifth pixel is greater than the first value and less than the second value, then the sixth weight is calculated based on the fourth weight, the fifth weight, and the pixel value of the fifth pixel.

[0068] Set the weight of the sixth pixel to the sixth weight value.

[0069] In this embodiment, it is equivalent to calculating the superposition weight of different texture regions based on the edge image, so that the edge details in the superimposed image are smoother, which can effectively reduce the problem of abrupt changes in details after superposition noise, thereby improving the image processing effect.

[0070] In a second aspect, a chip system is provided, the chip system including a processor coupled to a memory, the processor executing a computer program stored in the memory to implement the method as described in any of the first aspects.

[0071] Thirdly, an electronic device is provided, the electronic device including a processor for running a computer program stored in a memory, such that the electronic device performs the method as described in any of the first aspects.

[0072] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by one or more processors, implements the method as described in any of the first aspects.

[0073] Fifthly, a computer program product is provided that, when run on an electronic device, enables the electronic device to perform the method described in any of the first aspects. Attached Figure Description

[0074] Figure 1 This is a schematic diagram of a denoised image provided in an embodiment of this application;

[0075] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0076] Figure 3 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application;

[0077] Figure 4 This is a schematic flowchart of the image processing method provided in the embodiments of this application;

[0078] Figure 5 This is a flowchart illustrating the overlay processing method provided in an embodiment of this application;

[0079] Figure 6 This is a schematic diagram of the fusion weights provided in an embodiment of this application;

[0080] Figure 7 This is a schematic diagram of the edge image provided in the embodiments of this application;

[0081] Figure 8 This is a schematic diagram of the superimposed weights provided in the embodiments of this application;

[0082] Figure 9 This is a schematic diagram of the image processing flow provided in the embodiments of this application. Detailed Implementation

[0083] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limiting purposes, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details.

[0084] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0085] It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between the associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0086] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," "fourth," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0087] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0088] During the process of a camera capturing an image, the image sensor inevitably receives some noise signals. To improve image quality, noise reduction processing is usually performed on the captured image to filter out the noise. However, since noise and signals are difficult to distinguish, the noise-reduced image is prone to loss of detail.

[0089] See Figure 1 This is a schematic diagram of a denoised image provided in an embodiment of this application. Figure 1 As shown in region 11, the denoised image is missing some image details, resulting in a blurry and unclear image.

[0090] Based on this, embodiments of this application provide an image processing method. In this embodiment, noise information representing image details is obtained from the original image, and this noise information is superimposed on the denoised image to compensate for the lack of details in the denoised image, effectively improving the image processing effect.

[0091] The image processing method provided in this application can be applied to electronic devices with screens and shooting functions. These electronic devices can be tablets, mobile phones, cameras, camcorders, smart TVs, wearable devices, smart screens, laptops, ultra-mobile personal computers (UMPCs), netbooks, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, and other electronic devices. This application does not limit the specific type of electronic device.

[0092] See Figure 2This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a touch sensor 180K, an ambient light sensor 180L, etc.

[0093] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0094] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. For example, processor 110 is used to execute the image processing method described in the embodiments of this application.

[0095] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0096] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0097] External memory 120 generally refers to external storage. In the embodiments of this application, external storage refers to storage other than the memory of electronic devices and the cache of processors. This storage is generally non-volatile memory.

[0098] Internal memory 121, also known as "RAM," can be used to store executable program code for a computer, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a given function (such as sound playback, image playback, etc.).

[0099] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized display, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1. In some embodiments, electronic device 100 displays a user interface through the display screen 194.

[0100] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0101] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0102] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. For example, camera 193 is used to capture images of a user looking at display screen 194.

[0103] Electronic device 100 also includes various sensors that can convert different physical signals into electrical signals. For example, pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. Gyroscope sensor 180B can be used to determine the motion posture of electronic device 100. Barometric pressure sensor 180C is used to measure air pressure. Magnetic sensor 180D includes a Hall sensor. Accelerometer sensor 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). Distance sensor 180F is used to measure distance. Electronic device 100 can measure distance using infrared or laser. Proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photodetector, such as a photodiode. Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 according to the sensed ambient light brightness. Fingerprint sensor 180H is used to collect fingerprints. Electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application lock, fingerprint photography, fingerprint answering of calls, etc. Temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. The bone conduction sensor 180M can acquire vibration signals.

[0104] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0105] For example, in this embodiment of the application, the touch sensor 180K can detect the user's click operation on the application icon, and pass the detected click operation to the application processor to determine that the click operation is used to start or run the application, and then execute the application's running operation.

[0106] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0107] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0108] The above is a detailed description of the embodiments of this application using electronic device 100 as an example. It should be understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on electronic device 100. Electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0109] In addition, an operating system runs on top of the aforementioned components. Examples include iOS, Android (an open-source operating system), and Windows. Applications can be installed and run on this operating system.

[0110] The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0111] Figure 3 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0112] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer, and the driver layer.

[0113] The application layer can include a series of application packages. For example... Figure 3 As shown, an application package can include a camera and a gallery. For example, images captured by the camera app can be stored in the gallery app.

[0114] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 3 As shown, the application framework layer may include a camera access interface, which provides an application programming interface and programming framework for camera applications.

[0115] The hardware abstraction layer (HAL) is an interface layer located between the operating system kernel and the hardware circuitry; it is used to abstract the hardware. For example... Figure 3 As shown, the hardware abstraction layer can include camera devices and algorithm libraries. The camera devices are used to abstract camera hardware into instances for use by upper-layer applications. The algorithm library provides various data processing algorithms. For example... Figure 3 As shown, the algorithm library includes the image processing algorithms provided in the embodiments of this application.

[0116] The driver layer is used to drive hardware devices. For example... Figure 3 As shown, the driver layer may include a camera driver for driving camera-related hardware devices, such as image sensors and image signal processors.

[0117] See Figure 3 The image processing procedure is described below.

[0118] 1. Display the first interface of the camera application on the screen of the electronic device.

[0119] The first interface is the shooting interface of the camera application. The first interface includes a first control, which is used to trigger shooting.

[0120] 2. In response to the user's first operation on the first control, the application layer sends a shooting command to the hardware abstraction layer through the camera access interface of the application framework layer.

[0121] The first operation can be a click or a double click, etc.

[0122] 3. The hardware abstraction layer receives the shooting command and sends the driving command to the camera driver in the driver layer.

[0123] 4. The camera driver in the driver layer receives the driver instruction and drives the image sensor of the camera in the hardware layer to acquire signals.

[0124] 5. The image sensor in the hardware layer sends the acquired signals to the camera device in the hardware abstraction layer through the camera driver in the driver layer.

[0125] 6. The camera device in the hardware abstraction layer generates a raw image (RAW format image) based on the signal acquired by the image sensor and sends the raw image to the algorithm library of the hardware abstraction layer.

[0126] The filter covering the image sensor allows it to capture color information. For example, this filter could be a red / green / blue Bayer filter, which only allows red / green / blue light to pass through. This allows the image sensor to detect the intensity of the red / green / blue light and then convert it into corresponding electrical signals to generate a RAW format image. RAW format images typically use a Bayer array arrangement; therefore, RAW images can also be called Bayer array images.

[0127] 7. The algorithm library of the hardware abstraction layer uses the image processing algorithm of the embodiment of this application to process the original image, obtain the processed image (YUV format image), and send the processed image to the camera device of the hardware abstraction layer.

[0128] 8. The camera device in the hardware abstraction layer sends a driving command to the camera driver in the driver layer based on the processed image, so as to drive the image signal processor in the hardware layer to generate an RGB image (second image) based on the processed image.

[0129] 9. The image signal processor in the hardware layer sends the generated RGB image to the camera device in the hardware abstraction layer through the camera driver, and then sends it to the camera application in the application layer through the camera access interface in the application framework layer.

[0130] 10. The camera application at the application layer receives the RGB image and displays a second interface, which includes the RGB image.

[0131] 11. The camera application at the application layer can send the RGB image to the gallery application so that the user can view the RGB image in the gallery application.

[0132] The image processing method of this application embodiment is described below.

[0133] See Figure 4This is a schematic flowchart of the image processing method provided in an embodiment of this application. It is intended as an example and not a limitation. Figure 4 As shown, the image processing method may include the following steps:

[0134] S401, Denoise the original image to obtain the first denoised image.

[0135] As described in step 6 above, the raw image is an image captured by the camera's image sensor that has not been processed by the image signal processor. The raw image can be in RAW format.

[0136] In some implementations, filters can be used to denoise the original image. Optionally, the filters can be median filters, mean filters, Wiener filters, etc.

[0137] In other implementations, neural networks can be used to denoise the original image. Specifically, the original image is input into a trained neural network, which outputs a denoised first image.

[0138] In this embodiment of the application, the method of noise reduction is not specifically limited.

[0139] S402, Obtain a first noisy image to represent image details based on the original image.

[0140] In one embodiment, step S402 may include:

[0141] The difference image is obtained by subtracting the first image from the original image.

[0142] The channel data used to represent brightness values ​​in the difference image are obtained to obtain the first noise image.

[0143] The image subtraction process involves subtracting the pixel value of the second pixel from the pixel value of the first pixel. The first pixel is a pixel in the original image, and the second pixel is a pixel in the first image that corresponds to the first pixel.

[0144] In this embodiment, the first image and the original image have the same image size. For example, if the first image is 480×640, then the original image is also 480×640. In other words, each pixel in the first image has a corresponding pixel in the original image. For example, if the pixel value of the pixel in the first row and first column of the first image is 96, and the pixel value of the pixel in the first row and first column of the original image is 5, then the pixel value of the pixel in the first row and first column of the difference image is 96-5=91.

[0145] In some implementations, the first image and the first noise image are RGB images, and correspondingly, the difference image is also an RGB image. The difference image can be converted into a YUV image, and the Y channel data can be obtained from the YUV image to obtain the first noise image.

[0146] YUV is a color encoding method. In a YUV image, the Y channel represents the brightness of the image, while the U and V channels represent the color.

[0147] In this embodiment, the first noisy image includes only the channel data of the brightness value and does not include the data of the color channel. Accordingly, after subsequent superposition processing, the image details are increased without changing the color of the image, which helps to improve the image processing effect.

[0148] As described in the above embodiment, the original image is a RAW format image. RAW format images are single-channel intensity images, which are not conducive to subsequent image processing. To address this issue, in one embodiment, image subtraction processing may include:

[0149] The first image is de-mosaiced to obtain the processed first image;

[0150] The original image is de-mosaiced to obtain the processed original image;

[0151] The difference image is obtained by subtracting the processed first image from the processed original image.

[0152] In this embodiment of the application, the de-mosaic processing can perform color calibration on the Bayer array image to correct deviations that may exist due to color distortion or inaccurate camera colors, and reconstruct a full-color image, that is, form an RGB image.

[0153] It should be noted that the first image and the original image mentioned in the subsequent embodiments of this application may refer to the first image and the original image after de-mosaic processing, that is, the first image and the original image in the subsequent embodiments are both RGB images.

[0154] S403, the first image and the first noisy image are superimposed to obtain the processed second image.

[0155] Figure 4 In the aforementioned embodiment, noise information representing image details is obtained from the original image and superimposed onto the denoised image. This compensates for the lack of detail in the denoised image, effectively improving the image processing performance. Furthermore, since the noisy image is the original image's own noise signal, it is closer to the original hardware noise signal, making the processed image more closely resemble the user's subjective experience.

[0156] In one implementation, the overlay process may include adding the pixel value of a pixel in the first image to the pixel value of the corresponding pixel in the first noisy image. For example, if the pixel value of the pixel in the first row and first column of the first image is 96, and the pixel value of the pixel in the first row and first column of the first noisy image is 2, then the pixel value of the pixel in the first row and first column of the second image is 96 + 2 = 98.

[0157] The above overlay processing method is relatively simple to calculate, but the image after overlay processing is prone to noise protrusion, that is, the noise in the overlay area is more obvious, resulting in an unnatural display effect of the image.

[0158] Based on this, the embodiments of this application provide a superposition processing method.

[0159] See Figure 5 This is a flowchart illustrating the overlay processing method provided in an embodiment of this application. It is intended as an example and not a limitation. Figure 5 As shown, the overlay processing method in step S403 may include:

[0160] S501, Obtain the second noise image and the third noise image based on the first noise image.

[0161] In some implementations, different convolution kernels can be used to perform Gaussian filtering on the first noisy image to obtain the second and third noisy images.

[0162] The image frequency of the second noise image is higher than that of the third noise image.

[0163] It is understood that the second noise image corresponds to high-frequency noise, and the third noise image corresponds to low-frequency noise. High-frequency noise typically contains fewer image details, while low-frequency noise typically contains more image details. In this embodiment, by acquiring noise images of different frequencies, appropriate noise images can be selected for superposition during subsequent overlay processes, which helps to improve the flexibility and effectiveness of image processing.

[0164] In some implementations, the mean image of the second and third noisy images can be superimposed on the first image to obtain the processed second image. Specifically, the mean image is calculated by averaging the pixel value of each pixel in the second noisy image with the corresponding pixel value in the third noisy image.

[0165] This implementation method is equivalent to calculating the intermediate frequency noise based on the high-frequency and low-frequency noise, and then superimposing the intermediate frequency noise onto the denoised first image. Because the first image is superimposed with intermediate frequency noise, details in different regions of the superimposed image cannot be effectively distinguished. To solve this problem, this application embodiment uses the method described in S502.

[0166] S502, based on the brightness values ​​of the pixels in the first image, perform noise fusion processing on the second and third noise images to obtain the fourth noise image.

[0167] In this embodiment, the fusion weights of the second and third noise images can be determined based on the brightness values ​​of the pixels in the first image. Then, noise fusion processing is performed on the second and third noise images based on the fusion weights to obtain the fourth noise image.

[0168] In one implementation, if the brightness value of the third pixel is less than a preset threshold, the brightness value of the fourth pixel is determined as the first brightness value; if the brightness value of the third pixel is greater than the preset threshold, the brightness value of the fourth pixel is determined as the second brightness value.

[0169] Wherein, the third pixel is a pixel in the first image, the fourth pixel is a pixel in the fourth noisy image corresponding to the third pixel, the first brightness value is the brightness value in the third noisy image corresponding to the third pixel, and the second brightness value is the brightness value in the second noisy image corresponding to the third pixel.

[0170] In another implementation, if the brightness value of the third pixel is less than the first threshold, the brightness value of the fourth pixel is determined based on the first weight, the first brightness value, and the second brightness value; if the brightness value of the third pixel is greater than the second threshold, the brightness value of the fourth pixel is determined based on the second weight, the first brightness value, and the second brightness value; if the first brightness value of the third pixel is greater than the first threshold and less than the second threshold, a third weight is determined based on the first weight, the second weight, and the first brightness value, and the brightness value of the fourth pixel is determined based on the first brightness value, the second brightness value, and the third weight.

[0171] For example, see Figure 6 This is a schematic diagram of the fusion weights provided in an embodiment of this application. For example... Figure 6As shown in (a) of the diagram, the weights of the third noise image are used in the first implementation. It can be seen that when the brightness value of the third pixel is less than a preset threshold, the weight of the third noise image is 1, and correspondingly, the weight of the second noise image is 0, meaning only the brightness value (first brightness value) of the pixel corresponding to the third pixel in the third noise image is used. When the brightness value of the third pixel is greater than the preset threshold, the weight of the third noise image is 0, and correspondingly, the weight of the second noise image is 1, meaning only the brightness value (second brightness value) of the pixel corresponding to the third pixel in the second noise image is used.

[0172] like Figure 6 As shown in (b) of the diagram, the weights of the third noise image are represented in the second implementation. When the brightness value of the third pixel is less than the first threshold, the brightness value of the fourth pixel can be calculated using the formula noise = α1 × noise1 + noise2(1-α1), where noise represents the brightness value of the fourth pixel, noise1 represents the first brightness value, noise2 represents the second brightness value, and α1 represents the first weight. When the brightness value of the third pixel is greater than the second threshold, the brightness value of the fourth pixel can be calculated using the formula noise = α2 × noise1 + noise2(1-α2), where α2 represents the second weight. When the brightness value of the third pixel is greater than the first threshold and less than the second threshold, the brightness value of the fourth pixel can be calculated using the formula noise = α3 × noise1 + noise2(1-α3), where α3 represents the third weight.

[0173] The third weight can be calculated using the formula: Where t3 represents the brightness value of the third pixel, t1 represents the first threshold, and t2 represents the second threshold.

[0174] contrast Figure 6 The two weight calculation methods shown have different approaches. The first method essentially sets the weights of the second and third noisy images to 1 and 0, or 0 and 1. In this method, pixels near a preset threshold in the first image are prone to abrupt changes after noise is added, affecting the image display. The second method sets two thresholds, with the area between them representing a transition region in image brightness. Within this transition region, the weights of the second and third noisy images are smoothly adjusted, resulting in smoother details in the superimposed fourth noisy image. This effectively reduces abrupt changes in detail after noise addition, thus improving the image processing effect.

[0175] S503, the fourth noise image and the first image are superimposed to obtain the processed second image.

[0176] Figure 5In this embodiment, the weights of noise images of different frequencies are set according to the brightness information of the denoised first image, so as to achieve the fusion of noise images of different frequencies in different brightness regions of the image. In this way, the problem of noise images in the superimposed image can be effectively reduced, which is beneficial to improving the display effect of the processed image.

[0177] In some embodiments, step S503 may include:

[0178] I. Obtain the overlay weights based on the edge information in the first image;

[0179] II. The fourth noise image and the first image are superimposed according to the superposition weight to obtain the processed second image.

[0180] In this embodiment, image edges are considered during the noise superposition process. Since the image details differ between image edges and non-image edges, this method allows for the adaptive superposition of noise of varying intensities in different regions of the image. This reduces abrupt changes in image edge details after noise superposition, resulting in a more natural-looking image and improving image processing performance.

[0181] In some implementations, the steps for obtaining the superposition weights in I may include:

[0182] Edge information in the first image is detected to obtain the edge image of the first image; the superposition weight is obtained based on the edge image.

[0183] In some examples, an image detection model can be used to detect edge information in the first image. This image detection model can be a neural network model or other algorithmic model capable of image edge detection.

[0184] In some implementations, the process of acquiring the edge image in I may include:

[0185] The channel data representing brightness values ​​in the first image are downsampled to obtain a sampled image;

[0186] The sampled image is subjected to a first blurring process to obtain a first blurred image;

[0187] The first blurred image is subjected to edge detection processing to obtain the edge image.

[0188] In some examples, the first image can be converted from RGB format to YUV format. Then, the Y channel data in the YUV format image is downsampled to obtain a sampled image.

[0189] In this embodiment, downsampling is used to improve the efficiency of subsequent data processing.

[0190] In some examples, the first blurring process can be Gaussian blurring, which is to apply Gaussian filtering to the sampled image.

[0191] In this embodiment of the application, the first blurring process can smooth the uneven image edges in the sampled image, which can effectively reduce the abruptness of image edge details in the superimposed image and improve the image processing effect.

[0192] In some implementations, the process of acquiring the edge image in I may include:

[0193] The edge image is subjected to a second blurring process to obtain the blurred edge image;

[0194] The superposition weight is obtained based on the blurred edge image.

[0195] In some examples, the second blurring process can be Gaussian blurring, which is Gaussian filtering applied to the edge image.

[0196] In this embodiment of the application, the second blurring process can smooth the uneven parts of the edge image, which can effectively reduce the abruptness of image edge details in the superimposed image and improve the image processing effect.

[0197] For example, see Figure 7 This is a schematic diagram of an edge image provided in an embodiment of this application. For example... Figure 7 Image (a) shows the edge image. Figure 7 As shown in (b) above, this is the edge image after the second blurring process. A comparison reveals that the edge image after the second blurring process is smoother.

[0198] In this embodiment of the application, the superposition weight includes the weight corresponding to each pixel in the fourth noisy image.

[0199] In some implementations, the steps for obtaining the superposition weights in I may include:

[0200] If the pixel value of the fifth pixel is the first value, then the weight of the sixth pixel is set to the fourth weight.

[0201] If the pixel value of the fifth pixel is the second value, then the weight of the sixth pixel is set to the fifth weight.

[0202] If the pixel value of the fifth pixel is greater than the first value and less than the second value, then the sixth weight is calculated based on the fourth weight, the fifth weight, and the pixel value of the fifth pixel, and the weight of the sixth pixel is set as the sixth weight.

[0203] Wherein, the fifth pixel is a pixel in the blurred edge image, the sixth pixel is a pixel in the fourth noise image corresponding to the fifth pixel, the first value is the minimum brightness value in the blurred edge image, the second value is the maximum brightness value in the blurred edge image, and the fourth weight is less than the fifth weight.

[0204] For example, see Figure 8 This is a schematic diagram of the superimposed weights provided in an embodiment of this application. For example... Figure 8 As shown, the first value is 0, and the second value is 255, meaning the brightness value in the edge image ranges from 0 to 255. The sixth weight can be calculated using the formula... Where α6 represents the sixth weight, t5 represents the brightness value of the fifth pixel, α4 represents the fourth weight, and α5 represents the fifth weight.

[0205] In this embodiment, it is equivalent to calculating the superposition weight of different texture regions based on the edge image, so that the edge details in the superimposed image are smoother, which can effectively reduce the problem of abrupt changes in details after superposition noise, thereby improving the image processing effect.

[0206] In some implementations, the overlay process in step II may include:

[0207] The brightness value of each pixel in the fourth noisy image is multiplied by the corresponding weight in the superposition weight to obtain the processed fifth noisy image; the fifth noisy image is superimposed on the first image to obtain the processed second image.

[0208] The method of superimposing the fifth noise image with the first image is the same as the superimposition method described in step S403, and can be found in the description of the embodiment in step S403.

[0209] To better understand the image processing execution process described above, the following will... Figure 9 A schematic diagram illustrating the image processing flow provided in the embodiments of this application.

[0210] S901, acquire the original image.

[0211] The process of acquiring the original image can be found above. Figure 3 The description of steps 1-6 in the embodiments.

[0212] S902, the original image is denoised to obtain the first image.

[0213] Step S902 is the same as step S401 above, and can be found in the description in the embodiment of S401.

[0214] S903, perform de-mosaic processing on the first image to obtain the processed first image.

[0215] S904 performs de-mosaic processing on the original image to obtain the processed original image.

[0216] S905, perform image subtraction between the processed first image and the processed original image to obtain the difference image.

[0217] S906, acquire the channel data used to represent the brightness value in the difference image to obtain the first noise image.

[0218] Steps S903-S906 are the same as the process described in step S402 above, and can be found in the description in the embodiment of S402.

[0219] S907, Obtain the second noise image and the third noise image based on the first noise image.

[0220] Step S907 is the same as step S501 above, and can be found in the description in the embodiment of S501.

[0221] S908 performs conversion processing on the first image to obtain an image in YUV format.

[0222] S909 obtains the fusion weight between the second and third noisy images based on the brightness values ​​of the YUV format images.

[0223] The implementation method of step S909 can be found above. Figure 6 Description in the embodiments.

[0224] S910 detects edge information in YUV format images and obtains edge images.

[0225] S911, perform a second blurring process on the edge image to obtain a blurred edge image.

[0226] S912, obtain the superposition weight based on the blurred edge image.

[0227] The implementation of steps S910-S912 can be found in the description of the embodiment of step I above.

[0228] S913, perform noise fusion processing on the second and third noise images according to the fusion weight and the superposition weight to obtain the fourth noise image.

[0229] S914, the fourth noise image and the first image are superimposed to obtain the processed second image.

[0230] In some implementations, the second and third noise images can be fused according to the fusion weights to obtain the fourth noise image; then the fourth noise image and the first image can be superimposed according to the superposition weights.

[0231] The noise fusion processing method can be found in the description of step S502 in the embodiment.

[0232] In this embodiment, noise information representing image details is obtained from the original image and superimposed onto the denoised image, compensating for the lack of detail in the denoised image and effectively improving the image processing effect. Furthermore, during the noise superposition process, information such as brightness and image edges is comprehensively considered, and signals of different intensities are superimposed on different regions of the image to obtain a more natural image effect.

[0233] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0234] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0235] This application also provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0236] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0237] This application also provides a chip system, which includes a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the steps of any method embodiment of this application. The chip system can be a single chip or a chip module composed of multiple chips.

[0238] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0239] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. Finally, it should be noted that the above descriptions are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, the method includes: The electronic device displays a first interface, which is a shooting interface for a camera application. The first interface includes a first control, which is used to trigger shooting. In response to the user's first operation on the first control, the original image captured by the image sensor of the camera in the electronic device is obtained; The original image is denoised to obtain a denoised first image; Obtain the brightness channel data of the difference image between the first image and the original image to obtain a first noise image for representing image details; Based on the first noise image, a second noise image and a third noise image are obtained at two frequency components, wherein the image frequency of the second noise image is higher than the image frequency of the third noise image; The second noise image and the third noise image are fused according to the weight of each pixel to obtain a fourth noise image, wherein the weight of each pixel is determined according to the brightness value of the pixel in the first image; The first image and the fourth noisy image are superimposed to obtain the processed second image; The second interface is displayed on the screen, and the second interface includes the second image.

2. The method according to claim 1, characterized in that, The step of obtaining the Y channel data of the difference image between the first image and the original image to obtain a first noise image for representing image details includes: The difference image is obtained by subtracting the first image from the original image. The first noise image is obtained by acquiring the channel data used to represent the brightness value in the difference image. The image subtraction process involves subtracting the pixel value of a second pixel from the pixel value of a first pixel. The first pixel is a pixel in the original image, and the second pixel is a pixel in the first image that corresponds to the first pixel.

3. The method according to claim 1, characterized in that, The step of fusing the second noise image and the third noise image according to the weight of each pixel to obtain the fourth noise image includes: If the brightness value of the third pixel is less than the first threshold, the brightness value of the fourth pixel is determined according to the first weight, the first brightness value, and the second brightness value. Wherein, the third pixel is a pixel in the first image, the fourth pixel is a pixel in the fourth noisy image corresponding to the third pixel, the first brightness value is the brightness value in the third noisy image corresponding to the third pixel, and the second brightness value is the brightness value in the second noisy image corresponding to the third pixel.

4. The method according to claim 3, characterized in that, The step of performing noise fusion processing on the second and third noise images based on the brightness values ​​of pixels in the first image to obtain a fourth noise image includes: If the brightness value of the third pixel is greater than the second threshold, then the brightness value of the fourth pixel is determined based on the second weight, the first brightness value, and the second brightness value.

5. The method according to claim 4, characterized in that, The step of performing noise fusion processing on the second and third noise images based on the brightness values ​​of pixels in the first image to obtain a fourth noise image includes: If the brightness value of the third pixel is greater than the first threshold and less than the second threshold, then the third weight is determined based on the first weight, the second weight, and the first brightness value. The brightness value of the fourth pixel is determined based on the first brightness value, the second brightness value, and the third weight.

6. The method according to any one of claims 1 to 5, characterized in that, The step of superimposing the fourth noise image and the first image to obtain the processed second image includes: The superposition weight is obtained based on the edge information in the first image; The fourth noise image and the first image are superimposed according to the superposition weight to obtain the processed second image.

7. The method according to claim 6, characterized in that, The step of obtaining the overlay weight based on the edge information in the first image includes: Detect edge information in the first image to obtain the edge image of the first image; The overlay weights are obtained based on the edge image.

8. The method according to claim 7, characterized in that, The step of detecting edge information in the first image to obtain an edge image of the first image includes: The channel data representing brightness values ​​in the first image are downsampled to obtain a sampled image; The sampled image is subjected to a first blurring process to obtain a first blurred image; The first blurred image is subjected to edge detection processing to obtain the edge image.

9. The method according to claim 8, characterized in that, The step of obtaining the overlay weights based on the edge image includes: The edge image is subjected to a second blurring process to obtain the blurred edge image; The superposition weight is obtained based on the blurred edge image.

10. The method according to claim 9, characterized in that, The superposition weight includes the weight corresponding to each pixel in the fourth noisy image; The step of obtaining the superposition weight based on the blurred edge image includes: If the pixel value of the fifth pixel is the first value, then the weight of the sixth pixel is set to the fourth weight. Wherein, the fifth pixel is a pixel in the blurred edge image, the sixth pixel is a pixel in the fourth noise image corresponding to the fifth pixel, and the first value is the minimum brightness value in the blurred edge image.

11. The method according to claim 10, characterized in that, The step of obtaining the superposition weight based on the blurred edge image includes: If the pixel value of the fifth pixel is the second value, then the weight of the sixth pixel is set to the fifth weight. Wherein, the second value is the maximum brightness value in the blurred edge image, and the fourth weight is less than the fifth weight.

12. The method according to claim 11, characterized in that, The step of obtaining the superposition weight based on the blurred edge image includes: If the pixel value of the fifth pixel is greater than the first value and less than the second value, then the sixth weight is calculated based on the fourth weight, the fifth weight, and the pixel value of the fifth pixel. Set the weight of the sixth pixel to the sixth weight value.

13. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 12.

Images