Image processing method and device

Abstract

The embodiment of the invention provides an image processing method and device, and relates to the technical field of terminals. The method comprises: in response to an operation for shooting an image, obtaining a plurality of frames of images, the plurality of frames of images comprising at least one frame of long frame image, at least one frame of short frame image and at least one frame of middle frame image; performing image fusion on part or all of the multiple frames of images to obtain a first image; performing color correction on the target area in the first image by using the at least one long-frame image to obtain a target image; wherein the target area is an area with color noise in the first image. Therefore, the signal-to-noise ratio of the target area in the first image can be improved, the color distribution of the target area in the first image is more uniform and fine, the color noise points of the target area in the first image are reduced, and the image quality of the target image is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method and device. Background Art

[0002] At present, some electronic devices can improve the image quality of images taken by electronic devices by multi-frame noise reduction processing. For example, when a user uses a camera of an electronic device with a high dynamic range (HDR) function to take an image, the camera of the electronic device can take pictures at different exposures to obtain multiple frames of images; then, the electronic device can perform image fusion and noise reduction processing on the multiple frames of images to obtain a target image.

[0003] However, in scenes such as low light or high dynamic range, the image quality of the target image obtained by the electronic device may still be low. For example, the target image taken by the electronic device in a low light scene has more color noise. Summary of the invention

[0004] The present application provides an image processing method and device, which are applied to the field of terminal technology, and can reduce color noise in a target image in low light or high dynamic range scenes, improve the image quality of the target image, and enhance the user shooting experience.

[0005] In a first aspect, an embodiment of the present application proposes an image processing method. The method includes: in response to an operation for capturing an image, acquiring multiple frames of images, the multiple frames of images including at least one long frame image, at least one short frame image, and at least one medium frame image; performing image fusion on part or all of the multiple frames of images to obtain a first image; using at least one long frame image, performing color correction on a target area in the first image to obtain a target image; wherein the target area is an area in the first image where color noise exists.

[0006] The image processing method of the present application, because the target area in the long-frame image acquires more photons, the signal-to-noise ratio of the target area in the long-frame image is higher, and the color distribution is more uniform and delicate. The electronic device uses the long-frame image to perform color correction on the target area in the first image, which helps to improve the signal-to-noise ratio of the target area in the first image, making the color distribution of the target area in the first image more uniform and delicate, reducing the color noise in the target area in the first image, thereby improving the image quality of the target image.

[0007] Optionally, performing image fusion on part or all of the multiple frames of images to obtain the first image includes: performing image fusion, noise reduction processing, and demosaicing processing on part or all of the multiple frames of images to obtain the first image. In this way, the noise of the first image is small, which facilitates improving the correction effect on the target area in the first image.

[0008] The image fusion, noise reduction and demosaicing of part or all of the multiple frames of images may be implemented, for example, in the following manner: part or all of the multiple frames of images are input into a first Net model to obtain a first image. The first Net model may refer to the description below.

[0009] In combination with the first aspect, in some possible implementations, the method further includes: performing image segmentation on the second image in the multiple frames of images to obtain a mask image corresponding to the target area in the second image; and performing color correction on the target area in the first image using at least one long frame image, including: performing color correction on the target area in the first image using at least one long frame image and the mask image. In this way, the electronic device can identify the target area through the mask image, thereby facilitating color correction of the target area in the first image.

[0010] In combination with the first aspect, in some possible implementations, color correction is performed on a target area in a first image using at least one long frame image and a mask image, including: obtaining a correction coefficient of the target area in the first image using at least one long frame image, the mask image, and the first image; and performing color correction on the target area in the first image using the correction coefficient.

[0011] Among them, the correction coefficient may include a correction coefficient for each pixel in the target area in the first image. The correction coefficient may be a coefficient corresponding to the format of the first image. For example, if the first image is an image in red green blue (RGB) format, the correction coefficient is a coefficient of at least one of the R channel, G channel or B channel used to correct the target area in the first image; if the first image is an image in YUV format, the correction coefficient is a coefficient of the U channel and / or V channel used to correct the target area in the first image. In this way, the electronic device can perform pixel-by-pixel color correction on the target area in the first image, thereby reducing color noise in the target area.

[0012] In combination with the first aspect, in some possible implementations, the correction coefficient includes a first coefficient and a second coefficient, the first coefficient is used to correct the chromaticity value of the target area in the first image, and the second coefficient is used to correct the concentration value of the target area in the first image; using the correction coefficient to perform color correction on the target area in the first image includes: correcting the chromaticity value of the target area in the first image using the first coefficient to obtain a corrected chromaticity value, and the corrected chromaticity value U 1 Satisfies the following formula: 1 =U 0 +C U ×U 0 , where U 0is the chromaticity value of the target area before correction, C U is the first coefficient; the density value of the target area in the first image is corrected using the second coefficient to obtain a corrected density value, the corrected density value V 1 Satisfies the following formula: V 1 =V 0 +C V ×V 0 , where V 0 is the concentration value of the target area before correction, C V is the second coefficient.

[0013] The first coefficient may be, for example, the correction coefficient of the U channel in the following text; the second coefficient may be, for example, the correction coefficient of the V channel in the following text; and the target area may be, for example, the sky area in the following text. Since the chromaticity value (U in YUV) and the concentration value (V in YUV) can describe the image color and saturation, and the color of the specified pixel. Therefore, by correcting the chromaticity value and concentration value of the target area in the first image, the image color and saturation of the target area in the first image can be corrected, so that the color distribution of the target area in the first image is more uniform and delicate, thereby reducing the color noise of the target area in the first image.

[0014] In combination with the first aspect, in some possible implementations, obtaining a correction coefficient of a target area using at least one long-frame image, a mask image, and a first image includes: converting at least one long-frame image into a third image, the third image having the same format as the first image; obtaining the correction coefficient of the target area using the third image, the mask image, and the first image. In this way, since the third image and the first image have the same format, it is easier for the electronic device to compare the third image and the first image pixel by pixel to determine the correction coefficient of each channel of the target area in the first image. For example, if both the third image and the first image are images in RGB format, the electronic device can compare the R channel, G channel, and B channel of pixel A in the third image and pixel B in the first image to obtain the correction coefficient corresponding to pixel B. Among them, pixel A corresponds to pixel B in position.

[0015] In combination with the first aspect, in some possible implementations, the second image belongs to a frame of at least one frame-in-frame image.

[0016] In this way, compared with the short frame image, the signal-to-noise ratio of the medium frame image is higher, that is, the edge of the target area in the medium frame image may be clearer, so that the accuracy of the target area determined by the medium frame image is higher. In addition, after the frame selection module transmits part or all of the selected medium frame image to the image signal processor, the image signal processor can determine the mask image. This allows the image signal processor to determine the mask image before acquiring the image sequence, which helps to improve the efficiency of acquiring the mask image.

[0017] In combination with the first aspect, in some possible implementations, performing image segmentation on the second image in the multiple frames of images includes: performing image segmentation on the second image when it is determined that the ambient brightness is less than or equal to a preset brightness.

[0018] When the ambient brightness is high, that is, when the current shooting scene is not a dark light scene, the number of photons obtained by the medium frame image and the short frame image is also high, so the color noise in the target area of ​​the target image may be small or non-existent. Therefore, the electronic device can obtain the target image by using the long frame image to perform color correction on the target area of ​​the first image in a dark light scene; in a non-dark light scene, the electronic device may not obtain the mask image, nor does it need to use the long frame image to perform color correction on the target area of ​​the first image, that is, the first image or the image obtained after the format conversion of the first image is the target image, so that the power consumption of the electronic device is relatively low.

[0019] In combination with the first aspect, in some possible implementations, the method further includes: converting the first image into a brightness, chroma, and density YUV format to obtain a fourth image; and performing color correction on the target area in the first image, including: performing color correction on the target area in the fourth image. In this way, the electronic device can correct the image color and saturation of the target area in the fourth image by correcting the U channel (chroma value) and the V channel (density value) of the target area, so that the color distribution of the target area is more uniform and delicate, and the color noise of the target area is reduced.

[0020] In combination with the first aspect, in some possible implementations, color correction is performed on the target area in the first image using at least one long frame image and a mask image, including: when the proportion of the target area in the mask image is greater than or equal to a preset threshold, color correction is performed on the target area in the first image using at least one long frame image and the mask image.

[0021] The preset threshold is a preset positive value, such as 30%. In this way, when the target area in the mask image occupies a large proportion, the target area may have more color noise, and at least one long frame image can be used to perform color correction on the target area in the first image to reduce the color noise in the target area; when the target area in the mask image occupies a small proportion or there is no target area, the electronic device may not use at least one long frame image to perform color correction on the target area in the first image, so that the power consumption of the electronic device is relatively small.

[0022] In combination with the first aspect, in some possible implementations, performing image fusion on part or all of multiple frame images to obtain a first image includes: performing image fusion on at least one short frame image and at least one medium frame image to obtain the first image.

[0023] Since long-frame images are prone to jitter and blur, not fusing the long-frame images can reduce the impact of the jitter and blur of the long-frame images on the target image. In addition, since the effect of noise reduction processing on images with jitter and blur may be poor, when subsequent electronic devices perform noise reduction processing on the fused image, the noise reduction effect is better, which helps to further improve the image quality of the target image.

[0024] In addition, long-frame images are not fused, so that electronic devices do not need to perform pixel-by-pixel registration of long-frame images and short-frame images, which is beneficial to reducing power consumption of electronic devices.

[0025] In a second aspect, an embodiment of the present application provides an image processing device, which may be an electronic device, or a chip or chip system in an electronic device. The image processing device may include a display unit and a processing unit. When the image processing device is an electronic device, the display unit may be a display screen. The display unit is used to perform the display step so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect.

[0026] When the image processing device is an electronic device, the processing unit may be a processor. The image processing device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. When the image processing device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit in the chip (e.g., a register, a cache, etc.), or a storage unit in the electronic device located outside the chip (e.g., a read-only memory, a random access memory, etc.).

[0027] Exemplarily, the processing unit is used to perform image fusion on part or all of the multiple frames of images to obtain a first image; the display unit is used to display the target image, etc.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0030] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0031] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0032] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (e.g., a read-only memory, a random access memory, etc.).

[0033] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of a process of an image processing method;

[0036] Figure 3 A schematic block diagram of the hardware architecture of an electronic device provided in an embodiment of the present application;

[0037] Figure 4 A schematic block diagram of the software architecture of the electronic device provided in the embodiment of the present application;

[0038] Figure 5 A flowchart of an image processing method provided in an embodiment of the present application;

[0039] Figure 6 A schematic diagram of a mask image acquisition process provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of a reference frame image provided in an embodiment of the present application;

[0041] Figure 8 A schematic diagram of a long frame image provided in an embodiment of the present application;

[0042] Fig. 9 A flowchart of another image processing method provided in an embodiment of the present application;

[0043] Fig.10 A schematic diagram of a target display image 1 provided in an embodiment of the present application;

[0044] Fig.11 A schematic diagram of a target display image 2 provided in an embodiment of the present application;

[0045] Fig.12 A flowchart of another image processing method provided in an embodiment of the present application;

[0046] Fig.13 A schematic block diagram of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0048] 1. Stagger high dynamic range (SHDR), also known as stagger, is an image framing method that uses "rows" as the output unit. SHDR can achieve the simultaneous acquisition of multiple frames of images with different exposure times in one shot, such as acquiring long frames, short frames, and normal frames in one shot. The SHDR framing method can reduce the time interval between frames, thereby reducing the appearance of ghost images.

[0049] 2. Normal frame, also known as normal exposure image, N (normal) frame image, N frame, medium frame or medium frame image, is an image captured by the camera when the exposure is 0EV. In other words, the exposure of a normal exposure image is 0EV. The 0EV here is a relative value, not that the exposure is 0. Exemplarily, exposure = exposure time * sensitivity (ISO). Assuming that the normal exposure image is captured when the ISO is 200 and the exposure time is 50 milliseconds, the actual exposure corresponding to 0EV is the product of 200 and 50 milliseconds.

[0050] 3. Short frame, also called short frame image, S (short) frame or S frame image, is an image captured by the camera when the exposure is less than 0EV.

[0051] 4. Long frame, also known as long frame image, L (long) frame or L frame image, is an image captured by the camera when the exposure is greater than 0EV.

[0052] 5. YUV is a color encoding method, where Y (Luminance, Luma) represents brightness, and U and V represent chrominance and concentration (Chrominance, Chroma).

[0053] 6. A linear RGB image can be understood as an image with a linear relationship between the brightness in the physical space and the image display brightness. For example, a linear RGB image can be an RGB image that has not been gamma corrected.

[0054] 7. Other terms

[0055] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0056] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0057] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0058] 8. Electronic devices

[0059] The electronic device of the embodiment of the present application may include a handheld device with a camera function, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (public landmobile networks). The terminal equipment in the Network, PLMN, etc. is not limited to this in the embodiments of the present application.

[0060] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0061] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0062] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0063] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0064] Figure 1 A schematic diagram of a shooting scene provided in an embodiment of the present application. When the electronic device receives an input operation of turning on the camera by the user, the electronic device may display the following Figure 1 The shooting interface shown in (a) is shown in the figure. Figure 1 As shown in the interface (a) in FIG. 1 , the shooting interface displayed by the electronic device may include a shooting button 101 and a preview screen 102. The preview screen 102 may display the screen to be shot by the user; when the user clicks the shooting button 101, the electronic device may capture the image displayed in the preview screen 102, and then the electronic device may display the following image: Figure 1 The image interface shown in (b) is shown in the figure. Figure 1 As shown in the interface (b) in the figure, the interface may be an interface in the album of the electronic device. The interface (b) may include a target display image 103 captured by the electronic device and multiple buttons such as share, favorite, edit, delete, and more.

[0065] However, in low-light scenes or high dynamic range scenes, when the exposure time of the electronic device is short, the signal-to-noise ratio of the target display image captured may be low. Among them, the high dynamic range scene, also known as the HDR scene, can be understood as a scene in which the difference between the maximum brightness and the minimum brightness in the image captured by the electronic device exceeds the preset difference.

[0066] If the electronic device performs noise reduction processing on the captured image before displaying the target display image, and then displays the target display image after the noise reduction processing, it may cause loss of details in the target display image; if the electronic device performs gamma brightening or gamma correction processing on the captured image before displaying the target display image, and then displays the target display image after the gamma correction, it may amplify the noise in the dark area of ​​the target display image, making the noise in the target display image more obvious.

[0067] Based on this, currently electronic devices can use multi-frame noise reduction processing to obtain the target display image to improve the image quality of the target display image. Among them, multi-frame noise reduction processing can be understood as the process of electronic devices performing image fusion, noise reduction and other processing on multiple frames of images with different exposure amounts.

[0068] For example, Figure 1 As shown in the interface (a) in the figure, when the electronic device receives the user's trigger operation on the shooting button 101, the electronic device can frame based on HDR or SHDR and obtain an image sequence. That is, the electronic device can capture N frames of images at different exposure amounts and / or exposure times, and the N frames of images can include a frame of long frame image, b frame of short frame image and c frame of medium frame image, where N, a, b and c are positive integers. Further, the electronic device can perform fusion processing and noise reduction processing on the N frames of images to obtain a target display image 103. That is, the electronic device can display as shown below. Figure 1 The interface shown in (b).

[0069] It should be noted that Figure 1 The interface (a) is an example of a shooting interface displayed on an electronic device, and the interface (b) is an example of an interface for an electronic device to display a captured target display image 103. For example, the interface (a) and the interface (b) may further include more or fewer controls or buttons. Figure 1 Interface (a) and interface (b) do not constitute a limitation on the embodiments of the present application.

[0070] Next, combine Figure 2 Taking the above-mentioned N frames of images including two medium frame images, two short frame images and one long frame image as an example, the process of the electronic device performing fusion processing on the N frames of images to obtain the target display image 1 by performing noise reduction processing is explained.

[0071] Figure 2 FIG. 2 is a schematic diagram of a process of an image processing method 200. Figure 2 As shown, after receiving the shooting instruction, the electronic device can obtain N frames of images, including two medium frame images, two short frame images and one long frame image. The electronic device inputs the N frames of images into a neural network (Net) model 1, and obtains a linear (liner) red green blue (RGB) image 1 output by the Net model 1. Then, the electronic device can convert the linear RGB image 1 into an image 1 in YUV format, and filter the image 1 in YUV format to obtain a target display image 1.

[0072] It should be understood that Net model 1 can also be called denoising Net, etc., and can be a model with one or more functions of image fusion, denoising, interpolation and demosaicing. In addition, Net model 1 can be a convolutional neural network model, a one-dimensional convolutional neural network model, a deep learning convolutional neural network model or a two-dimensional convolutional neural network model. For example, the denoising Net model can be HDR Net, residual neural network (ResNet), U-Net, transformer Net or restormer Net, etc., and this application does not make specific limitations on this.

[0073] It should be understood that converting the linear RGB image 1 into the image 1 in the YUV format can be achieved through tools such as RGB2YUV, and the present application does not limit the specific method used for image format conversion.

[0074] It should also be understood that the filtering process may be a median filtering process or a mean filtering process, etc., which can reduce the noise of the YUV format image 1 and improve the signal-to-noise ratio of the YUV format image 1. The present application does not specifically limit the filtering process method.

[0075] Optionally, the filtering process can be implemented in the following manner: the electronic device performs filtering process on the UV channel of the YUV format image 1 based on the Y channel (brightness) in the YUV format image 1. For example, for an area in the YUV format image 1 where the brightness is greater than or equal to a first threshold, filtering process can be performed in one manner; for an area in the YUV format image 1 where the brightness is less than or equal to a second threshold, filtering process can be performed in another manner.

[0076] It should be noted that the process of obtaining the target display image 1 through N frames of images can also be called multi-frame noise reduction processing. Multi-frame noise reduction processing helps to make the image details of the target display image 1 displayed by the electronic device clearer and improve the image quality of the target display image 1.

[0077] For short-frame images taken by electronic devices, due to the shorter exposure time, the number of photons obtained in the dark areas of the short-frame images is relatively small, which makes the signal-to-noise ratio of the dark areas in the short-frame images lower, and makes it easy to generate color noise in the dark areas of the target display image.

[0078] In addition, for long-frame images, due to the long exposure time, the dark areas in the long-frame images can obtain more photons, which makes the signal-to-noise ratio of the long-frame images higher. However, due to the long exposure time, the long-frame images are prone to jitter blur. Jitter blur may make the noise reduction effect of the electronic noise reduction processing on the fused image worse, which may cause the color noise in the dark areas of the target display image to be more obvious.

[0079] It can be seen that after the electronic device performs multi-frame noise reduction processing on N frames of images, the dark area of ​​the target display image obtained may have low signal-to-noise ratio, color noise, etc. For example, the sky area of ​​the image captured by the electronic device in a dark light scene has more color noise, etc.

[0080] It should be noted that, in the embodiment of the present application, the dark area in the image is an area with relatively low brightness in the image, for example, it may be an area in the image with brightness lower than the third threshold. Alternatively, it may be a dark area predefined based on experience. For example, based on experience, the brightness of the sky area photographed in a dark light scene is usually low, and color noise is easily generated. Then, for an image captured by an electronic device in a dark light scene, the dark area of ​​the image may refer to the sky area.

[0081] In order to solve the above technical problems, the present application provides an image processing method. Since the signal-to-noise ratio of the dark area in the long-frame image is usually higher than that of the dark area in the short-frame image and the medium-frame image, the electronic device can use the long-frame image to perform color correction on the dark area in the fused image of part or all of the N-frame images to obtain the target display image. In this way, through color correction, the image color and saturation of the dark area in the fused image can be corrected, so that the color of the dark area in the target display image is more uniform and delicate, which helps to improve the signal-to-noise ratio of the dark area in the target display image and reduce the color noise in the dark area in the target display image.

[0082] Combine the following Figures 3 to 12 The image processing method of the present application is described in detail. The image processing method of the embodiment of the present application can be executed by an electronic device equipped with a camera, or by a chip, a chip system or a processor that supports the electronic device to implement the image processing method, or by a logic module or software that can implement all or part of the functions of the electronic device. The present application does not impose specific restrictions on this. The image processing method of the embodiment of the present application is described in detail below with the electronic device as the execution subject.

[0083] In order to better understand the electronic device in the embodiment of the present application, Figure 3 and Figure 4 The hardware structure and software structure of the electronic device in the embodiment of the present application are described in detail.

[0084] Figure 3 A schematic structural diagram of an electronic device 300 applicable to the present application is shown.

[0085] The electronic device 300 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0086] It should be noted that Figure 3 The structure shown does not constitute a specific limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may include Figure 3 The electronic device 300 may include more or fewer components than those shown. Figure 3 A combination of some of the components shown, or the electronic device 300 may include Figure 3 Subcomponents of some of the components shown. For example, Figure 3 The illustrated proximity light sensor 180G may be optional. Figure 3 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Different processing units may be independent devices or integrated devices.

[0088] In a possible implementation, the processor 110 may start the camera 193; obtain N frames of images in the current photographing environment; and process the N frames of images to obtain a target display image.

[0089] In one possible implementation, the processor 110 may determine the camera 193 that needs to be called, the exposure amount, the exposure time, and other image output parameters according to different ambient illumination and / or ambient brightness, and instruct the camera 193 to capture an image.

[0090] The ambient light sensor 180L is used to sense the ambient light brightness and to automatically adjust the white balance when taking photos. Optionally, the processor 110 can obtain the ambient illumination and / or ambient brightness from the ambient light sensor 180L.

[0091] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0092] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0093] The electronic device 300 can realize the shooting function through ISP, camera 193, GPU, display screen 194 and application processor.

[0094] The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. In some embodiments, the ISP can be set in the camera 193. Exemplarily, the electrical signal transmitted to the ISP by the camera photosensitive element can be an image sequence, and the ISP can pre-process the image sequence. The ISP can also output the pre-processed image sequence to a digital signal processor (DSP) or GPU for further processing, such as image fusion, noise reduction processing, and color correction, so as to obtain a target image.

[0095] In some embodiments, the electronic device 300 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

[0096] The digital signal processor is used to process digital signals, and in addition to processing digital image signals, it can also process other digital signals. For example, the digital signal processor can determine whether the proportion of the target area in the mask image is greater than or equal to a preset proportion.

[0097] The electronic device 300 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information. Exemplarily, the processor 110 may call the GPU to perform image fusion and noise reduction processing on at least one short frame image and at least one medium frame image; the processor 110 may also call the GPU to perform color correction on a target area, such as a sky area.

[0098] The display screen 194 may be used to display target images or thumbnails, etc.

[0099] It should be understood that Figure 3 The connection relationship between the modules shown is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 300. Optionally, the modules of the electronic device 300 may also adopt a combination of multiple connection modes in the above embodiments.

[0100] Figure 4 It is a schematic diagram of the architecture (including software system and some hardware) of the embodiment of the present application. Figure 4As shown, the application architecture is divided into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the hardware abstraction layer HAL, the driver layer, and the hardware layer.

[0101] like Figure 4 As shown, the application layer may include application programs such as camera, gallery, and system UI (systemUI). The system UI is used to display the interface of the electronic device, such as displaying the target display image captured by the electronic device.

[0102] like Figure 4 As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction module and a camera algorithm library. Among them, the camera hardware abstraction module includes multiple camera devices. The camera algorithm library includes a post-processing algorithm module, a decision module, and a frame selection module.

[0103] It should be understood that the decision module can also be placed in other layers. As a possible implementation, the decision module can be placed in the application layer or the application framework layer.

[0104] The driver layer is used to drive hardware resources. The driver layer can include multiple driver modules. Figure 4 As shown, the driver layer includes camera device driver, digital signal processor driver and graphics processor driver, etc.

[0105] The hardware layer includes sensors, image signal processors, digital signal processors, and graphics processors. The sensors may include multiple sensors, TOF cameras, and multi-spectral sensors. The image signal processor may include an ISP module, etc.

[0106] For example, a user may click on a camera application. When the user clicks on the camera to take a photo, the photo taking instruction may be sent to the camera hardware abstract module through the camera access interface. The camera hardware abstract module calls the camera device driver and the camera algorithm library. The decision module in the camera algorithm library may instruct the frame selection module to select a frame; it may also determine the image output parameters according to parameters such as the ambient illumination. The image output parameters may include information including the camera information, exposure, exposure time, etc. used to indicate the electronic device to take an image. The image output parameters are sent to the camera hardware abstract module. The camera hardware abstract module sends the image output parameters configured by the decision module to the camera device driver. The camera device driver sends the image output parameters sent by the camera hardware abstract module to the hardware layer, for example, the image output mode of the sensor is sent to the sensor, and the parameter configuration of the ISP module is sent to the image signal processor. The sensor outputs the image based on the image output mode of the sensor, for example, which camera or sensor needs to be called. The image signal processor performs corresponding preprocessing based on the parameter configuration of the ISP module. The camera algorithm library is also used to send digital signals to the digital signal processor driver in the driver layer so that the digital signal processor driver calls the digital signal processor in the hardware layer for digital signal processing. The digital signal processor can return the processed digital signal to the camera algorithm library through the digital signal processor driver. The camera algorithm library is also used to send digital signals to the graphics signal processor driver in the driver layer so that the graphics signal processor driver calls the graphics processor in the hardware layer to process the digital signal. The graphics processor can return the processed graphic data to the camera algorithm library through the graphics processor driver. For example, the image processor can filter and correct the image.

[0107] In addition, the image output by the image signal processor can be sent to the camera device driver. The camera device driver can send the image output by the image signal processor to the camera hardware abstraction module. The camera hardware abstraction module can send the image to the post-processing algorithm module for further processing, or send the image to the camera access interface. The camera access interface can send the image returned by the camera hardware abstraction module to the camera.

[0108] Among them, the post-processing algorithm module may include a first Net model and a second Net model. The first Net model may be used to perform image fusion, noise reduction, and de-mosaicing on short-frame images and medium-frame images. The second Net model may be used to determine the correction coefficient of the U channel and the correction coefficient of the V channel. The camera algorithm library may also call the image processor driver to drive the image processor to use the correction coefficient of the U channel and the correction coefficient of the V channel to perform UV channel correction on the target area in the image.

[0109] It should be understood that the first Net model and the second Net model can be types such as convolutional neural network models, one-dimensional convolutional neural network models, deep learning convolutional neural network models, or two-dimensional convolutional neural network models. For example, the first Net model and the second Net model can be HDR Net, residual neural network (ResNet), U-Net, transformer Net, or stormer Net, etc. This application does not make specific limitations on this.

[0110] It should also be understood that the first Net model can also be called a denoising Net model, and the image output by the first Net model can be a linear RGB image, or an image in other formats; the image input to the second Net model can be a linear RGB image, or an image in YUV or other formats, and this application does not make specific limitations on this.

[0111] The above describes in detail the software system used in the embodiments of the present application.

[0112] Understandably, Figure 4 ] is an example of a software module included in an electronic device. Each layer of the software structure of the electronic device may also include more or fewer software modules. Exemplarily, the application layer of the electronic device may also include other application programs, such as information, alarm clock, weather, stopwatch, compass, timer, flashlight, calendar, Alipay and other application programs. This application does not specifically limit this.

[0113] Figure 5 The flowchart of an image processing method 500 provided in an embodiment of the present application is shown in FIG. The method 500 may be executed by an electronic device, and the software structure of the electronic device may be as follows: Figure 4 As shown, the hardware structure of the electronic device can be as follows Figure 3 As shown. Figure 5 As shown, the method 500 includes the following steps:

[0114] S10: The camera obtains a first instruction, where the first instruction is used to instruct to capture an image.

[0115] Optionally, the first instruction may be issued when the electronic device receives a down event of a user pressing a shooting button, or may be issued when the electronic device recognizes a photo taking instruction.

[0116] Compared to the electronic device sending the first instruction when recognizing the photo taking instruction, the electronic device sending the first instruction when receiving the down event of the user pressing the shooting button will be earlier, that is, the electronic device will be more efficient in obtaining the first instruction. The shooting button can be, for example, the shooting button 101 of the electronic device in scene 100.

[0117] S11. The camera calls a camera access interface to indicate a first instruction to a camera hardware abstraction module.

[0118] S12. The camera hardware abstract module instructs the decision module in the camera algorithm library to determine the image output parameters.

[0119] Optionally, the image output parameters include but are not limited to one or more of the following: exposure time and exposure amount of the sensor (sensor) output image, configuration parameters of the camera or ISP module that needs to be called, etc.

[0120] S13: The decision module determines that it is a dark light scene.

[0121] The decision module may determine whether the current shooting scene is a dark light scene according to parameters such as ambient illumination and / or ambient brightness, etc. A dark light scene may be, for example, a night scene, a movie theater scene, etc.

[0122] Exemplarily, when the decision module determines that the ambient brightness in the current shooting scene is lower than the preset brightness value, the current shooting scene is determined to be a dark light scene. That is, in response to the instruction to determine the image output parameters, the decision module can obtain parameters such as ambient illumination. The ambient illumination and / or ambient brightness can be obtained from the sensor through the camera hardware abstraction module.

[0123] In a possible implementation, when the decision module determines that the current shooting scene is not a dark light scene, for example, the ambient brightness in the current shooting scene is relatively high, the electronic device may acquire the target image according to the above method 200.

[0124] S14, the decision module indicates the image parameters to the camera hardware abstraction module.

[0125] In a possible implementation, the decision module may determine the image output parameters according to parameters such as ambient illumination and ambient brightness, that is, in response to an instruction to determine the image output parameters, the decision module may obtain parameters such as ambient illumination.

[0126] S15, the decision module instructs the frame selection module to select frames from the preview queue. In addition, the decision module may also instruct the frame selection module on the number of mid-frame images to be selected.

[0127] It should be understood that the preview queue can be understood as a queue for caching images displayed in the preview screen. The images stored in the preview queue are multiple mid-frame images. The multiple mid-frame images can be images captured by the electronic device when the user previews the screen.

[0128] S16. The frame selection module selects a frame from the preview queue.

[0129] Based on the instruction of the decision model, the frame selection module can select a corresponding number of mid-frame images from the preview queue to obtain selected mid-frame images.

[0130] It should be understood that the frame selection module can select a mid-frame image with higher image quality, such as an image with higher definition, from multiple mid-frame images, and this application does not make any specific limitation on this.

[0131] S17, the frame selection module transmits the selected mid-frame image to the post-processing algorithm module, so that the post-processing algorithm module can subsequently post-process the selected mid-frame image.

[0132] The selected middle frame image may be one frame or multiple frames, such as 2 frames, 4 frames, etc.

[0133] Optionally, the post-processing algorithm module can generate a thumbnail based on the selected mid-frame image, and call the camera access interface through the camera hardware abstraction module to transmit the thumbnail to the camera, so that the camera can call the display screen to display the thumbnail.

[0134] S18, the frame selection module transmits the image 1 in the selected middle frame image to the camera hardware abstraction module, and the image 1 may be part or all of the selected middle frame image. For example, the image 1 may be a frame image in the selected middle frame image.

[0135] S19, the camera hardware abstraction module transmits the image 1 to the image signal processor through the camera device driver.

[0136] S20 , the image signal processor performs image segmentation on the image 1 to obtain a mask image of the sky area in the image 1 .

[0137] It should be understood that the mask image may be a binary image. That is, the pixel value of the sky area in the mask image may be 1, and the pixel value of the non-sky area may be 0. For example, Figure 6 As shown, the frame image in the frame can be Figure 6 As shown in (a) in Figure 6 The sky area 601 in the image (a) is segmented to obtain Figure 6 In the mask image (b), the pixel value of the sky area 602 may be 1, and the pixel value of the area outside the sky area 602 may be 0.

[0138] When the number of image 1 is one frame, the image signal processor can directly obtain the mask image of the sky area in image 1. When the number of image 1 is multiple frames, the image signal processor can select a mid-frame image from image 1 for image segmentation to obtain the mask image of the sky area in the mid-frame image; or the image signal processor can perform image fusion on image 1 to obtain the mask image of the sky area in the fused image 1; or the image signal processor can obtain the mask image of the sky area in each mid-frame image in image 1 respectively.

[0139] In addition, after S14, the camera hardware abstraction module further executes S21, where the camera hardware abstraction module indicates the image parameters to the camera device driver.

[0140] It should be understood that S21 and S15 to S20 may be executed in parallel, or S21 may be executed in parallel or sequentially with S19, and this application does not make any specific limitation on this.

[0141] S22: The camera device driver drives the sensor to output an image and instructs the sensor on an image output method.

[0142] The image output method may include but is not limited to one or more of the following: the camera to be called, the exposure amount, the exposure time, the number of long frames to be taken, or the number of short frames to be taken.

[0143] In this way, the sensor can output images according to the output mode to obtain an image sequence. The image sequence can also be called a Bayer image. The image sequence can include short-frame images and long-frame images, and the number of short-frame images and long-frame images can be one frame or more frames.

[0144] S23. The sensor indicates the image sequence to the image signal processor.

[0145] S24, the camera device driver indicates configuration parameters to the image signal processor. The configuration parameters are part of the image output parameters. The configuration parameters can be understood as the parameters required by the ISP module for preprocessing the image sequence.

[0146] It should be understood that S24 can be executed in parallel with S22 and S23, and S24 can also be executed before or after S22 and S23, and this application does not make specific limitations on this.

[0147] S25. The image signal processor preprocesses the image sequence using the configuration parameters to obtain a preprocessed image sequence.

[0148] Optionally, the preprocessing may include, but is not limited to, one or more of the following: black level compensation, lens correction, bad pixel correction, or color interpolation. The configuration data includes parameters required for the preprocessing.

[0149] In a possible implementation, the sensor indicates to the image signal processor the image index of each long-frame image and / or each short-frame image in the image sequence. The image index is used to indicate whether the image is a short-frame image or a long-frame image. For example, when the image index is 0, it may indicate that the image is a short-frame image; and / or, when the image index is 1, it may indicate that the image is a long-frame image.

[0150] It should be understood that S15 to S20 and S21 to S25 may be executed in parallel or sequentially, and this application does not make any specific limitation on this.

[0151] In a possible implementation, S18 to S20 are optional. Exemplarily, the image signal processor may also perform image segmentation on the x-frame long-frame image in the preprocessed image sequence or the y-frame long-frame image in the image sequence to obtain a mask image of the sky area. x and y are integers greater than or equal to 1.

[0152] Compared with short-frame images, medium-frame images and long-frame images have higher brightness and higher signal-to-noise ratio. Therefore, the edge of the sky area in medium-frame images and long-frame images may be clearer, so that the electronic device has a higher accuracy in determining the sky area through long-frame images or medium-frame images.

[0153] Compared with short frame images and long frame images, medium frame images can be transmitted to the image signal processor before the image signal processor preprocesses the image sequence, so that the image signal processor can perform image segmentation on the medium frame images and obtain the mask image of the sky area with higher efficiency.

[0154] S26: The image signal processor drives the camera device to indicate the pre-processed image sequence and the mask image.

[0155] It should be understood that the image signal processor can transmit the preprocessed image sequence and the mask image to the camera device driver simultaneously or successively. For example, the image signal processor first transmits the preprocessed image sequence to the camera device driver; and then transmits the mask image to the camera device driver. This application does not make any specific limitations on this.

[0156] Optionally, the image signal processor further drives an image index indicating each long frame image and / or each short frame image in the pre-processed image sequence to the camera device.

[0157] S27. The camera device driver indicates the pre-processed image sequence and mask image to the camera hardware abstraction module.

[0158] It should be understood that the camera device driver can transmit the pre-processed image sequence and the mask image to the camera hardware abstraction module simultaneously or successively, and this application does not make specific limitations on this.

[0159] Optionally, the camera device driver further indicates to the camera hardware abstraction module an image index of each long frame image and / or each short frame image in the pre-processed image sequence.

[0160] S28. The camera hardware abstraction module indicates the pre-processed image sequence and mask image to the post-processing algorithm module.

[0161] It should be understood that the camera hardware abstraction module can transmit the pre-processed image sequence and the mask image to the post-processing algorithm module simultaneously or successively, and this application does not make specific limitations on this.

[0162] Optionally, the camera hardware abstraction module further indicates the image index of each long frame image and / or each short frame image in the pre-processed image sequence to the post-processing algorithm module. In this way, it is convenient for the post-processing algorithm module to identify the long frame image and the short frame image in the pre-processed image sequence, and then perform different post-processing on the long frame image and the short frame image respectively.

[0163] S29, the post-processing algorithm module determines that the proportion of the sky area in the mask image is greater than a preset proportion.

[0164] It should be understood that the preset ratio may be a preset value between 0 and 1, such as 0.3, 40%, etc.

[0165] Optionally, the post-processing algorithm module can determine the proportion of the sky area in the mask image in the following way: the post-processing algorithm module calculates the ratio of the number of pixels with a pixel value of 1 in the mask image to the total number of pixels in the mask image, and the ratio is the proportion of the sky area in the mask image.

[0166] S30, the post-processing algorithm module inputs the short frame image in the pre-processed image sequence and the medium frame image selected by the frame selection module into the first Net module, and outputs a linear RGB image A.

[0167] It should be understood that the image format of the short frame image in the preprocessed image sequence and the medium frame image selected by the frame selection module can also be RAW. The linear RGB image A is a frame image, that is, the linear RGB image can be a frame image obtained by image fusion and denoising of the short frame image and the medium frame image.

[0168] The image fusion may be a process of performing pixel processing on the reference frame in the middle frame image selected by the frame selection module using the remaining frames. The remaining frames may be short frame images in the preprocessed image sequence and images other than the reference frame in the middle frame image selected by the frame selection module.

[0169] It should be noted that the reference frame is a medium frame image, and its brightness is usually lower than that of a long frame image. Figure 7 As shown, long frame images can be Figure 8 As shown in the figure, the dark area in the long frame image has more photons and a higher signal-to-noise ratio.

[0170] S31 , the post-processing algorithm module converts the long frame image into a linear RGB image B. For example, the post-processing algorithm module may convert the long frame image into a linear RGB image B using a RAW2RGB tool.

[0171] It should be understood that S31 and S30 may be executed in parallel, or may be executed before or after S30, and this application does not make any specific limitation on this.

[0172] S32, the post-processing algorithm module inputs the linear RGB image A, the linear RGB image B and the mask image into the second Net model, and outputs the correction coefficient of the U channel and the correction coefficient of the V channel.

[0173] It should be understood that the first Net model and the second Net model can refer to the above description, which will not be repeated here. The first Net model can be the same as Net model 1.

[0174] The correction coefficient of the U channel includes the correction coefficient of the U channel corresponding to each pixel in the sky area; for example, the correction coefficient of the U channel corresponding to the first pixel in the sky area can be: the ratio of the difference between the chromaticity value of the first pixel in the linear RGB image B and the chromaticity value of the first pixel in the linear RGB image A, and the chromaticity value of the first pixel in the linear RGB image A, where the first pixel is any pixel in the sky area.

[0175] The correction coefficient of the V channel includes the correction coefficient of the V channel corresponding to each pixel in the sky area; for example, the correction coefficient of the V channel corresponding to the first pixel in the sky area can be: the difference between the concentration value of the first pixel in the linear RGB image B and the concentration value of the first pixel in the linear RGB image A, and the ratio of the concentration value of the first pixel in the linear RGB image A, where the first pixel is any pixel in the sky area.

[0176] Optionally, the second Net model can be trained in the following manner: obtain multiple frames of sample images, the multiple frames of sample images include at least one frame of long frame sample image, at least one frame of short frame sample image and at least one frame of medium frame sample image; perform image fusion on at least one frame of short frame sample image and at least one frame of medium frame sample image to obtain a first sample image; convert at least one frame of long frame sample image into a second sample image, and the format of the second sample image is the same as that of the first sample image; perform image segmentation on the third sample image in the multiple frames of sample images to obtain a mask sample image of the target area in the third sample image; input the second sample image, the first sample image and the mask sample image into the initial model to obtain the target sample image; compare the target sample image and the preset image pixel by pixel to determine the loss function, and the image quality of the preset image is better than that of the target sample image; use the loss function to train the initial model to obtain the second Net model.

[0177] Among them, the multiple frames of sample images and the preset images can be images preset in the electronic device. The process of obtaining the first sample image is similar to the process of the electronic device determining the linear RGB image A; the target area can be the sky area; the process of converting at least one frame of the long frame sample image into the second sample image is similar to S31; the way of obtaining the mask sample image is similar to the way the electronic device obtains the mask image of the sky area; the initial model is the untrained second Net model.

[0178] Optionally, at least one frame of long-frame sample images may include a long-frame sample image with jitter blur, so that when the correction coefficient is obtained through the second Net model, the influence of jitter blur on the target display image can be reduced.

[0179] Optionally, when the loss function is lower than a preset loss function, the electronic device may determine that the initial model training is completed, that is, the first model is obtained.

[0180] S33, the post-processing algorithm module converts the linear RGB image A into the YUV format image A. For example, the post-processing algorithm module converts the linear RGB image A into the YUV format image A using an RGB2YUV tool.

[0181] S34, the post-processing algorithm module uses the correction coefficient of the U channel to correct the U channel of the image A in the YUV format, and uses the correction coefficient of the V channel to correct the V channel of the image A in the YUV format to obtain the target image A in the YUV format.

[0182] It should be noted that the correction coefficient of the U channel includes the correction coefficient of the U channel for each pixel in the sky area of ​​the YUV format image A; the correction coefficient of the V channel includes the correction coefficient of the V channel for each pixel in the sky area of ​​the YUV format image A. The electronic device performs pixel-by-pixel correction processing on the sky area in the YUV format image A.

[0183] In a possible implementation, the post-processing algorithm module performs correction processing on the U channel of the sky area in the YUV format image A by using the following formula: 3 =U 2 +C 0 ×U 2 , where U 3 is the corrected value of the U channel of the sky area in the YUV format image A, U 2 is the value of the U channel of the sky area in the YUV format image A before correction, C 0 is the correction coefficient of U channel.

[0184] In which, when the correction coefficients of the U channels corresponding to multiple pixel points in the sky area are the same, the electronic device can correct the U channels of the multiple pixel points at the same time.

[0185] In a possible implementation, the post-processing algorithm module performs correction processing on the V channel of the sky area in the YUV format image A by using the following formula: 3 =V 2 +C 1 ×V 2 , where V 3 V is the corrected value of the V channel of the sky area in the YUV format image A. 2 is the value of the V channel of the sky area in the YUV format image A before correction, C 1 is the correction coefficient of the V channel.

[0186] It should be understood that in some embodiments, if in S29, the post-processing algorithm module determines that the proportion of the sky area in the mask image is less than or equal to the preset proportion, the post-processing algorithm module may not execute S31, S32 and S34, that is, the image A in YUV format is the target image A in YUV format.

[0187] In this way, when the proportion of the sky area in the mask image is less than or equal to the preset proportion, the electronic device may not perform correction processing on the sky area in the image after the fusion of the medium frame image and the short frame image, which helps to reduce the power consumption of the electronic device.

[0188] S35, the post-processing algorithm module converts the target image A in the YUV format into the target image A in the JPEG format.

[0189] S36, the post-processing algorithm module transmits the target image A in JPEG format to the camera hardware abstraction module.

[0190] S37. The camera hardware abstract module calls an interface to transfer the target image A in JPEG format to the album.

[0191] S38. The photo album saves the target image A in JPEG format.

[0192] It should be understood that in some possible implementations, S35 is optional, and the target image A in JPEG format in S36 to S38 can be replaced by the target image A in YUV format; in other possible implementations, in S35, the post-processing algorithm module can also convert the target image A in YUV format into a target image A in other formats, and the target image A in JPEG format in S36 to S38 can be replaced by the target image A in other formats. Other formats can be, for example, PNG format, GIF format, etc.

[0193] In some possible implementations, after the post-processing algorithm module inputs the medium frame image and the short frame image to the first Net model, the first Net model may also output an image A in YUV format, where the image A in YUV format is a frame image, that is, the image A in YUV format is a frame image obtained after the medium frame image and the short frame image are fused and denoised. Correspondingly, in S31, the post-processing algorithm also converts the long frame image into an image B in YUV format. In addition, the second Net model may use the image A in YUV format, the image B in YUV format, and the mask image to determine the correction coefficients of the U channel and the correction coefficients of the V channel.

[0194] In some other possible implementations, S33 is optional. In S32, the second Net model can use the RGB image A, the RGB image B, and the mask image to determine the correction coefficient of the R channel, the correction coefficient of the G channel, and the correction coefficient of the B channel; in S34, the post-processing algorithm module can use the correction coefficient of the R channel, the correction coefficient of the G channel, and the correction coefficient of the B channel to perform correction processing on the RGB image A to obtain the target image A in RGB format. Subsequently, the post-processing algorithm module can convert the target image A in RGB format into a target image A in other formats.

[0195] It should be understood that in this embodiment, the post-processing algorithm module can use the correction coefficient of the R channel, the correction coefficient of the G channel, and the correction coefficient of the B channel to perform correction processing on the RGB image A in a manner similar to the above-mentioned S34 implementation. Please refer to the above description and will not be repeated here.

[0196] Fig. 9 FIG. 9 is a schematic diagram of an image processing method 900 provided in an embodiment of the present application. Fig. 9 As shown, the electronic device receives an instruction to shoot an image, and obtains 2 short-frame images, 2 medium-frame images, and 1 long-frame image. The electronic device performs image segmentation on one of the two medium-frame images to obtain a sky mask image of the sky area of ​​the one medium-frame image. The sky mask image can be, for example, Figure 6 As shown in (b) in .

[0197] When the electronic device determines that the proportion of the sky area in the sky mask image in the one frame of image is greater than the preset proportion, the electronic device converts the one frame of long frame image into a linear RGB image 3 through the RAW2RGB tool. The process of the electronic device obtaining the linear RGB image 3 is similar to the process of obtaining the linear RGB image B in the method 500, and the above description can be referred to, which will not be repeated here.

[0198] The electronic device also inputs 2 short frame images and 2 medium frame images into Net model 2, and outputs linear RGB image 2. Net model 2 may be the same as the first Net model in method 500 or Net model 1 in method 200. The process of the electronic device obtaining linear RGB image 2 is similar to the process of obtaining linear RGB image A through the first Net model in method 500, and the above description may be referred to, which will not be repeated here.

[0199] The electronic device inputs the linear RGB image 2, the linear RGB image 3, and the sky mask image into the Net model 3, and outputs the correction coefficient of the U channel and the correction coefficient of the V channel. The Net model 3 may be the same as the second Net model in the method 500. The correction coefficient of the U channel may also be understood as the correction coefficient of the U channel in the method 500; the correction coefficient of the V channel may also be understood as the correction coefficient of the V channel in the method 500.

[0200] The electronic device converts the linear RGB image 2 into a YUV format image 2 through the RGB2YUV tool, and performs color correction on the U channel of the YUV format image 2 using the correction coefficient of the U channel; and performs color correction on the V channel of the YUV format image 2 using the correction coefficient of the V channel, thereby obtaining a YUV format target display image 2. The process of the electronic device performing color correction on the YUV format image 2 using the correction coefficient is similar to the implementation of S34 in the method 500, and can refer to the above description, which will not be repeated here.

[0201] It should be noted that, when the electronic device determines that the proportion of the sky area in the sky mask image in the one frame image is less than or equal to the preset proportion, the electronic device may not use one long frame image to obtain the correction coefficient of the U channel and the correction coefficient of the V channel. That is, after the electronic device converts the linear RGB image 2 into the image 2 in the YUV format, the image 2 in the YUV format is the target display image 2 in the YUV format.

[0202] Exemplarily, the target display image 1 obtained by the electronic device through the method 200 can be as follows: Fig.10 The target display image 1 may have a lot of color noise 1001. The target display image 2 obtained by the electronic device through the method 900 may be as shown in FIG. Fig.11 Compared with the target display image 1, the target display image 2 has less color noise in the sky area.

[0203] Fig.12 FIG. 1 is a flow chart of an image processing method 1200 provided in an embodiment of the present application. Fig.12 As shown, the method 1200 includes the following steps:

[0204] S1201. In response to an operation for capturing an image, a plurality of image frames are acquired, where the plurality of image frames include at least one long frame image, at least one short frame image, and at least one medium frame image.

[0205] It should be understood that the operation of capturing an image may be, for example, an operation of a user clicking the capture button 101. The multiple frames of images may be, for example, a preprocessed image sequence and a middle frame image selected by the frame selection module in the method 500. The multiple frames of images may be images in RAW format.

[0206] S1202: Perform image fusion on part or all of the multiple frames of images to obtain a first image.

[0207] Case 1: image fusion is performed on a portion of multiple frame images. S1202 can be implemented in the following manner: image fusion is performed on at least one short frame image and at least one medium frame image to obtain a first image. In this way, the influence of jitter blur in the long frame image on the target image can be reduced, which helps to further improve the image quality of the target image.

[0208] It should be understood that the first image can be, for example, the linear RGB image A in method 500, and the way in which the electronic device obtains the first image is similar to the way in which the electronic device obtains the linear RGB image A in method 500. Please refer to the above description and will not be repeated here.

[0209] Optionally, the first image may also be an image in other formats, for example, the first image is an image in YUV format.

[0210] In case 2, all the multiple frame images are fused to obtain a first image.

[0211] It should be understood that the first image can be, for example, the linear RGB image 1 in method 200, and the way in which the electronic device obtains the first image is similar to the way in which the electronic device obtains the linear RGB image 1 in method 200. Please refer to the above description and will not be repeated here.

[0212] S1203. Use at least one long frame image to perform color correction on a target area in the first image to obtain a target image; wherein the target area is an area in the first image where color noise exists.

[0213] Among them, color noise can also be called color noise, color noise, etc. The area where the color noise exists in the first image can be an area defined according to experience. Then the target area can be a predefined area, such as a sky area, etc. The target area can also be understood as the above-mentioned dark area, etc., and the dark area can be an area defined according to experience, such as a sky area; or, it can also be an area in the first image whose brightness is lower than a fourth threshold, etc. The fourth threshold is a preset positive value.

[0214] It should be understood that the target image may be, for example, the target image A in the YUV format in method 500. The electronic device obtains the target image in a manner similar to the electronic device obtaining the target image A in 500, which can be referred to the above description and will not be repeated here.

[0215] The image processing method of the present application, because the target area in the long-frame image acquires more photons, the signal-to-noise ratio of the target area in the long-frame image is higher, and the color distribution is more uniform and delicate. The electronic device uses the long-frame image to perform color correction on the target area in the first image, which helps to improve the signal-to-noise ratio of the target area in the first image, making the color distribution of the target area in the first image more uniform and delicate, reducing the color noise in the target area in the first image, thereby improving the image quality of the target image.

[0216] As an optional embodiment, method 1200 also includes: performing image segmentation on the second image in the multiple frame images to obtain a mask image corresponding to the target area in the second image; S1203 can be implemented in the following manner: using at least one frame of long frame image and mask image to perform color correction on the target area in the first image.

[0217] In this way, the electronic device can identify the target area through the mask image, thereby facilitating color correction of the target area in the first image.

[0218] The second image may be any one of the multiple frames. Figure 6 As shown in (a) in FIG. 1 , the mask image can be, for example, Figure 6 As shown in (b) in FIG. 1 , in the mask image, the pixel value of the pixel point in the target area can be 1, and the pixel value of the pixel point in the remaining area can be 0.

[0219] The second image may be a medium frame image, a long frame image or a short frame image, as described below.

[0220] In a first possible implementation manner, the second image belongs to a frame of at least one frame-in-frame image.

[0221] In this way, compared with the short frame image, the signal-to-noise ratio of the medium frame image is higher, that is, the edge of the target area in the medium frame image may be clearer, so that the accuracy of the target area determined by the medium frame image is higher. In addition, after the frame selection module transmits part or all of the selected medium frame image to the image signal processor, the image signal processor can determine the mask image. This allows the image signal processor to determine the mask image before acquiring the image sequence, which helps to improve the efficiency of acquiring the mask image.

[0222] It should be understood that the implementation of this embodiment is similar to the implementation of the electronic device acquiring the mask image of the sky area in method 500, wherein the second image can be understood as a frame-in-frame image in method 500, and can be referred to the above description, which will not be repeated here.

[0223] In a second possible implementation manner, the second image belongs to a frame of at least one long frame image.

[0224] In combination with method 500, after the image signal processor obtains the preprocessed image sequence, it can perform image segmentation on a long frame image in the preprocessed image sequence to obtain a mask image of the target area of ​​the long frame image. The long frame image is the second image. In addition, the frame selection module does not need to transmit part or all of the selected medium frame image to the image signal processor, which helps to reduce the power consumption of the electronic device.

[0225] It should be understood that the implementation method in which the second image belongs to one of at least one short-frame image is similar to the second possible implementation method described above, and will not be described in detail here.

[0226] In addition, the electronic device performs color correction on the target area in the first image in the following manner.

[0227] In a possible implementation, color correction is performed on a target area in a first image using at least one long-frame image and a mask image. This can be implemented in the following manner: a correction coefficient of the target area in the first image is obtained using at least one long-frame image, a mask image, and the first image; and color correction is performed on the target area in the first image using the correction coefficient.

[0228] The correction coefficient may be a coefficient corresponding to the format of the first image. For example, if the first image is an image in RGB format, the correction coefficient is a coefficient of at least one of the R channel, the G channel, or the B channel used to correct the target area in the first image; if the first image is an image in YUV format, the correction coefficient is a coefficient of the U channel and / or the V channel used to correct the target area in the first image.

[0229] Exemplarily, the correction coefficient includes a first coefficient and a second coefficient, the first coefficient is used to correct the chromaticity value of the target area in the first image, and the second coefficient is used to correct the concentration value of the target area in the first image; using the correction coefficient to perform color correction on the target area in the first image includes: using the first coefficient to correct the chromaticity value of the target area in the first image to obtain a corrected chromaticity value, and the corrected chromaticity value U 1 Satisfies the following formula: 1 =U 0 +C U ×U 0, where U 0 is the chromaticity value of the target area before correction, C U is the first coefficient; the density value of the target area in the first image is corrected using the second coefficient to obtain a corrected density value, the corrected density value V 1 Satisfies the following formula: V 1 =V 0 +C V ×V 0 , where V 0 is the concentration value of the target area before correction, C V is the second coefficient.

[0230] Since the chromaticity value (U in YUV) and the density value (V in YUV) can describe the image color and saturation and the color of the specified pixel, by correcting the chromaticity value and density value of the target area in the first image, the image color and saturation of the target area in the first image can be corrected, so that the color distribution of the target area in the first image is more uniform and delicate, thereby reducing the color noise of the target area in the first image.

[0231] It should be understood that the way in which the electronic device uses the first coefficient and the second coefficient to perform color correction is similar to the implementation of S34 in method 500, and reference may be made to the above description, which will not be repeated here.

[0232] In a possible implementation, the correction coefficient of the target area is obtained by using at least one long-frame image, a mask image, and a first image. This can be implemented in the following way: converting at least one long-frame image into a third image, the third image having the same format as the first image; and obtaining the correction coefficient of the target area by using the third image, the mask image, and the first image.

[0233] In this way, since the third image and the first image have the same format, it is easier for the electronic device to compare the third image with the first image pixel by pixel and determine the correction coefficients of each channel of the target area in the first image. The correction coefficients may be coefficients that make the target area of ​​the first image closer to the target area of ​​the third image.

[0234] In case 1, the third image and the first image are linear RGB images, and the correction coefficients include: the correction coefficient of the R channel of the target area in the first image; the correction coefficient of the G channel of the target area in the first image; and the correction coefficient of the B channel of the target area in the first image. In this way, the electronic device can correct the R channel, G channel, and B channel of the first image respectively. The correction coefficient of the R channel may include the correction coefficient of the R channel corresponding to each pixel in the target area; the correction coefficient of the G channel may include the correction coefficient of the G channel corresponding to each pixel in the target area; and the correction coefficient of the B channel may include the correction coefficient of the B channel corresponding to each pixel in the target area.

[0235] In case one, the electronic device performs image fusion on at least one frame of image and converts it into a linear RGB image.

[0236] Case 2: The third image and the first image are linear RGB images, and the method 1200 further includes: converting the first image into a brightness, chroma, and density YUV format to obtain a fourth image; correspondingly, color correction of the target area in the first image is color correction of the target area in the fourth image. The correction coefficient includes a first coefficient and a second coefficient.

[0237] The electronic device can correct the image color and saturation of the target area in the fourth image by correcting the U channel (chromaticity value) and V channel (concentration value) of the target area, so that the color distribution of the target area is more uniform and delicate, and the color noise of the target area is reduced.

[0238] It should be understood that the implementation of the second situation is similar to the implementation of the method 500. The fourth image may be, for example, the image A in the YUV format in S33. Please refer to the above description and it will not be repeated here.

[0239] Case 3: The third image and the first image are images in YUV format, and the correction coefficient includes a first coefficient and a second coefficient. That is, the electronic device can determine the first coefficient and the second coefficient using the third image in YUV format, the first image in YUV format, and the mask image.

[0240] It should be understood that the correction method for each channel is similar to the correction method for the U channel or the V channel in method 500, and reference may be made to the above description, which will not be repeated here.

[0241] It should also be noted that the above cases 1 to 3 are only examples, and the format of the first image and / or the third image may also be other formats, which are not listed here one by one for the sake of brevity.

[0242] In a possible implementation, image segmentation of the second image in the multiple frames of images may be implemented in the following manner: when it is determined that the ambient brightness is less than or equal to a preset brightness, image segmentation is performed on the second image.

[0243] When the ambient brightness is high, that is, when the current shooting scene is not a dark light scene, the number of photons obtained by the medium frame image and the short frame image is also high, so that the color noise in the target area of ​​the target image is low. Therefore, the electronic device can obtain the target image by using the long frame image to perform color correction on the target area of ​​the first image in a dark light scene; in a non-dark light scene, the electronic device does not need to obtain the mask image, nor does it need to use the long frame image to perform color correction on the target area of ​​the first image, that is, the first image or the image obtained after the format conversion of the first image is the target image, so that the power consumption of the electronic device is relatively low.

[0244] In addition, image segmentation of the second image in the multiple frames of images may also be implemented in the following manner: when it is determined that the shooting scene is an HDR scene and / or a dark light scene, image segmentation is performed on the second image.

[0245] Among them, dark light scenes and HDR scenes can be judged based on parameters such as ambient brightness and ambient illumination.

[0246] In a possible implementation, color correction is performed on a target area in a first image using at least one long-frame image and a mask image. This can be implemented in the following manner: when a proportion of the target area in the mask image is greater than or equal to a preset threshold, color correction is performed on the target area in the first image using at least one long-frame image and a mask image.

[0247] It should be understood that the implementation of this embodiment is similar to the implementation of S29 in method 500. The preset threshold value may be, for example, a preset ratio in S29. Please refer to the above description and will not be repeated here.

[0248] It should be noted that the correction coefficient in the embodiment of the present application can also be called an offset coefficient, etc., color correction can also be called offset processing, etc., and color noise can also be called color noise, color noise, etc. This application does not make any specific limitations on this.

[0249] It should be noted that the module names involved in the embodiments of the present application can be defined as other names as long as the functions of each module can be achieved, and there is no specific restriction on the names of the modules.

[0250] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0251] The image processing method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above list sorting method.

[0252] Fig.13 A schematic block diagram of an image processing device 1300 provided in an embodiment of the present application. The device 1300 includes a processor 1301, a communication interface 1302, and a memory 1303. Among them, the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through an internal connection path, the memory 1303 is used to store instructions, and the processor 1301 is used to execute the instructions stored in the memory 1303. The communication interface 1302 can be used to send signals to other devices (such as the processor 1301 or the touch screen of an electronic device), and can also be used to receive signals from other devices (such as the memory 1303). Exemplarily, the communication interface 1302 reads the instructions stored in the memory 1303 and sends the instructions to the processor 1301.

[0253] It should be understood that the device 1300 can be specifically an electronic device in the above-mentioned embodiment, and can be used to execute the various steps and / or processes corresponding to the electronic device in the above-mentioned method embodiment. Optionally, the memory 1303 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 1301 may be used to execute instructions stored in the memory, and when the processor 1301 executes instructions stored in the memory, the processor 1301 is used to execute the various steps and / or processes of the above-mentioned method embodiment.

[0254] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0255] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor executes the instructions in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0256] The image processing method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic devices include terminal devices, and the specific device form of the terminal devices can refer to the above related descriptions, which will not be repeated here.

[0257] An embodiment of the present application provides a terminal device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the above method.

[0258] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0259] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0260] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0261] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0262] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0263] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: In response to an operation for capturing an image, a plurality of frames of images are acquired, wherein the plurality of frames of images include at least one long frame image, at least one short frame image, and at least one medium frame image; Performing image fusion on part or all of the multiple frames of images to obtain a first image; Using the at least one long-frame image, color correction is performed on a target area in the first image to obtain a target image; wherein the target area is an area in the first image where color noise exists.

2. The method according to claim 1, characterized in that The method further comprises: Performing image segmentation on a second image in the multiple frames of images to obtain a mask image corresponding to the target area in the second image; The step of using the at least one long frame image to perform color correction on the target area in the first image includes: The target area in the first image is color corrected by using the at least one long frame image and the mask image.

3. The method according to claim 2, characterized in that The step of using the at least one long frame image and the mask image to perform color correction on the target area in the first image includes: Obtaining a correction coefficient of the target area in the first image by using the at least one long frame image, the mask image and the first image; The correction coefficient is used to perform color correction on the target area in the first image.

4. The method according to claim 3, characterized in that The correction coefficient includes a first coefficient and a second coefficient, the first coefficient is used to correct the chromaticity value of the target area in the first image, and the second coefficient is used to correct the density value of the target area in the first image; The step of using the correction coefficient to perform color correction on the target area in the first image includes: The chromaticity value of the target area in the first image is corrected by using the first coefficient to obtain a corrected chromaticity value, and the corrected chromaticity value U1 satisfies the following formula: U1=U0+C U ×U0, Wherein, U0 is the chromaticity value of the target area before correction, C U is the first coefficient; The density value of the target area in the first image is corrected by using the second coefficient to obtain a corrected density value, and the corrected density value V1 satisfies the following formula: V1=V0+C V ×V0, Wherein, V0 is the concentration value of the target area before correction, C V is the second coefficient.

5. The method according to claim 3 or 4, characterized in that: The obtaining of a correction coefficient of the target area in the first image by using the at least one long frame image, the mask image and the first image includes: Converting the at least one long-frame image into a third image, wherein the third image has the same format as the first image; The correction coefficient is obtained by using the third image, the mask image and the first image.

6. The method according to any one of claims 2 to 5, characterized in that The second image belongs to a frame among the at least one frame.

7. The method according to any one of claims 2 to 6, characterized in that The performing image segmentation on the second image in the multiple frames of images comprises: When it is determined that the ambient brightness is less than or equal to the preset brightness, image segmentation is performed on the second image.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Convert the first image into a brightness, chroma, and density YUV format to obtain a fourth image; The performing color correction on the target area in the first image includes: Color correction is performed on the target area in the fourth image.

9. The method according to any one of claims 2 to 8, characterized in that The step of using the at least one long frame image and the mask image to perform color correction on the target area in the first image includes: When the proportion of the target area in the mask image is greater than or equal to a preset threshold, color correction is performed on the target area in the first image using the at least one long frame image and the mask image.

10. The method according to any one of claims 1 to 9, characterized in that The step of fusing part or all of the multiple frames of images to obtain a first image includes: The at least one short-frame image and the at least one medium-frame image are fused to obtain the first image.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the electronic device executes the method as described in any one of claims 1 to 10.

12. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises computer instructions, and when the computer instructions are executed on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product comprises a computer program code, and when the computer program code is run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 10.

Images