Image processing method and apparatus

Abstract

Embodiments of the present application provide an image processing method and device, and relate to the technical field of terminals. The method comprises: in response to an operation for shooting an image, acquiring multiple frames of images, the multiple frames of images comprising at least one long frame of image, at least one short frame of image, and at least one medium frame of image; performing image fusion on part or all of the multiple frames of images to obtain a first image; and performing color correction on a target region in the first image by using the at least one long frame of image to obtain a target image; wherein the target region is a region in the first image in which color noise exists. In this way, the signal-to-noise ratio of the target region in the first image is improved, the color distribution of the target region in the first image is more uniform and delicate, the color noise points of the target region in the first image are reduced, and thus the image quality of the target image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to an image processing method and device. BACKGROUND

[0002] Currently, some electronic devices can improve the image quality of the images captured by the electronic devices through multi-frame noise reduction processing. For example, when a user captures an image by using a camera of an electronic device with a high dynamic range (HDR) function, the camera of the electronic device can capture multiple images at different exposure levels. Then, the electronic device can perform image fusion and noise reduction processing on the multiple images to obtain a target image.

[0003] However, in dark light or high dynamic range scenarios, the image quality of the target image obtained by the electronic device can still be low. For example, the target image captured by the electronic device in a dark light scenario has many color noise points. SUMMARY

[0004] Embodiments of the present application provide an image processing method and device, which are applied to the technical field of terminals. In dark light or high dynamic range scenarios, the color noise points in the target image can be reduced, the image quality of the target image can be improved, and the user experience of image capturing can be improved.

[0005] In a first aspect, an image processing method is provided. The method comprises: in response to an operation for capturing an image, obtaining multiple images, the multiple images comprising at least one long-frame image, at least one short-frame image, and at least one medium-frame image; performing image fusion on part or all of the multiple images to obtain a first image; and performing color correction on a target region in the first image by using the at least one long-frame image to obtain a target image, wherein the target region is a region in the first image that has color noise.

[0006] The image processing method provided by the present application can improve the signal-to-noise ratio of the target region in the first image, make the color distribution of the target region in the first image more uniform and delicate, reduce the color noise points of the target region in the first image, and thus improve the image quality of the target image.

[0007] Optionally, performing image fusion on part or all of the multiple images to obtain the first image comprises: performing image fusion, noise reduction processing, and demosaicing processing on part or all of the multiple images to obtain the first image. In this way, the first image has less noise, which facilitates the correction effect on the target region in the first image.

[0008] The image fusion, the noise reduction processing, and the demosaicing processing on part or all of the multiple frames of images can be implemented by, for example, inputting part or all of the multiple frames of images into a first Net model to obtain a first image. The first Net model can refer to the description below.

[0009] With reference to the first aspect, in some possible implementations, the method further includes: performing image segmentation on a second image in the multiple frames of images to obtain a mask image corresponding to a target region in the second image; and performing color correction on the target region in the first image by using the at least one long frame of image, including: performing color correction on the target region in the first image by using the at least one long frame of image and the mask image. In this way, the electronic device can identify the target region by using the mask image, so as to facilitate the color correction on the target region in the first image.

[0010] With reference to the first aspect, in some possible implementations, the performing color correction on the target region in the first image by using the at least one long frame of image and the mask image includes: obtaining a correction coefficient of the target region in the first image by using the at least one long frame of image, the mask image, and the first image; and performing color correction on the target region in the first image by using the correction coefficient.

[0011] The correction coefficient can include a correction coefficient of each pixel point in the target region in the first image. The correction coefficient can be a coefficient corresponding to a format of the first image. For example, the first image is an image in a red green blue (RGB) format, and the correction coefficient is a coefficient for correcting at least one of an R channel, a G channel, or a B channel in the target region in the first image; or the first image is an image in a YUV format, and the correction coefficient is a coefficient for correcting a U channel and / or a V channel in the target region in the first image. In this way, the electronic device can perform pixel-by-pixel color correction on the target region in the first image, so as to reduce color noise in the target region.

[0012] With reference to the first aspect, in some possible implementations, the correction coefficient includes a first coefficient and a second coefficient, the first coefficient is used for correcting a chroma value of the target region in the first image, and the second coefficient is used for correcting a concentration value of the target region in the first image; and the performing color correction on the target region in the first image by using the correction coefficient includes: correcting the chroma value of the target region in the first image by using the first coefficient to obtain a corrected chroma value, and the corrected chroma value U1 satisfies the following formula: U1=U0+C U ×U0, where U0 is the chroma value of the target region before correction, and C Uis a first coefficient; and correcting the concentration value of the target region in the first image by using a second coefficient to obtain a corrected concentration value, the corrected concentration value V1 satisfies the following formula: V1 = V0 + C V ×V0, wherein V0 is the concentration value of the target region before correction, C V is the second coefficient.

[0013] The first coefficient may be, for example, a correction coefficient of a U channel below; and the second coefficient may be, for example, a correction coefficient of a V channel below, and the target region may be, for example, a sky region below. Since the chroma value (U in YUV) and the concentration value (V in YUV) can describe the image color and saturation and specify the color of a pixel. Therefore, by correcting the chroma value and the concentration value of the target region in the first image, the image color and saturation of the target region in the first image can be corrected, so that the color distribution of the target region in the first image is more uniform and delicate, thereby reducing the color noise of the target region in the first image.

[0014] With reference to the first aspect, in some possible implementation manners, the correction coefficient of the target region is obtained by using the at least one long frame image, the mask image and the first image, including: converting the at least one long frame image into a third image, the third image having the same format as the first image; and obtaining the correction coefficient of the target region by using the third image, the mask image and the first image. In this way, since the third image and the first image have the same format, the electronic device can more easily compare the third image and the first image pixel by pixel to determine the correction coefficient of each channel of the target region in the first image. For example, if the third image and the first image are both images in RGB format, the electronic device can compare the R channel, the G channel and the B channel of a pixel point A in the third image and a pixel point B in the first image to obtain the correction coefficient corresponding to the pixel point B. The pixel point A corresponds to the pixel point B in position.

[0015] With reference to the first aspect, in some possible implementation manners, the second image belongs to one of the at least one middle frame image.

[0016] In this way, compared with the short frame image, the middle frame image has a higher signal-to-noise ratio, that is, the edge of the target region in the middle frame image can be clearer, so that the accuracy of the target region determined by the middle frame image is higher. In addition, after the selected part or all of the middle frame images are transmitted to the image signal processor by the frame selection module, the image signal processor can determine the mask image. This makes the image signal processor can determine the mask image before obtaining the image sequence, which helps to improve the acquisition efficiency of the mask image.

[0017] With reference to the first aspect, in some possible implementation manners, the image segmentation on the second image in the plurality of images includes: performing image segmentation on the second image in a case where the ambient brightness is less than or equal to a preset brightness.

[0018] Since the number of photons obtained by the mid-frame image and the short-frame image is also relatively large when the ambient brightness is relatively large, that is, the current shooting scene is not a dark-light scene, the color noise in the target region of the target image can be relatively small or non-existent. Therefore, the electronic device can obtain the target image by using the long-frame image to perform color correction on the target region of the first image in a dark-light scene. In a non-dark-light scene, the electronic device can not obtain the mask image, and does not need to use the long-frame image to perform color correction on the target region of the first image, that is, the first image or an image obtained by performing format conversion on the first image is the target image, so that the power consumption of the electronic device is relatively small.

[0019] With reference to the first aspect, in some possible implementation manners, the method further includes: converting the first image into a luminance chrominance density YUV format to obtain a fourth image; and performing color correction on the target region in the first image includes: performing color correction on the target region in the fourth image. In this way, the electronic device can correct the image color and saturation of the target region in the fourth image by correcting the U channel (chrominance value) and the V channel (density value) of the target region, so that the color distribution of the target region is more uniform and delicate, and the color noise of the target region is reduced.

[0020] With reference to the first aspect, in some possible implementation manners, the color correction on the target region in the first image by using the at least one long-frame image and the mask image includes: in a case where the proportion of the target region in the mask image is greater than or equal to a preset threshold, performing the color correction on the target region in the first image by using the at least one long-frame image and the mask image.

[0021] The preset threshold is a preset positive value, for example, 30% or the like. In this way, when the proportion of the target region in the mask image is relatively large, the color noise of the target region can be relatively large, and the at least one long-frame image can be used to perform color correction on the target region in the first image to reduce the color noise in the target region. When the proportion of the target region in the mask image is relatively small or the target region does not exist, the electronic device can also not use the at least one long-frame image to perform color correction on the target region in the first image, so that the power consumption of the electronic device is relatively small.

[0022] With reference to the first aspect, in some possible implementation manners, the image fusion on part or all of the plurality of images to obtain the first image includes: performing image fusion on the at least one short-frame image and the at least one mid-frame image to obtain the first image.

[0023] Since the long-frame image is prone to jitter blur, the long-frame image is not fused, so that the influence of the jitter blur of the long-frame image on the target image can be reduced; and since the effect of the noise reduction processing on the image with jitter blur can be poor, when the subsequent electronic device performs noise reduction processing on the fused image, the noise reduction effect is better, which is helpful to further improve the image quality of the target image.

[0024] In addition, the long-frame image is not fused, so that the electronic device does not need to perform pixel-by-pixel registration on the long-frame image and the short-frame image, which is helpful to reduce the power consumption of the electronic device.

[0025] In a second aspect, an embodiment of the present application provides an image processing apparatus. The image processing apparatus can be an electronic device, or a chip or chip system in the electronic device. The image processing apparatus can include a display unit and a processing unit. When the image processing apparatus is an electronic device, the display unit can be a display screen. The display unit is configured to perform the display step, so that the electronic device implements an image processing method described in the first aspect or any possible implementation manner of the first aspect.

[0026] When the image processing apparatus is an electronic device, the processing unit can be a processor. The image processing apparatus can further include a storage unit, which can be a memory. The storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation manner of the first aspect. When the image processing apparatus is a chip or chip system in the electronic device, the processing unit can be a processor. The processing unit executes the instructions stored in the storage unit, so that the electronic device implements an image processing method described in the first aspect or any possible implementation manner of the first aspect. The storage unit can be a storage unit (for example, a register, a cache, etc.) in the chip, or a storage unit (for example, a read-only memory, a random access memory, etc.) in the electronic device and located outside the chip.

[0027] For example, the processing unit is configured to perform image fusion on part or all of the multiple-frame images to obtain a first image; and the display unit is configured to display a target image.

[0028] In a third aspect, an embodiment of the present application provides an electronic device including a processor and a memory. The memory is configured to store code instructions, and the processor is configured to run the code instructions to perform the method described in the first aspect or any possible implementation manner of the first aspect.

[0029] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0030] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program, and when the computer program is run on a computer, the computer is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0031] In a sixth aspect, the present application provides a chip or chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is configured to run a computer program or instructions to execute the method described in the first aspect or any possible implementation manner of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0032] In a possible implementation, the chip or chip system described in the present application further includes at least one memory, and the at least one memory stores instructions. The memory can be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0033] It should be understood that the second aspect to the sixth aspect of the present application correspond to the technical solution of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding possible implementation manner are similar, which will not be repeated. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present application;

[0035] Figure 2 A process schematic diagram of an image processing method;

[0036] Figure 3 A schematic block diagram of a hardware architecture of an electronic device provided by an embodiment of the present application;

[0037] Figure 4 A schematic block diagram of a software architecture of an electronic device provided by an embodiment of the present application;

[0038] Figure 5 A flowchart of an image processing method provided by an embodiment of the present application;

[0039] Figure 6 A schematic diagram of a mask image acquisition process provided by an embodiment of the present application;

[0040] Figure 7 A schematic diagram of a reference frame image provided for an embodiment of the present application;

[0041] Figure 8 A schematic diagram of a long frame image provided for an embodiment of the present application;

[0042] Figure 9 A flowchart of another image processing method provided for an embodiment of the present application;

[0043] Figure 10 A schematic diagram of a target display image 1 provided for an embodiment of the present application;

[0044] Figure 11 A schematic diagram of a target display image 2 provided for an embodiment of the present application;

[0045] Figure 12 A flowchart of another image processing method provided for an embodiment of the present application;

[0046] Figure 13 A schematic block diagram of an image processing apparatus provided for an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to clearly describe the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0048] 1. Stagger high dynamic range (SHDR), which can also be referred to as stagger, is an image output frame mode with "row" as the output unit. SHDR can realize the acquisition of multiple frames of images with different exposure times at one time of shooting, for example, the acquisition of long frames, short frames and normal frames at one time of shooting. The SHDR output frame mode can reduce the time interval between frames, thereby reducing the occurrence of ghosting.

[0049] 2. Normal frame, which can also be referred to as normal exposure image, N (normal) frame image, N frame, middle frame or middle frame image, etc., is an image obtained by a camera under the condition that the exposure amount is 0 EV. That is, the exposure amount of the normal exposure image is 0 EV. Here, 0 EV is a relative value, not that the exposure amount is 0. Exemplarily, exposure amount = exposure time * ISO (sensitivity). Assuming that the normal exposure image is obtained under the condition that ISO is 200 and the exposure time is 50 milliseconds, 0 EV actually corresponds to the product of 200 and 50 milliseconds.

[0050] 3. Short frame, which can also be referred to as short frame image, S (short) frame or S frame image, is an image captured by a camera in a case where the exposure is less than 0 EV.

[0051] 4. Long frame, which can also be referred to as long frame image, L (long) frame or L frame image, is an image captured by a camera in a case where the exposure is greater than 0 EV.

[0052] 5. YUV is a color encoding method. Among them, Y (Luminance, Luma) represents brightness, and U and V represent chroma (Chrominance, Chroma).

[0053] 6. Linear RGB image can be understood as an image having a linear relationship between luminance in a physical space and image display luminance. For example, the linear RGB image can be an RGB image that has not undergone gamma correction.

[0054] 7. Other terms

[0055] In the embodiments of the present application, the same items or similar items having substantially the same functions and effects are distinguished by using "first", "second", and the like. For example, the first chip and the second chip are only used to distinguish different chips, and do not limit the order. Those skilled in the art can understand that "first", "second", and the like do not limit the number and execution order, and "first", "second", and the like do not necessarily mean different.

[0056] It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design schemes. Rather, the words "exemplary" or "for example" are used in the sense of presenting a related concept in a specific manner.

[0057] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The association relationship of the associated objects is described by "and / or", which means that there can be three kinds of relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c, can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0058] 8. Electronic device

[0059] The electronic device of the embodiments of the present application can include a handheld device with a camera function, a vehicle-mounted device, etc. For example, some electronic devices are: a mobile phone, a tablet computer, a palm computer, a notebook computer, a mobile Internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with a wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc., and the embodiments of the present application are not limited thereto.

[0060] As an example but not limitation, in the embodiments of the present application, the electronic device can also be a wearable device. The wearable device can also be referred to as a wearable smart device, which is a general term for devices that are designed and developed by applying wearable technology to daily wear, such as glasses, gloves, watches, clothing, and shoes, etc. The wearable device is a portable device that is directly worn on the body or integrated into the clothes or accessories of the user. The wearable device is not only a hardware device, but also a device that realizes powerful functions through software support and data interaction and cloud interaction. The general wearable smart device includes a device with full functions and large size, which can realize complete or partial functions without relying on a smart phone, such as a smart watch or smart glasses, etc., and a device that focuses on a certain application function and needs to cooperate with other devices such as a smart phone, such as various smart wristbands and smart jewelry for monitoring vital signs, etc.

[0061] Furthermore, in this embodiment of the application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0062] The electronic devices in the embodiments of this application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0063] In this embodiment, the electronic device or various network devices include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0064] Figure 1 This is a schematic diagram of a shooting scenario provided in an embodiment of this application. When the electronic device receives an input operation from the user to open the camera, the electronic device can display as shown below. Figure 1 The shooting interface shown in (a) is as follows. Figure 1 As shown in interface (a), the shooting interface displayed by the electronic device may include a shooting button 101 and a preview screen 102. The preview screen 102 can display the scene to be captured by the user; when the user clicks the shooting button 101, the electronic device can capture the image displayed in the preview screen 102, and then the electronic device can display the image as shown in the preview screen 102. Figure 1 The graphical interface shown in (b) is as follows. Figure 1 As shown in interface (b), this interface can be the interface in the photo album of an electronic device. Interface (b) may include the target display image 103 captured by the electronic device, as well as multiple buttons such as share, favorite, edit, delete, and more.

[0065] However, in low-light or high dynamic range (HDR) scenes, the shorter exposure time of electronic devices can result in a lower signal-to-noise ratio in the captured image. HDR scenes, also known as high dynamic range images, can be understood as scenes where the difference between the maximum and minimum brightness values ​​in an image captured by an electronic device exceeds a preset difference.

[0066] If an electronic device performs noise reduction processing on the captured image before displaying the target image, it may cause a loss of detail in the target image. If the electronic device performs gamma brightening or gamma correction processing on the captured image before displaying the target image, it may amplify the noise in the dark areas of the target image, making the noise in the target image more obvious.

[0067] Therefore, electronic devices can currently employ multi-frame noise reduction processing to obtain the target display image, thereby improving the image quality of the target display image. Multi-frame noise reduction processing can be understood as the process by which electronic devices perform image fusion, noise reduction, and other processing on multiple frames of images with different exposure levels.

[0068] For example, such as Figure 1 As shown in interface (a), when the electronic device receives a user's trigger operation on the shooting button 101, the electronic device can output frames based on HDR or SHDR and acquire an image sequence. That is, the electronic device can capture N frames of images at different exposure levels and / or exposure times. These N frames can include a long frame image, b short frame images, and c medium frame images, where N, a, b, and c are positive integers. Furthermore, the electronic device can perform fusion processing and noise reduction processing on the N frames to obtain the target display image 103. That is, the electronic device can display... Figure 1 The interface shown in (b) is shown in the middle.

[0069] It should be noted that, Figure 1 Interface (a) is an example of a camera's display interface on an electronic device, and interface (b) is an example of an electronic device's display interface showing the captured target image 103. For example, interfaces (a) and (b) may also include more or fewer controls or buttons. Figure 1 Interfaces (a) and (b) in this application do not constitute a limitation on the embodiments of this application.

[0070] Below, in conjunction with Figure 2 Taking the above N frames of images, including two medium-frame images, two short-frame images, and one long-frame image, as an example, the process of noise reduction processing of the electronic device to fuse N frames of images to obtain the target display image 1 is explained.

[0071] Figure 2 This is a schematic diagram of an image processing method 200. For example... Figure 2 As shown, after receiving the shooting command, the electronic device can acquire N frames of images, including two medium-length frames, two short frames, and one long frame. The electronic device inputs these N frames into a neural network (Net) model 1 and obtains a linear red-green-blue (RGB) image 1 output by Net model 1. Then, the electronic device can convert this linear RGB image 1 into a YUV format image 1 and perform filtering on the YUV format image 1 to obtain the target display image 1.

[0072] It should be understood that Net model 1 can also be called denoising Net, etc., and can be a model with one or more functions such as image fusion, denoising, interpolation, and demosaic. In addition, Net model 1 can be a convolutional neural network model, a one-dimensional convolutional neural network model, a deep learning convolutional neural network model, or a two-dimensional convolutional neural network model, etc. For example, the denoising Net model can be HDR Net, Residual Neural Network (ResNet), U-Net, Transformer Net, or Restormer Net, etc., and this application does not make specific limitations in this regard.

[0073] It should be understood that converting a linear RGB image 1 to a YUV format image 1 can be achieved using tools such as RGB2YUV, and this application does not limit the specific method used for image format conversion.

[0074] It should also be understood that the filtering process can be median filtering or mean filtering, which can reduce the noise of the YUV format image 1 and improve the signal-to-noise ratio of the YUV format image 1. This application does not specifically limit the filtering process.

[0075] Optionally, the filtering process can be implemented by the electronic device filtering the UV channels of the YUV format image 1 based on the Y channel (luminance). For example, for regions in the YUV format image 1 with luminance greater than or equal to a first threshold, one filtering method can be used; for regions in the YUV format image 1 with luminance less than or equal to a second threshold, another filtering method can be used.

[0076] It should be noted that the process of obtaining the target display image 1 from N frames can also be called multi-frame noise reduction processing. Multi-frame noise reduction processing helps to make the image details of the target display image 1 displayed on the electronic device clearer, thus improving the image quality of the target display image 1.

[0077] For the short-frame image photographed by the electronic device, since the short-frame image undergoes a short exposure time, the dark region in the short-frame image obtains a small number of photons, and further, the signal-to-noise ratio of the dark region in the short-frame image is low, and color noise is likely to occur in the dark region of the target display image.

[0078] In addition, for the long-frame image, since the long-frame image undergoes a long exposure time, the dark region in the long-frame image can obtain a large number of photons, and further, the signal-to-noise ratio of the long-frame image is high. However, since the long-frame image undergoes a long exposure time, the long-frame image is prone to blur caused by shaking. The blur caused by shaking can reduce the noise reduction effect of the electronic device on the fused image, and thus can cause the color noise in the dark region of the target display image to be more obvious.

[0079] Therefore, after the electronic device performs multi-frame noise reduction processing on the N images, the dark region of the target display image obtained can have a low signal-to-noise ratio and color noise. For example, the sky region of an image photographed by the electronic device in a dark light scene has more color noise.

[0080] It should be noted that in the embodiments of the present application, the dark region in the image is a region with a relatively low brightness in the image, for example, a region with a brightness lower than a third threshold in the image. Alternatively, it can also be a dark region predefined according to experience. For example, according to experience, the brightness of the sky region photographed in a dark light scene is usually low and is prone to color noise. Therefore, for an image photographed by the electronic device in a dark light scene, the dark region of the image can refer to the sky region.

[0081] To solve the above technical problems, the present application provides an image processing method. Since the signal-to-noise ratio of the dark region in the long-frame image is usually higher than that of the dark region in the short-frame image and the medium-frame image, the electronic device can use the long-frame image to perform color correction on the dark region in the fused image of part or all of the N images to obtain a target display image. In this way, through color correction, the image color and saturation of the dark region in the fused image can be corrected, the color of the dark region in the target display image is more uniform and delicate, which helps to improve the signal-to-noise ratio of the dark region in the target display image and reduce the color noise in the dark region of the target display image.

[0082] The following will be described in combination with Figures 3 to 12The image processing method of the present application is described in detail. The image processing method of the embodiment of the present application can be executed by an electronic device provided with a camera, or can be executed by a chip, a chip system or a processor supporting the electronic device to implement the image processing method, or can be executed by a logic module or software capable of implementing all or part of the functions of the electronic device, and the present application does not make specific limitations thereon. The image processing method of the embodiment of the present application is described in detail below with the electronic device as the main body of execution.

[0083] In order to better understand the electronic device in the embodiment of the present application, the following describes the electronic device in the embodiment of the present application in conjunction with Figure 3 and Figure 4 The hardware structure and software structure of the electronic device in the embodiment of the present application are described in detail.

[0084] Figure 3 A structural schematic diagram of an electronic device 300 suitable for the present application is shown.

[0085] The electronic device 300 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0086] It should be noted that, Figure 3 The structures shown do not constitute specific limitations on the electronic device 300. In other embodiments of the present application, the electronic device 300 can include more or fewer components than those shown, or the electronic device 300 can include a combination of some of the components shown, or the electronic device 300 can include sub-components of some of the components shown. For example, Figure 3 The proximity light sensor 180G shown can be optional. Figure 3 The proximity light sensor 180G shown can be optional. Figure 3 The proximity light sensor 180G shown can be optional. Figure 3 The proximity light sensor 180G shown can be optional.Figure 3 The illustrated components can be implemented in hardware, software, or a combination of both.

[0087] The processor 110 can include one or more processing units. For example, the processor 110 can include at least one of an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU). Among them, different processing units can be independent devices, or can be integrated devices.

[0088] In a possible implementation, the processor 110 can start the camera 193 based on the starting; acquire N frames of images in the current photographing environment; and process the N frames of images to obtain a target display image.

[0089] In a possible implementation, the processor 110 can determine the camera 193, the exposure amount, the exposure time, and other out-image parameters that need to be called according to different ambient illuminance and / or ambient brightness, and instruct the camera 193 to capture an image.

[0090] The ambient light sensor 180L is used to sense ambient light brightness, and is used to automatically adjust white balance when photographing. Optionally, the processor 110 can acquire ambient illuminance and / or ambient brightness from the ambient light sensor 180L.

[0091] The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0092] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that have just been used or are repeatedly used by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0093] The electronic device 300 can realize the photographing function through the ISP, the camera 193, the GPU, the display screen 194, and the application processor.

[0094] ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture a still image or a video. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and is converted into an image visible to the naked eye. In some embodiments, the ISP can be arranged in the camera 193. Exemplarily, the electrical signal transmitted by the camera photosensitive element to the ISP can be an image sequence, and the ISP can pre-process the image sequence. The ISP can also output the pre-processed image sequence to a digital signal processor (DSP) or a GPU for further processing, such as image fusion, noise reduction processing, and color correction, so as to obtain a target image.

[0095] In some embodiments, the electronic device 300 can include 1 or N cameras 193, where N is a positive integer greater than 1.

[0096] The digital signal processor is used to process digital signals, which can process not only digital image signals but also other digital signals. For example, the digital signal processor can determine whether the proportion of a target region in a mask image is greater than or equal to a preset proportion, etc.

[0097] The electronic device 300 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information. Exemplarily, the processor 110 can call the GPU to perform image fusion and noise reduction processing on at least one frame of short frame image and at least one frame of middle frame image; the processor 110 can also call the GPU to perform color correction on a target region, such as a sky region, etc.

[0098] The display screen 194 can be used to display a target image or a thumbnail, etc.

[0099] It should be understood that Figure 3 The connection relationship between the modules shown is only illustrative and does not constitute a limitation on the connection relationship between the modules of the electronic device 300. Alternatively, the modules of the electronic device 300 can also use a combination of the above-mentioned various connection modes.

[0100] Figure 4 is a schematic diagram of an architecture (including a software system and part of hardware) to which the embodiments of the present application are applied. As shown in Figure 4As shown, the application architecture is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the application architecture can be divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), the driver layer, and the hardware layer.

[0101] like Figure 4 As shown, the application layer can include applications such as camera, gallery, and system UI. The system UI is used to display the interface of the electronic device, such as displaying the target image captured by the electronic device.

[0102] like Figure 4 As shown, the application framework layer includes a camera access interface. The camera access interface includes camera management and camera devices. The hardware abstraction layer includes a camera hardware abstraction module and a camera algorithm library. The camera hardware abstraction module includes multiple camera devices. The camera algorithm library includes a post-processing algorithm module, a decision module, and a frame selection module.

[0103] It should be understood that the decision-making module can also be placed in other layers. As one possible implementation, the decision-making module can be placed in the application layer or the application framework layer.

[0104] The driver layer is used to drive hardware resources. The driver layer can include multiple driver modules. For example... Figure 4 As shown, the driver layer includes camera device drivers, digital signal processor drivers, and graphics processor drivers, etc.

[0105] The hardware layer includes sensors, image signal processors, digital signal processors, and graphics processors. The sensors may include multiple sensors, a Time-of-Flight (TOF) camera, and a multispectral sensor. The image signal processor may include an ISP module.

[0106] For example, a user can click a camera application. When the user clicks the camera to take a picture, a picture taking instruction can be sent to the camera hardware abstraction module through the camera access interface. The camera hardware abstraction module calls the camera device driver and calls the camera algorithm library. The decision module in the camera algorithm library can instruct the frame selection module to select a frame; it can also determine the out-picture parameters according to the environmental illumination and other parameters. The out-picture parameters can include information indicating the camera to be called by the electronic device to take a picture, the exposure amount, the exposure time, etc. The out-picture parameters are sent to the camera hardware abstraction module. The camera hardware abstraction module sends the out-picture parameters configured by the decision module to the camera device driver. The camera device driver sends the out-picture parameters sent by the camera hardware abstraction module to the hardware layer, such as sending the out-picture mode of the sensor to the sensor and sending the parameter configuration of the ISP module to the image signal processor. The sensor performs out-picture based on the out-picture mode of the sensor, for example, which camera or sensor to call. The image signal processor performs corresponding preprocessing based on the parameter configuration of the ISP module. The camera algorithm library is also used to send digital signals to the digital signal processor driver in the driver layer, so that the digital signal processor driver calls the digital signal processor in the hardware layer to perform digital signal processing. The digital signal processor can return the processed digital signal to the camera algorithm library through the digital signal processor driver. The camera algorithm library is also used to send digital signals to the graphics signal processor driver in the driver layer, so that the graphics signal processor driver calls the graphics processor in the hardware layer to perform digital signal processing. The graphics processor can return the processed graphics data to the camera algorithm library through the graphics processor driver. For example, the image processor can perform filtering, correction, etc.

[0107] In addition, the image output by the image signal processor can be sent to the camera device driver. The camera device driver can send the image output by the image signal processor to the camera hardware abstraction module. The camera hardware abstraction module can send the image to the post-processing algorithm module for further processing, or send the image to the camera access interface. The camera access interface can send the image returned by the camera hardware abstraction module to the camera.

[0108] The post-processing algorithm module can include a first Net model and a second Net model. The first Net model can be used for image fusion, noise reduction processing, and demosaicing processing of short frame images and medium frame images. The second Net model can be used to determine the correction coefficients of the U channel and the correction coefficients of the V channel. The camera algorithm library can also call the image processor driver to drive the image processor to correct the target region in the image using the correction coefficients of the U channel and the correction coefficients of the V channel.

[0109] It should be understood that the first Net model and the second Net model can be a convolutional neural network model, a one-dimensional convolutional neural network model, a deep learning convolutional neural network model, or a two-dimensional convolutional neural network model, etc., for example, the first Net model and the second Net model can be an HDR Net, a residual neural network (ResNet), a U-Net, a transformer Net, or a restormer Net, etc., which are not specifically limited in the present application.

[0110] It should also be understood that the first Net model can also be referred to as a denoising Net model, and the image output by the first Net model can be a linear RGB image or the like, or an image in other formats; the image input in the second Net model can be a linear RGB image, or an image in YUV format or the like, which are not specifically limited in the present application.

[0111] The software system to which the embodiments of the present application are applied is described in detail above.

[0112] It can be understood that Figure 4 An example of software modules included in the electronic device is shown in FIG. 10. More or fewer software modules can also be included in each layer of the software structure of the electronic device. For example, the application layer of the electronic device can also include other application programs, such as information, alarm, weather, stopwatch, compass, timer, flashlight, calendar, Alipay, etc. The present application does not make specific limitations thereto.

[0113] Figure 5 A flowchart of an image processing method 500 provided by the embodiments of the present application is shown in FIG. 11. The method 500 can be executed by an electronic device, and the software structure of the electronic device can be as shown in FIG. 10. Figure 4 The hardware structure of the electronic device can be as shown in FIG. 9. As shown in FIG. 11, the method 500 includes the following steps: Figure 3 Figure 5 The hardware structure of the electronic device can be as shown in FIG. 9. As shown in FIG. 11, the method 500 includes the following steps:

[0114] S10, the camera acquires a first instruction, the first instruction being used to instruct to shoot an image.

[0115] Optionally, the first instruction can be issued when the electronic device receives a down event of a user pressing a shooting button, or the first instruction can also be issued when the electronic device recognizes a shooting instruction.

[0116] Compared with the electronic device issuing the first instruction when recognizing the shooting instruction, the electronic device issuing the first instruction when receiving the down event of the user pressing the shooting button will be earlier in time, that is, the efficiency of the electronic device acquiring the first instruction will be higher. The shooting button can be, for example, a shooting button 101 of the electronic device in the scene 100.

[0117] ​S11, the camera calls the camera access interface to indicate the first instruction to the camera hardware abstraction module.

[0118] S12, the camera hardware abstraction module instructs a decision module in the camera algorithm library to determine the out-picture parameter.

[0119] Optionally, the out-picture parameter includes, but is not limited to, one or more of the following: exposure time, exposure amount, configuration parameters of a camera or an ISP module that need to be called, and the like of a sensor out-picture.

[0120] S13, the decision module determines that the current shooting scene is a dark light scene.

[0121] The decision module can determine whether the current shooting scene is a dark light scene according to parameters such as ambient illuminance and / or ambient brightness. The dark light scene may, for example, be a night scene, a cinema scene, and the like.

[0122] Exemplarily, when the decision module determines that the ambient brightness in the current shooting scene is lower than a preset brightness value, it is determined that the current shooting scene is a dark light scene. That is, in response to the instruction to determine the out-picture parameter, the decision module can obtain parameters such as ambient illuminance. The ambient illuminance and / or ambient brightness can be obtained from the sensor by the camera hardware abstraction module.

[0123] In a possible implementation, in the case where the decision module determines that the current shooting scene is not a dark light scene, for example, the ambient brightness in the current shooting scene is large, the electronic device can obtain the target image according to the method 200 described above.

[0124] S14, the decision module instructs the camera hardware abstraction module to indicate the out-picture parameter.

[0125] In a possible implementation, the decision module can determine the out-picture parameter according to parameters such as ambient illuminance and ambient brightness, that is, in response to the instruction to determine the out-picture parameter, the decision module can obtain parameters such as ambient illuminance.

[0126] S15, the decision module instructs the frame selection module to select frames from the preview queue. In addition, the decision module can also instruct the frame selection module to indicate the number of mid-frame images that need to be selected.

[0127] It should be understood that the preview queue can be understood as a queue for buffering images displayed in the preview picture. The images stored in the preview queue are a plurality of mid-frame images. The plurality of mid-frame images can be images obtained by the electronic device when the user previews the picture.

[0128] S16, the frame selection module selects frames from the preview queue.

[0129] Based on the instruction of the decision module, the frame selection module can select a corresponding number of mid-frame images from the preview queue to obtain the selected mid-frame images.

[0130] It should be understood that the frame selection module can select a mid-frame image with higher image quality, such as higher definition, from a plurality of mid-frame images, and the present application does not make a specific limitation thereto.

[0131] S17, the frame selection module transmits the selected mid-frame image to the post-processing algorithm module, so that the post-processing algorithm module subsequently performs post-processing on the selected mid-frame image.

[0132] The selected mid-frame image can be one or more images, for example, 2 images, 4 images, etc.

[0133] Optionally, the post-processing algorithm module can generate a thumbnail based on the selected mid-frame image, and transmit the thumbnail to the camera through the camera hardware abstraction module calling the camera access interface, so that the camera can call the display screen to display the thumbnail.

[0134] S18, the frame selection module transmits image 1 in the selected mid-frame image to the camera hardware abstraction module, and the image 1 can be part or all of the selected mid-frame image. For example, the image 1 can be one frame of the selected mid-frame image, etc.

[0135] S19, the camera hardware abstraction module transmits the image 1 to the image signal processor through the camera device driver.

[0136] S20, the image signal processor performs image segmentation on the image 1 to obtain a mask image of the sky region in the image 1.

[0137] It should be understood that the mask image can be a binary image. That is, the pixel value of the sky region in the mask image can be 1, and the pixel value of the non-sky region can be 0. For example, as shown in Figure 6 , the one frame of mid-frame image can be as shown in Figure 6 (a). By performing image segmentation on the sky region 601 in the image (a) in Figure 6 , a mask image as shown in Figure 6 (b) is obtained. In the mask image (b), the pixel value of the sky region 602 can be 1, and the pixel value of the region outside the sky region 602 can be 0.

[0138] When the number of images 1 is one frame, the image signal processor can directly obtain the mask image of the sky region in the image 1. When the number of images 1 is multiple frames, the image signal processor can select one frame of mid-frame image from the images 1 for image segmentation to obtain the mask image of the sky region in the one frame of mid-frame image; or the image signal processor can perform image fusion on the images 1 to obtain the mask image of the sky region in the fused image 1; or the image signal processor can obtain the mask image of the sky region in each mid-frame image in the images 1, respectively.

[0139] Further, after S14, the camera hardware abstraction module further performs S21, the camera hardware abstraction module indicates the graph parameters to the camera device driver.

[0140] It should be understood that S21 can be performed in parallel with S15 to S20, or S21 can be performed in parallel with or after S19, which is not limited in the present application.

[0141] S22, the camera device driver drives the sensor to output the graph, and indicates the graph mode to the sensor.

[0142] The graph mode can include but is not limited to one or more of the following: a camera to be called, an exposure amount, an exposure time, a number of long frames to be shot, or a number of short frames to be shot.

[0143] In this way, the sensor can output the graph according to the graph mode to obtain an image sequence. The image sequence can also be referred to as a bayer image. The image sequence can include short frame images and long frame images, and the number of short frame images and long frame images can be one or more.

[0144] S23, the sensor indicates the image sequence to the image signal processor.

[0145] S24, the camera device driver indicates the configuration parameters to the image signal processor. The configuration parameters belong to part of the graph parameters. The configuration parameters can be understood as parameters required by the ISP module for pre-processing of the image sequence.

[0146] It should be understood that S24 can be performed in parallel with S22 and S23, or S24 can be performed before or after S22 and S23, which is not limited in the present application.

[0147] S25, the image signal processor pre-processes the image sequence using the configuration parameters to obtain a pre-processed image sequence.

[0148] Optionally, the pre-processing can include but is not limited to one or more of the following: black level compensation, lens correction, bad pixel correction, or color interpolation. The configuration data includes parameters required for pre-processing.

[0149] In a possible implementation, the sensor indicates an image index of each long frame image and / or each short frame image in the image sequence to the image signal processor. The image index is used to indicate whether the image is a short frame image or a long frame image. For example, when the index of the image is 0, it can represent that the image is a short frame image; and / or when the index of the image is 1, it can represent that the image is a long frame image.

[0150] It should be understood that S15 to S20 and S21 to S25 can be executed in parallel or sequentially, and the present application does not make a specific limitation in this regard.

[0151] In a possible implementation, S18 to S20 are optional content. Illustratively, the image signal processor can also perform image segmentation on the x-frame long frame image in the preprocessed image sequence or the y-frame long frame image in the image sequence to obtain the mask image of the sky region. x and y are integers greater than or equal to 1.

[0152] Compared with the short frame image, the middle frame image and the long frame image have a larger brightness and a higher signal-to-noise ratio. Therefore, the edge of the sky region in the middle frame image and the long frame image can be clearer, so that the electronic device has a higher accuracy in determining the sky region through the long frame image or the middle frame image.

[0153] Compared with the short frame image and the long frame image, the middle frame image can be transmitted to the image signal processor before the image signal processor pre-processes the image sequence, so that the image signal processor can perform image segmentation on the middle frame image to obtain the mask image of the sky region with a higher efficiency.

[0154] S26, the image signal processor indicates the pre-processed image sequence and the mask image to the camera device driver.

[0155] It should be understood that the image signal processor can transmit the pre-processed image sequence and the mask image to the camera device driver simultaneously or sequentially, for example, the image signal processor first transmits the pre-processed image sequence to the camera device driver, and then transmits the mask image to the camera device driver, and the present application does not make a specific limitation in this regard.

[0156] Optionally, the image signal processor also indicates the image index of each long frame image and / or each short frame image in the pre-processed image sequence to the camera device driver.

[0157] S27, the camera device driver indicates the pre-processed image sequence and the mask image to the camera hardware abstraction module.

[0158] It should be understood that the camera device driver can transmit the pre-processed image sequence and the mask image to the camera hardware abstraction module simultaneously or sequentially, and the present application does not make a specific limitation in this regard.

[0159] Optionally, the camera device driver also indicates the image index of each long frame image and / or each short frame image in the pre-processed image sequence to the camera hardware abstraction module.

[0160] S28, the camera hardware abstraction module indicates the pre-processed image sequence and the mask image to the post-processing algorithm module.

[0161] It should be understood that the camera hardware abstraction module can transmit the pre-processed image sequence and the mask image to the post-processing algorithm module simultaneously or in sequence, and the present application does not make a specific limitation thereon.

[0162] Optionally, the camera hardware abstraction module also indicates to the post-processing algorithm module the image index of each long frame image and / or each short frame image in the pre-processed image sequence. In this way, it is convenient for the post-processing algorithm module to identify the long frame image and the short frame image in the pre-processed image sequence, and then to perform different post-processing on the long frame image and the short frame image, respectively.

[0163] S29, the post-processing algorithm module determines that the proportion of the sky region in the mask image is greater than a preset proportion.

[0164] It should be understood that the preset proportion can be a preset value between 0 and 1, such as 0.3, 40%, etc.

[0165] Optionally, the post-processing algorithm module can determine the proportion of the sky region in the mask image in the following manner: the post-processing algorithm module calculates the ratio of the number of pixel points with a pixel value of 1 in the mask image to the total number of pixel points in the mask image, and the ratio is the proportion of the sky region in the mask image.

[0166] S30, the post-processing algorithm module inputs the short frame image in the pre-processed image sequence and the mid frame image selected by the frame selection module into the first Net module, and outputs a linear RGB image A.

[0167] It should be understood that the image format of the short frame image in the pre-processed image sequence and the mid frame image selected by the frame selection module can also be RAW. The linear RGB image A is one frame of image, i.e., the linear RGB image can be one frame of image obtained by image fusion and denoising processing on the short frame image and the mid frame image.

[0168] Wherein, the image fusion can be a process of pixel processing on a reference frame in the mid frame image selected by the frame selection module by using the remaining frames. The remaining frames can be images other than the reference frame in the short frame image in the pre-processed image sequence and the mid frame image selected by the frame selection module.

[0169] It should be noted that the reference frame is the mid frame image, and the brightness thereof is usually lower than that of the long frame image. Exemplarily, the reference frame can be as shown in Figure 7 The long frame image can be as shown in Figure 8 The number of photons obtained in the dark area of the long frame image is more, and the signal-to-noise ratio of the dark area of the long frame image is higher.

[0170] S31, the post-processing algorithm module converts the long frame image into a linear RGB image B. For example, the post-processing algorithm module can convert the long frame image into the linear RGB image B by using a RAW2RGB tool.

[0171] It should be understood that S31 can be performed in parallel with S30, or before or after S30, and the present application does not make a specific limitation thereon.

[0172] S32, the post-processing algorithm module inputs the linear RGB image A, the linear RGB image B and the mask image into a second Net model, and outputs a correction coefficient of a U channel and a correction coefficient of a V channel.

[0173] It should be understood that the first Net model and the second Net model can refer to the description in the foregoing, and will not be described herein again. The first Net model can be the same as the Net model 1.

[0174] The correction coefficient of the U channel includes a correction coefficient of the U channel corresponding to each pixel point in the sky region; for example, the correction coefficient of the U channel corresponding to a first pixel point in the sky region can be a ratio of a difference between a chroma value of the first pixel point in the linear RGB image B and a chroma value of the first pixel point in the linear RGB image A, and the chroma value of the first pixel point in the linear RGB image A, the first pixel point being any pixel point in the sky region.

[0175] The correction coefficient of the V channel includes a correction coefficient of the V channel corresponding to each pixel point in the sky region; for example, the correction coefficient of the V channel corresponding to a first pixel point in the sky region can be a ratio of a difference between a density value of the first pixel point in the linear RGB image B and a density value of the first pixel point in the linear RGB image A, and the density value of the first pixel point in the linear RGB image A, the first pixel point being any pixel point in the sky region.

[0176] Optionally, the second Net model can be obtained by the following training method: obtaining a plurality of sample images, the plurality of sample images including at least one long-frame sample image, at least one short-frame sample image and at least one medium-frame sample image; performing image fusion on the at least one short-frame sample image and the at least one medium-frame sample image to obtain a first sample image; converting the at least one long-frame sample image into a second sample image, the second sample image having the same format as the first sample image; performing image segmentation on a third sample image in the plurality of sample images to obtain a mask sample image of a target region in the third sample image; inputting the second sample image, the first sample image and the mask sample image into an initial model to obtain a target sample image; performing pixel-by-pixel comparison between the target sample image and a preset image to determine a loss function, the image quality of the preset image being better than that of the target sample image; and training the initial model by using the loss function to obtain the second Net model.

[0177] The plurality of sample images and the preset image can be images preset in the electronic device. The process of obtaining the first sample image is similar to the process of the electronic device determining the linear RGB image A. The target region can be a sky region. The process of converting the at least one long-frame sample image into the second sample image is similar to S31. The manner of obtaining the mask sample image is similar to the manner of the electronic device obtaining the mask image of the sky region. The initial model is an untrained second Net model.

[0178] Optionally, the at least one long-frame sample image can include a long-frame sample image with existing blur. In this way, when the correction coefficient is obtained by the second Net model, the influence of the blur on the target display image can be reduced.

[0179] Optionally, in a case where the loss function is lower than the preset loss function, the electronic device can determine that the initial model is trained, that is, the first model is obtained.

[0180] S33, the post-processing algorithm module converts the linear RGB image A into a YUV format image A. For example, the post-processing algorithm module converts the linear RGB image A into the YUV format image A by using an RGB2YUV tool.

[0181] S34, the post-processing algorithm module performs correction processing on a U channel of the YUV format image A by using the correction coefficient of the U channel, and performs correction processing on a V channel of the YUV format image A by using the correction coefficient of the V channel, to obtain a YUV format target image A.

[0182] It should be noted that the correction coefficient of the U channel includes a correction coefficient of the U channel for each pixel point in the sky region in the YUV format image A. The correction coefficient of the V channel includes a correction coefficient of the V channel for each pixel point in the sky region in the YUV format image A. The electronic device performs pixel-by-pixel correction processing on the sky region in the YUV format image A.

[0183] In a possible implementation, the post-processing algorithm module performs correction processing on the U channel of the sky region in the YUV format image A by the following formula: U3=U2+C0xU2, where U3 is a value of the U channel of the sky region in the YUV format image A after correction, U2 is a value of the U channel of the sky region in the YUV format image A before correction, and C0 is the correction coefficient of the U channel.

[0184] In a case where the correction coefficients of the U channels corresponding to a plurality of pixel points in the sky region are the same, the electronic device can simultaneously correct the U channels of the plurality of pixel points.

[0185] In a possible implementation, the post-processing algorithm module corrects the V channel of the sky region in the YUV format image A by the following formula: V3 = V2 + C1 * V2, where V3 is the value of the corrected V channel of the sky region in the YUV format image A, V2 is the value of the uncorrected V channel of the sky region in the YUV format image A, and C1 is the correction coefficient of the V channel.

[0186] It should be understood that in some embodiments, if the post-processing algorithm module determines in S29 that the proportion of the sky region in the mask image is less than or equal to the preset proportion, the post-processing algorithm module can not perform S31, S32, and S34, that is, the YUV format image A is the YUV format target image A.

[0187] In this way, when the proportion of the sky region in the mask image is less than or equal to the preset proportion, the electronic device can not perform the correction processing on the sky region in the image obtained by fusing the medium frame image and the short frame image, which helps to reduce the power consumption of the electronic device.

[0188] S35, the post-processing algorithm module converts the YUV format target image A into a JEPG format target image A.

[0189] S36, the post-processing algorithm module transmits the JEPG format target image A to the camera hardware abstraction module.

[0190] S37, the camera hardware abstraction module calls an interface to transmit the JEPG format target image A to the album.

[0191] S38, the album saves the JEPG format target image A.

[0192] It should be understood that in some possible embodiments, S35 is optional, and the JEPG format target image A in S36 to S38 can be replaced by the YUV format target image A; in another possible embodiment, the post-processing algorithm module can also convert the YUV format target image A into a target image A in other formats in S35, and the JEPG format target image A in S36 to S38 can be replaced by the target image A in other formats. The other formats can be, for example, PNG format, GIF format, etc.

[0193] In some possible implementation manners, after the post-processing algorithm module inputs the mid-frame image and the short-frame image into the first Net model, the first Net model can also output a YUV format image A, which is a frame image, i.e., the YUV format image A is a frame image obtained after fusion processing and noise reduction processing are performed on the mid-frame image and the short-frame image. Correspondingly, in S31, the post-processing algorithm also converts the long-frame image into a YUV format image B. Moreover, the second Net model can determine the correction coefficient of the U channel and the correction coefficient of the V channel by using the YUV format image A, the YUV format image B, and the mask image.

[0194] In some possible implementation manners, after the post-processing algorithm module inputs the mid-frame image and the short-frame image into the first Net model, the first Net model can also output a YUV format image A, which is a frame image, i.e., the YUV format image A is a frame image obtained after fusion processing and noise reduction processing are performed on the mid-frame image and the short-frame image. Correspondingly, in S31, the post-processing algorithm also converts the long-frame image into a YUV format image B. Moreover, the second Net model can determine the correction coefficient of the U channel and the correction coefficient of the V channel by using the YUV format image A, the YUV format image B, and the mask image.

[0195] It should be understood that, in this embodiment, the post-processing algorithm module can correct the RGB image A by using the correction coefficient of the R channel, the correction coefficient of the G channel, and the correction coefficient of the B channel in a manner similar to the implementation manner of S34 described above, and details are referable to the description above, which will not be described herein again.

[0196] Figure 9 A process schematic diagram of an image processing method 900 provided by an embodiment of the present application is shown in FIG. 9. Figure 9 As shown in FIG. 9, the electronic device receives an instruction to capture an image, and obtains two short-frame images, two mid-frame images, and one long-frame image. The electronic device performs image segmentation on one of the two mid-frame images to obtain a sky mask image of a sky region of the one mid-frame image. The sky mask image may, for example, be as shown in (b) of FIG. 5. Figure 6

[0197] When the electronic device determines that the sky region in the sky mask image accounts for more than a preset proportion in the one image, the electronic device converts the one long-frame image into a linear RGB image 3 by using a RAW2RGB tool. The process of obtaining the linear RGB image 3 by the electronic device is similar to the process of obtaining the linear RGB image B in the method 500, and details are referable to the description above, which will not be described herein again.

[0198] ​The electronic device also inputs the 2-frame short-frame image and the 2-frame middle-frame image into a Net model 2, and outputs a linear RGB image 2. The Net model 2 can be the same as the first Net model in the method 500 or the Net model 1 in the method 200. The process of obtaining the linear RGB image 2 by the electronic device is similar to the process of obtaining the linear RGB image A by the first Net model in the method 500, and details are described above and will not be repeated here.

[0199] The electronic device inputs the linear RGB image 2, the linear RGB image 3, and the sky mask image into a Net model 3, and outputs a correction coefficient of a U channel and a correction coefficient of a V channel. The Net model 3 can be the same as the second Net model in the method 500. The correction coefficient of the U channel can also be understood as the correction coefficient of the U channel in the method 500, and the correction coefficient of the V channel can also be understood as the correction coefficient of the V channel in the method 500.

[0200] The electronic device converts the linear RGB image 2 into a YUV format image 2 by using an RGB2YUV tool, and performs color correction on a U channel of the YUV format image 2 by using the correction coefficient of the U channel. The electronic device performs color correction on a V channel of the YUV format image 2 by using the correction coefficient of the V channel, and obtains a YUV format target display image 2. The process of performing color correction on the YUV format image 2 by using the correction coefficient by the electronic device is similar to the implementation of S34 in the method 500, and details are described above and will not be repeated here.

[0201] It should be noted that, in a case where the electronic device determines that the proportion of the sky region in the sky mask image in the one frame image is less than or equal to the preset proportion, the electronic device can not obtain the correction coefficient of the U channel and the correction coefficient of the V channel by using the 1-frame long-frame image. That is, after the electronic device converts the linear RGB image 2 into the YUV format image 2, the YUV format image 2 is the YUV format target display image 2.

[0202] Exemplarily, the target display image 1 obtained by the electronic device through the method 200 can be as shown in FIG. 10A. The target display image 1 can have a large number of color noise points 1001. The target display image 2 obtained by the electronic device through the method 900 can be as shown in FIG. 10B. Compared with the target display image 1, the target display image 2 has fewer color noise points in the sky region. Figure 10 Figure 11

[0203] Figure 12 A flowchart of an image processing method 1200 provided by an embodiment of the present application is shown in FIG. 12A. As shown in FIG. 12B, the method 1200 includes the following steps: Figure 12

[0204] ​​​S1201, in response to an operation of shooting an image, obtaining a plurality of images, the plurality of images comprising at least one long frame image, at least one short frame image, and at least one middle frame image.

[0205] It should be understood that the operation of shooting an image may, for example, be an operation of a user clicking a shooting button 101. The plurality of images may, for example, be the pre-processed image sequence in the method 500 and the middle frame image selected by the frame selection module. The plurality of images can be images in RAW format.

[0206] S1202, performing image fusion on part or all of the plurality of images to obtain a first image.

[0207] Case 1: performing image fusion on part of the plurality of images. S1202 can be implemented by performing image fusion on at least one short frame image and at least one middle frame image to obtain the first image. In this way, the influence of the blur caused by the shaking in the long frame image on the target image can be reduced, which helps to further improve the image quality of the target image.

[0208] It should be understood that the first image may, for example, be the linear RGB image A in the method 500, and the manner in which the electronic device obtains the first image is similar to the manner in which the electronic device obtains the linear RGB image A in the method 500. For details, please refer to the description above, which will not be repeated here.

[0209] Alternatively, the first image can also be an image in other formats, for example, the first image is an image in YUV format, etc.

[0210] Case 2: performing image fusion on all of the plurality of images to obtain the first image.

[0211] It should be understood that the first image may, for example, be the linear RGB image 1 in the method 200, and the manner in which the electronic device obtains the first image is similar to the manner in which the electronic device obtains the linear RGB image 1 in the method 200. For details, please refer to the description above, which will not be repeated here.

[0212] S1203, using at least one long frame image to perform color correction on a target region in the first image to obtain a target image; wherein the target region is a region in the first image where color noise exists.

[0213] The color noise can also be referred to as color noise points, color noise, etc. The region in the first image where the color noise exists can be a region defined according to experience. The target region can be a pre-defined region, for example, a sky region, etc. The target region can also be understood as the dark region described above, etc. The dark region can be a region defined according to experience, for example, a sky region; or, it can also be a region in the first image whose brightness is lower than a fourth threshold value, etc. The fourth threshold value is a pre-determined positive value.

[0214] It should be understood that the target image may be, for example, a target image A in YUV format in the method 500. The electronic device obtains the target image in a manner similar to that in which the electronic device obtains the target image A in the method 500, and reference can be made to the foregoing description, which will not be repeated here.

[0215] The image processing method of the present application has more photons in the target region of the long-frame image, so that the signal-to-noise ratio of the target region in the long-frame image is higher, and the color distribution is more uniform and delicate. The electronic device performs color correction on the target region in the first image by using the long-frame image, which helps to improve the signal-to-noise ratio of the target region in the first image, so that the color distribution of the target region in the first image is more uniform and delicate, and the color noise of the target region in the first image is reduced, thereby improving the image quality of the target image.

[0216] As an optional embodiment, the method 1200 further includes: performing image segmentation on a second image in the plurality of images to obtain a mask image corresponding to the target region in the second image; and S1203 can be implemented by: performing color correction on the target region in the first image by using at least one long-frame image and the mask image.

[0217] In this way, by using the mask image, the electronic device can identify the target region, thereby facilitating color correction of the target region in the first image.

[0218] The second image can be any one of the plurality of images. For example, the second image can be as shown in (a) of FIG. 12A, and the mask image can be as shown in (b) of FIG. 12A. Figure 6 The second image can be any one of the plurality of images. For example, the second image can be as shown in (a) of FIG. 12A, and the mask image can be as shown in (b) of FIG. 12A. Figure 6 The pixel value of the pixel point in the target region in the mask image can be 1, and the pixel value of the pixel point in the remaining region can be 0.

[0219] The second image can be a medium-frame image, a long-frame image, or a short-frame image, and the details are as follows.

[0220] In a first possible implementation, the second image belongs to one of the at least one medium-frame image.

[0221] In this way, compared with the short-frame image, the medium-frame image has a higher signal-to-noise ratio, i.e., the edges of the target region in the medium-frame image can be clearer, so that the accuracy of the target region determined by the medium-frame image is higher. In addition, after the image signal processor receives part or all of the selected medium-frame images from the frame selection module, the image signal processor can determine the mask image. This makes the image signal processor able to determine the mask image before obtaining the image sequence, which helps to improve the efficiency of obtaining the mask image.

[0222] It should be understood that the implementation of the embodiment is similar to the implementation of the electronic device obtaining the mask image of the sky region in the method 500, where the second image can be understood as a frame image in a frame in the method 500, and the description can be referred to the description above, which will not be repeated here.

[0223] In a second possible implementation, the second image belongs to a frame in the at least one frame long frame image.

[0224] In combination with the method 500, after the image signal processor obtains the preprocessed image sequence, the image segmentation can be performed on a frame long frame image in the preprocessed image sequence to obtain the mask image of the target region in the frame long frame image. The frame long frame image is the second image. In addition, the selected frame module can not need to transmit part or all of the selected frame image to the image signal processor, which helps to reduce the power consumption of the electronic device.

[0225] It should be understood that the implementation of the second image belonging to a frame in the at least one frame short frame image is similar to the second possible implementation described above, which will not be repeated here.

[0226] In addition, the electronic device performs color correction on the target region in the first image in the following manner.

[0227] In a possible implementation, the color correction on the target region in the first image is performed by using the at least one frame long frame image and the mask image in the following manner: obtaining a correction coefficient of the target region in the first image by using the at least one frame long frame image, the mask image, and the first image; and performing color correction on the target region in the first image by using the correction coefficient.

[0228] The correction coefficient can correspond to a coefficient corresponding to the format of the first image. For example, when the first image is an image in RGB format, the correction coefficient is a coefficient for correcting at least one of the R channel, the G channel, or the B channel of the target region in the first image; and when the first image is an image in YUV format, the correction coefficient is a coefficient for correcting the U channel and / or the V channel of the target region in the first image.

[0229] For example, the correction coefficient includes a first coefficient and a second coefficient, the first coefficient is used to correct the chroma value of the target region in the first image, and the second coefficient is used to correct the density value of the target region in the first image; and the color correction on the target region in the first image by using the correction coefficient includes: correcting the chroma value of the target region in the first image by using the first coefficient to obtain a corrected chroma value, and the corrected chroma value U1 satisfies the following formula: U1=U0+C U ×U0, where U0 is the chroma value of the target region before correction, and C Uis a first coefficient; and correcting the chroma value of the target region in the first image by using a second coefficient to obtain a corrected chroma value, the corrected chroma value V1 satisfies the following formula: V1 = V0 + C V ×V0, wherein V0 is the chroma value of the target region before correction, C V is the second coefficient.

[0230] Since the chroma value (U in YUV) and the chroma value (V in YUV) can describe the image color and saturation and specify the color of a pixel. Therefore, by correcting the chroma value and the chroma value of the target region in the first image, the image color and saturation of the target region in the first image can be corrected, and the color distribution of the target region in the first image is more uniform and delicate, thereby reducing the color noise of the target region in the first image.

[0231] It should be understood that the electronic device corrects the color by using the first coefficient and the second coefficient in a manner similar to the implementation of S34 in the method 500, and the description can be referred to in the foregoing, which will not be described here again.

[0232] In a possible implementation, the correction coefficient of the target region is obtained by using the at least one long frame image, the mask image, and the first image, which can be implemented in the following manner: converting the at least one long frame image into a third image, the third image having the same format as the first image; and obtaining the correction coefficient of the target region by using the third image, the mask image, and the first image.

[0233] In this way, since the third image and the first image have the same format, the electronic device can more easily compare the third image and the first image pixel by pixel to determine the correction coefficient of each channel of the target region in the first image. The correction coefficient can be a coefficient that makes the target region of the first image closer to the target region of the third image.

[0234] In case one, the third image and the first image are linear RGB images, and the correction coefficient includes: a correction coefficient of an R channel of the target region in the first image; a correction coefficient of a G channel of the target region in the first image; and a correction coefficient of a B channel of the target region in the first image. In this way, the electronic device can correct the R channel, the G channel, and the B channel of the first image respectively. The correction coefficient of the R channel can include a correction coefficient of the R channel corresponding to each pixel point in the target region; the correction coefficient of the G channel can include a correction coefficient of the G channel corresponding to each pixel point in the target region; and the correction coefficient of the B channel can include a correction coefficient of the B channel corresponding to each pixel point in the target region.

[0235] In case one, the electronic device performs image fusion on the at least one image and converts the image into a linear RGB image.

[0236] In case two, the third image and the first image are linear RGB images, and the method 1200 further includes: converting the first image into a luminance chrominance density YUV format to obtain a fourth image; and performing color correction on the target region in the first image corresponds to performing color correction on the target region in the fourth image. The correction coefficient includes a first coefficient and a second coefficient.

[0237] The electronic device can correct the image color and saturation of the target region in the fourth image by correcting the U channel (chrominance value) and the V channel (density value) of the target region, so that the color distribution of the target region is more uniform and delicate, and the color noise of the target region is reduced.

[0238] It should be understood that the implementation of case two is similar to the implementation of the method 500, and the fourth image may, for example, be the image A in the YUV format in S33. For details, refer to the description above, which will not be repeated here.

[0239] In case three, the third image and the first image are YUV format images, and the correction coefficient includes a first coefficient and a second coefficient. That is, the electronic device can determine the first coefficient and the second coefficient by using the third image in the YUV format, the first image in the YUV format, and the mask image.

[0240] It should be understood that the correction manner of each channel is similar to the correction manner of the U channel or the V channel in the method 500. For details, refer to the description above, which will not be repeated here.

[0241] It should be further noted that the above cases one to three are only examples, and the format of the first image and / or the third image can also be other formats. For brevity, they will not be listed one by one here.

[0242] In a possible implementation, the image segmentation on the second image in the plurality of images can be implemented in the following manner: in a case where the ambient brightness is less than or equal to the preset brightness, performing image segmentation on the second image.

[0243] Since the number of photons obtained by the medium frame image and the short frame image is also large when the ambient brightness is large, that is, the current shooting scene is not a dark light scene, the color noise in the target region of the target image is small. Therefore, the electronic device can obtain the target image by using the long frame image to perform color correction on the target region of the first image in the dark light scene; in the non-dark light scene, the electronic device can not obtain the mask image, and does not need to use the long frame image to perform color correction on the target region of the first image, that is, the first image or the image obtained by performing format conversion on the first image is the target image, so that the power consumption of the electronic device is small.

[0244] In addition, the image segmentation on the second image in the plurality of images can also be implemented in the following manner: in the case that the shooting scene is judged as the HDR scene and / or the dark-light scene, the image segmentation is performed on the second image.

[0245] The dark-light scene and the HDR scene can be judged according to environmental brightness, environmental illumination and the like.

[0246] In a possible implementation, the color correction on the target region in the first image by using the at least one long-frame image and the mask image can be implemented in the following manner: in the case that the proportion of the target region in the mask image is greater than or equal to a preset threshold, the color correction is performed on the target region in the first image by using the at least one long-frame image and the mask image.

[0247] It should be understood that the implementation of this embodiment is similar to the implementation of S29 in the method 500, and the preset threshold can be, for example, the preset proportion in S29, which can be referred to the description above and will not be repeated here.

[0248] It should be noted that the correction coefficient in the embodiments of the present application can also be referred to as an offset coefficient, and the color correction can also be referred to as offset processing, and the color noise can also be referred to as color noise, and the present application does not make specific limitations.

[0249] It should be noted that the module names involved in the embodiments of the present application can be defined as other names, as long as the functions of the modules can be realized, and the names of the modules are not specifically limited.

[0250] It should be noted that the user information (including but not limited to user equipment information, user personal information and the like) and data (including but not limited to data for analysis, stored data, displayed data and the like) involved in the embodiments of the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for the user to select authorization or refusal.

[0251] The image processing method of the embodiments of the present application has been described above, and the device provided by the embodiments of the present application for executing the above method will be described below. Those skilled in the art can understand that the method and the device can be combined and referred to each other, and the related device provided by the embodiments of the present application can execute the steps in the above list sorting method.

[0252] Figure 13A schematic block diagram of an image processing apparatus 1300 is provided in embodiments of the present application. The apparatus 1300 includes a processor 1301, a communication interface 1302 and a memory 1303. The processor 1301, the communication interface 1302 and the memory 1303 communicate with each other through internal connection paths. The memory 1303 is configured to store instructions, and the processor 1301 is configured to execute the instructions stored in the memory 1303. The communication interface 1302 can be configured to send signals to other apparatuses (for example, the processor 1301 or the touch screen of an electronic device) and receive signals from other apparatuses (for example, the memory 1303). For example, the communication interface 1302 reads the instructions stored in the memory 1303 and sends the instructions to the processor 1301.

[0253] It should be understood that the apparatus 1300 can be embodied as an electronic device in the above-described embodiments, and can be configured to perform each step and / or process corresponding to the electronic device in the above-described method embodiments. Alternatively, the memory 1303 can include a read-only memory and a random access memory, and provide instructions and data for the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store device type information. The processor 1301 can be configured to execute the instructions stored in the memory, and when the processor 1301 executes the instructions stored in the memory, the processor 1301 is configured to perform each step and / or process of the above-described method embodiments.

[0254] It should be understood that in embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0255] In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is in the memory, and the processor executes the instructions in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0256] The image processing method provided by the embodiments of the present application can be applied to an electronic device with a communication function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the related description above, which will not be repeated here.

[0257] The embodiments of the present application provide a terminal device, which includes: a processor and a memory; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory, so that the terminal device executes the above method.

[0258] The embodiments of the present application provide a chip. The chip includes a processor, which is configured to invoke a computer program in a memory to execute the technical solutions in the above embodiments. The implementation principle and technical effects are similar to those of the above related embodiments, which will not be repeated here.

[0259] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by the processor to implement the above method. The method described in the above embodiments can be implemented by software, hardware, firmware or any combination thereof, in whole or in part. If implemented in software, the functions can be stored as one or more instructions or codes on a computer readable medium or transmitted on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can also include any medium that can carry computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0260] In a possible implementation, the computer readable medium can include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is targeted at carrying or storing desired program codes in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately called a computer readable medium. For example, if software is transmitted from a website, server or other remote source using a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of medium. As used herein, magnetic disks and optical disks include compact disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, in which magnetic disks usually reproduce data magnetically, and optical disks reproduce data optically with laser. The above combinations should also be included in the scope of computer readable medium.

[0261] The embodiment of the present application provides a computer program product, which comprises a computer program. When the computer program is executed, the computer program causes the computer to execute the above method.

[0262] The embodiment of the present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or blocks.

[0263] The above detailed description of the embodiments of the present application further explains the purposes, technical solutions and beneficial effects of the present application. It should be understood that the above is only a specific embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: in response to an operation for shooting an image, acquiring a plurality of frames of images, the plurality of frames of images comprising at least one frame of long frame images, at least one frame of short frame images, and at least one frame of medium frame images; performing image fusion on the at least one frame of short frame images and the at least one frame of medium frame images in the plurality of frames of images to obtain a first image; converting the at least one frame of long frame images into a third image, the third image having the same format as the first image; and using the third image, a mask image, and the first image to calculate a pixel-level correction coefficient corresponding to the format of the first image; using the correction coefficient to perform color correction on a target region in the first image to obtain a target image; wherein the target region is a region in the first image in which color noise exists, the correction coefficient corresponds to the format of the first image, and the mask image is an image corresponding to the target region in a second image in the plurality of frames of images obtained by performing image segmentation on the second image.

2. The method of claim 1, wherein, The correction coefficient comprises a first coefficient and a second coefficient, the first coefficient is used to correct a chroma value of the target region in the first image, and the second coefficient is used to correct a density value of the target region in the first image. The using the correction coefficient to perform color correction on the target region in the first image comprises: using the first coefficient to correct the chroma value of the target region in the first image to obtain a corrected chroma value, the corrected chroma value U1 satisfying the following formula: U1 = U0 + C U x U0, wherein U0 is the chrominance value of the target region before correction, C U is the first coefficient; using the second coefficient to correct the density value of the target region in the first image to obtain a corrected density value, the corrected density value V1 satisfying the following formula: V1 = V0 + C V x V0, wherein V0 is the concentration value of the target region before correction, C V is the second coefficient.

3. The method according to claim 1 or 2, characterized in that, The second image belongs to one of the at least one frame of medium frame images.

4. The method of claim 3, wherein, The performing image segmentation on the second image in the plurality of frames of images comprises: in a case where it is determined that the ambient brightness is less than or equal to a preset brightness, performing image segmentation on the second image.

5. The method according to any one of claims 1-2, 4, characterized in that, The method further comprises: converting the first image into a luminance-chrominance-density YUV format to obtain a fourth image; The performing color correction on the target region in the first image comprises: performing color correction on the target region in the fourth image.

6. The method according to any one of claims 1-2, 4, characterized in that, The using the at least one frame of long frame images and the mask image to perform color correction on the target region in the first image comprises: in a case where a proportion of the target region in the mask image is greater than or equal to a preset threshold, using the at least one frame of long frame images and the mask image to perform color correction on the target region in the first image.

7. An electronic device, comprising: The electronic device comprises one or more processors and a memory; the memory is coupled to the one or more processors, the memory is configured to store computer program code, the computer program code comprises computer instructions, and the one or more processors invoke the computer instructions to enable the electronic device to perform the method according to any one of claims 1-6.

8. A chip system, characterized by The chip system is applied to an electronic device, and the chip system comprises one or more processors configured to invoke computer instructions to cause the electronic device to perform the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises computer instructions configured to cause an electronic device to perform the method of any one of claims 1-6 when the computer instructions are run on the electronic device.

10. A computer program product, characterised in that, The computer program product comprises computer program code configured to cause an electronic device to perform the method of any one of claims 1-6 when the computer program code is run on the electronic device.

Images