Image processing method and device, electronic equipment and computer readable storage medium

Abstract

The application relates to an image processing method and device, electronic equipment, a storage medium and a computer program product. The method comprises: performing motion region identification on at least two adjacent images taken by a camera to determine the motion regions of at least one group of moving objects; the at least two adjacent images comprise a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold; for each group of moving objects, first pixels with pixel values greater than a preset pixel threshold in the motion region of the first image are determined, and a reference region is determined from each motion region of the moving objects based on statistical parameters of the first pixels in the motion region of the first image; the at least two adjacent images are fused to obtain a target image; and the motion region of each group of moving objects is fused based on the corresponding reference region. The method can improve the image quality and definition of the fused image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image technology, and in particular to an image processing method and device, electronic equipment and a computer readable storage medium. BACKGROUND

[0002] Exposure fusion is a technique for generating high dynamic range (HDR) images by merging multiple images with different exposure levels to fuse their brightness and detail information to produce an image with a wider range of brightness.

[0003] Traditional image processing methods usually use the same camera settings to capture a series of images with different exposure levels, usually including low exposure, standard exposure and high exposure images, and then fuse multiple images to obtain a high dynamic range image. However, due to the presence of moving objects, a reference exposure image needs to be specified for exposure fusion when capturing images.

[0004] However, the reference image specified by the traditional image processing method is usually fixed, and there is a problem of low signal-to-noise ratio of the fused image, which leads to a decrease in overall image quality. SUMMARY

[0005] The embodiments of the present application provide an image processing method, device, electronic equipment, computer readable storage medium and computer program product, which can more accurately determine the reference region of the motion region for fusion, improve the signal-to-noise ratio of the fused image, and further improve the image quality and clarity of the fused image.

[0006] In a first aspect, the present application provides an image processing method. The method comprises:

[0007] performing motion region identification on at least two adjacent images captured to determine the motion region of at least one group of moving objects; the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and the motion region of each group of moving objects includes the motion region of the moving object in each image;

[0008] for each group of moving objects, determining a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determining a reference region from each motion region of the moving object based on the statistical parameters of each first pixel in the motion region of the first image;

[0009] fuse the at least two adjacent images to obtain a target image; the motion region of each group of moving objects is fused based on the corresponding reference region.

[0010] In a second aspect, the present application provides an image processing apparatus. The apparatus comprises:

[0011] a motion region identification module configured to identify motion regions of at least one group of moving objects from at least two adjacent images, wherein the at least two adjacent images comprise a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and each motion region of the at least one group of moving objects comprises a motion region of the moving object in each image;

[0012] a reference region determination module configured to determine, for each motion region of the at least one group of moving objects, a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image, and determine a reference region from each motion region of the at least one group of moving objects based on statistical parameters of the first pixel in the motion region of the first image;

[0013] a fusion module configured to fuse the at least two adjacent images to obtain a target image, wherein each motion region of the at least one group of moving objects is fused based on the corresponding reference region.

[0014] In a third aspect, the present application provides an electronic device. The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the following steps:

[0015] identify motion regions of at least one group of moving objects from at least two adjacent images, wherein the at least two adjacent images comprise a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and each motion region of the at least one group of moving objects comprises a motion region of the moving object in each image;

[0016] determine, for each motion region of the at least one group of moving objects, a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image, and determine a reference region from each motion region of the at least one group of moving objects based on statistical parameters of the first pixel in the motion region of the first image;

[0017] fuse the at least two adjacent images to obtain a target image, wherein each motion region of the at least one group of moving objects is fused based on the corresponding reference region.

[0018] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0019] performing motion region identification on the at least two adjacent images to determine motion regions of at least one group of motion objects; the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and the motion region of each group of motion objects includes a motion region of the motion object in each image;

[0020] for each motion region of each group of motion objects, determining a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image, and determining a reference region from each motion region of the motion object based on statistical parameters of each first pixel in the motion region of the first image;

[0021] fusing the at least two adjacent images to obtain a target image; the motion region of each group of motion objects is fused based on the corresponding reference region.

[0022] In a fifth aspect, the present application further provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the following steps:

[0023] performing motion region identification on the at least two adjacent images to determine motion regions of at least one group of motion objects; the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and the motion region of each group of motion objects includes a motion region of the motion object in each image;

[0024] for each motion region of each group of motion objects, determining a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image, and determining a reference region from each motion region of the motion object based on statistical parameters of each first pixel in the motion region of the first image;

[0025] fusing the at least two adjacent images to obtain a target image; the motion region of each group of motion objects is fused based on the corresponding reference region.

[0026] The aforementioned image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product involve an electronic device performing motion region recognition on at least two adjacent images to determine at least one group of motion regions for moving objects. The at least two adjacent images include a first image with exposure parameters greater than a preset exposure threshold and a second image with exposure parameters less than the preset exposure threshold. The motion region of each group of moving objects includes the motion region of the moving object in each image. For each group of motion regions, a first pixel with a pixel value greater than a preset pixel threshold is determined in the motion region of the first image. Based on the statistical parameters of each first pixel in the motion region of the first image, a reference region for that group of moving objects can be accurately determined from each motion region of the moving object. Then, the motion regions of each group of moving objects are fused with the corresponding reference region as a reference, thereby obtaining a target image with a higher signal-to-noise ratio, higher image quality, and higher clarity. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of an image processing method in one embodiment;

[0029] Figure 2 This is a schematic diagram illustrating the clustering process of the motion mask region in one embodiment;

[0030] Figure 3 This is a structural block diagram of an image processing device in one embodiment;

[0031] Figure 4 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] In one embodiment, such as Figure 1As shown, an image processing method is provided, and the embodiment is exemplified by applying the method to an electronic device, which can be a terminal or a server. It can be understood that the method can also be applied to a system including a terminal and a server and implemented through the interaction of the terminal and the server. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, a smart car, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0034] In the embodiment, the image processing method includes the following steps:

[0035] In step S102, motion region identification is performed on the photographed at least two adjacent images to determine the motion region of at least one group of motion objects. The at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold. The motion region of each group of motion objects includes the motion region of the motion object in each image.

[0036] The motion object can be a person, an animal, an object, or other objects, etc. The exposure parameter can include exposure time, aperture, etc. The preset exposure threshold can be set as needed.

[0037] It can be understood that the greater the exposure parameter, the greater the brightness of the image. The exposure parameter of the first image is greater than the preset exposure threshold, that is, the first image is a long-exposure image. The exposure parameter of the second image is less than the preset exposure threshold, that is, the second image is a short-exposure image. The number of first images can be one or more, and the number of second images can also be one or more.

[0038] The motion region of each group of motion objects includes the motion region of the same motion object in each image.

[0039] Optionally, the electronic device includes a camera module; the at least two adjacent images are obtained by photographing through the camera module; registration is performed on the at least two adjacent images to obtain at least two registered adjacent images; and motion region identification is performed on the at least two registered adjacent images to determine the motion region of at least one group of motion objects.

[0040] The electronic device performs global registration on the at least two adjacent images. It can be understood that global registration can ensure that the at least two registered adjacent images have local motion, and the parts of global motion are roughly aligned.

[0041] In other optional embodiments, the electronic device can also locally register the at least two adjacent images.

[0042] In step S104, for each motion region of each group of motion objects, a first pixel having a pixel value greater than a preset pixel threshold value in the motion region of the first image is determined, and a reference region is determined from each motion region of the motion objects based on a statistical parameter of each first pixel in the motion region of the first image.

[0043] The preset pixel threshold value can be set as needed. If there is a first pixel having a pixel value greater than the preset pixel threshold value in the motion region of the first image, it indicates that the brightness of the first pixel is high, and the first pixel is an overexposure point. If the pixel value in the motion region of the first image is less than or equal to the preset pixel threshold value, it indicates that the brightness of the pixel is low.

[0044] The statistical parameter is a parameter obtained by statistics, and is used to represent the overexposure of the motion region of the first image. The greater the statistical parameter, the greater the proportion of overexposure of the motion region of the first image.

[0045] Optionally, the statistical parameter can be the number of the first pixel in the motion region of the first image, can be the area of the first pixel in the motion region of the first image, can be the proportion of the number of the first pixel to the total number of pixels in the motion region of the first image, and the like, and is not limited thereto.

[0046] The reference region is a region used as a reference when the motion regions of the groups of motion objects are fused.

[0047] Optionally, for each motion region of each group of motion objects, the pixel value of each pixel in the motion region of the motion object of each first image is detected, and a first pixel having a pixel value greater than a preset pixel threshold value in the motion region of the motion object of each first image is determined. For each motion region of the motion object of each first image, each first pixel is counted to obtain a statistical parameter. Based on the statistical parameter of each first pixel in the motion region of the first image, a reference region is determined from each motion region of the motion object.

[0048] Optionally, for each motion region of each group of motion objects, if the statistical parameter in the motion region of each first image is greater than a preset statistical threshold value, the motion region of the motion object in the second image is determined as the reference region. If there is a statistical parameter in the motion region of the first image that is less than or equal to the preset statistical threshold value, the motion region of the motion object in the first image is determined as the reference region.

[0049] In step S106, at least two adjacent images are fused to obtain a target image. Each motion region of each group of motion objects is fused based on the corresponding reference region.

[0050] Optionally, the motion regions of each group of motion objects are fused with the corresponding reference region as a reference to obtain a target motion region; the non-motion regions in the at least two adjacent images are fused to obtain a target non-motion region; and the target motion region and the target non-motion region are spliced to obtain a target image.

[0051] The target image can be a high dynamic range image (HDR image).

[0052] Optionally, the electronic device forms a reference image by taking the reference region of each group of motion objects and the non-motion region as a reference; and fuses the at least two adjacent images with the reference image as a reference to obtain a target image.

[0053] In the image processing method, the electronic device performs motion region identification on the at least two adjacent images to determine the motion regions of at least one group of motion objects, the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and the motion regions of each group of motion objects include the motion regions of the motion objects in each image; for the motion regions of each group of motion objects, a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image is determined, and a reference region of the group of motion objects can be accurately determined from the motion regions of the motion objects based on statistical parameters of each first pixel in the motion region of the first image; then, the motion regions of each group of motion objects are fused with the corresponding reference region as a reference, and a target image with higher signal-to-noise ratio, higher image quality and higher definition can be obtained.

[0054] By using the image processing method, the reference region is more accurately determined from the motion regions of the motion objects based on the statistical parameters of each first pixel in the motion region of the first image in combination with the motion regions, which can minimize the image quality degradation caused by using a dark frame reference, ensure the image quality to some extent, and suppress ghosting at the same time.

[0055] In one embodiment, the motion region identification on the at least two adjacent images to determine the motion regions of at least one group of motion objects includes: performing motion pixel identification on the at least two adjacent images to determine each motion mask region in each image; and performing clustering processing on each motion mask region in each image to obtain the motion regions of at least one group of motion objects.

[0056] The motion mask region (MASK) is used to shield the processed image (all or part) with selected images, graphics or objects to control the image processing region or process. At least two adjacent images can be RAW images. The RAW image is a linear threshold image, which is the original image data collected by the image sensor. The value of the RAW image itself is proportional to the light intensity, so it is linear.

[0057] Optionally, the electronic device uses a preset motion pixel detection algorithm to identify the motion pixels in the at least two adjacent images, and determines the motion pixels in each image. The region where each motion pixel is located is taken as the motion mask region.

[0058] Optionally, the motion pixels in the at least two adjacent images are identified, and each motion mask region in each image is determined, including: identifying the motion pixels in the at least two adjacent images, and determining the pixels where each motion object is located in each image; and determining each motion mask region in each image based on the pixels where each motion object is located.

[0059] The motion pixel refers to the pixel where the motion object is located in the image.

[0060] Optionally, the electronic device detects the image difference (temporal diff) between the pixels at the corresponding positions of the at least two adjacent images; if the image difference meets a preset image difference condition, it indicates that the pixel is the pixel where the motion object is located; and if the image difference does not meet the preset image difference condition, it indicates that the pixel is a non-motion region pixel.

[0061] Optionally, the preset image difference condition can be temporal diff>σ*c; where c is a coefficient for judging the motion model, and σ is a noise standard deviation obtained from a measured noise model. The motion model and the noise model can be obtained by pre-training.

[0062] Optionally, the electronic device uses a noise model to estimate the noise level of the at least two adjacent images, determines the non-noise pixels, identifies the motion pixels in the non-noise pixels, and determines the pixels where the motion objects are located in each image.

[0063] It can be understood that the motion image differences of noise pixels and non-noise pixels are often not at the same level. The noise can be judged according to a preset noise judgment condition, such as using normal distribution. The probability of the motion image difference of the non-noise pixel being within 3σ is very low, so the threshold c can be set to 3 to distinguish the noise pixels and the non-noise pixels.

[0064] It is understandable that at least two adjacent images are RAW images, which are linear threshold images. The x-gain method can be used to align the brightness of at least two adjacent images, and then motion region identification can be performed on these aligned images. Here, gain is the exposure ratio. Motion region identification assumes that the brightness of the images is roughly the same, which can be achieved by multiplying by the exposure ratio for brightness alignment.

[0065] Optionally, after the electronic device determines the pixel containing the moving object in each image, it performs a smoothing operation on the pixels containing each moving object to obtain each motion mask region in each image. The smoothing operation may include operations such as smoothing, dilation, and erosion.

[0066] Optionally, the electronic device performs an erosion operation on each motion mask region to obtain eroded motion mask regions; the eroded motion mask regions are then clustered to obtain at least one set of motion regions for the moving objects. It is understood that by performing the erosion operation on each motion mask region, the electronic device can remove scattered points that resemble noise.

[0067] Optionally, clustering is performed on each motion mask region in each image to obtain at least one set of motion regions of the moving object, including: if adjacent motion mask regions in each image are connected, then the adjacent motion mask regions are merged into the motion region of the moving object; if adjacent motion mask regions in each image are not connected, then each motion mask region is used as the motion region of the moving object.

[0068] Optionally, the electronic device determines whether adjacent motion mask regions in each image are connected and marks each motion mask region; connected motion mask regions are marked with the same marking signal; motion mask regions with the same marking signal are connected to form the motion region of a moving object.

[0069] It is understandable that marking different motion mask regions that are not connected to each other with different markers can ensure the differentiation of different moving objects.

[0070] Optionally, in each image, if the distance between adjacent motion mask regions is less than a preset distance threshold, the adjacent motion mask regions are determined to be connected; if the distance between adjacent motion mask regions is greater than or equal to the preset distance threshold, the adjacent motion mask regions are determined to be disconnected. The preset distance threshold can be set as needed.

[0071] like Figure 2 As shown, the electronic device performs clustering processing on each motion mask region in the image, which can obtain three groups of motion regions of moving objects.

[0072] The distance between the adjacent motion mask regions can be the distance between the center positions of the adjacent motion mask regions, the distance between the gravity center positions of the adjacent motion mask regions, or the distance between other positions of the adjacent motion mask regions, which is not limited herein.

[0073] In this embodiment, the electronic device performs motion pixel recognition on the photographed at least two adjacent images, determines the motion mask regions in each image, and performs clustering processing on the motion mask regions in each image, so that the motion regions of at least one motion object can be accurately obtained.

[0074] Further, the electronic device performs motion pixel recognition on the image, so that the motion mask regions in the image can be accurately determined. The electronic device determines the motion regions of the motion object by judging whether the adjacent motion mask regions are connected.

[0075] In one embodiment, the reference region is determined from the motion region of the motion object based on the statistical parameter of each first pixel in the motion region of the first image, including: if the statistical parameter of each first pixel in the motion region of the first image is less than or equal to a preset statistical threshold, the motion region of the motion object in the first image is determined as the reference region; if the statistical parameter of each first pixel in the motion region of the first image is greater than the preset statistical threshold, the motion region of the motion object in the second image is determined as the reference region.

[0076] It can be understood that if the statistical parameter of each first pixel in the motion region of the first image is less than or equal to the preset statistical threshold, it indicates that the motion region of the first image is not overexposed, and the motion region of the motion object in the first image with the larger exposure parameter is determined as the reference region; if the statistical parameter of each first pixel in the motion region of the first image is greater than the preset statistical threshold, it indicates that the motion region of the first image is overexposed, and the motion region of the motion object in the second image with the smaller exposure parameter is determined as the reference region.

[0077] In this embodiment, the electronic device can determine whether the motion region of the first image is overexposed based on the statistical parameter in the motion region of the first image, so that the reference region can be more accurately determined; under the premise of no image ghosting and missing dynamic, the motion region in the first image with the larger exposure parameter is used as the reference region as much as possible, so that the signal-to-noise ratio of the fused target image can be improved.

[0078] In one embodiment, fusing at least two adjacent images to obtain a target image comprises: determining a first weight of each pixel in each image based on brightness of each pixel in each image; adjusting the first weight of each pixel in each motion region of each motion region of each group of motion objects based on a corresponding reference region to obtain a target weight of each pixel in each motion region; generating a weight map corresponding to each image based on the target weight of each pixel in the motion region of each image and the first weight of each pixel in the non-motion region; and fusing the at least two adjacent images based on the weight map of each image to obtain the target image.

[0079] The first weight is the first calculated weight and is the weight (fusion weight) of the non-motion region for fusion.

[0080] Optionally, the electronic device detects a pixel value of each pixel in each image, and takes the pixel value as the brightness of the pixel; and inputs the brightness of each pixel into a preset first weight formula to determine the first weight of the pixel.

[0081] The first weight can be designed in a Gaussian function manner, for example:

[0082]

[0083] wherein weight fuslon is the first weight of the pixel, σ is a pre-set parameter and is irrelevant to the noise itself, x is the brightness of the pixel, mid is a pre-set optimal exposure brightness, and e is an Exp function (exponent).

[0084] Optionally, adjusting the first weight of each pixel in each motion region based on a corresponding reference region to obtain a target weight of each pixel in each motion region comprises: determining a pixel difference between a second pixel in the reference region and a corresponding third pixel in the motion region other than the reference region for each motion region of each group of motion objects, and determining a second weight of the third pixel based on the pixel difference; adjusting the first weight based on the second weight of the third pixel to obtain a target weight of the third pixel; and adjusting the first weight of the second pixel based on the target weight of the third pixel to obtain a target weight of the second pixel.

[0085] For each motion region of each group of motion objects, the pixel in the reference region is the second pixel, and the pixel in the motion region other than the reference region is the third pixel; a pixel difference between the second pixel and the corresponding third pixel is determined; and a second weight of the third pixel is obtained by inputting the pixel difference into a preset second weight formula.

[0086] The second weight formula is as follows:

[0087]

[0088] wherein, weight deghost is the second weight of the third pixel, e is Exp function (exponent), diff is the pixel difference between the second pixel and the corresponding third pixel after the brightness alignment, offse and σ are preset parameters, which are irrelevant to the noise itself, and both need to be well adjusted in advance to obtain good ghosting removal effect.

[0089] Optionally, the electronic device performs normalization processing on the first weight of the pixels at the corresponding positions in the images after determining the first weight of each pixel in each image based on the brightness of each pixel in each image; and adjusts the first weight of each pixel in each motion region of each group of motion objects based on the corresponding reference region, to obtain the second weight of each pixel in each motion region. Wherein, the sum of the normalized first weights of the pixels at the corresponding positions in the images is 1.

[0090] Optionally, the electronic device multiplies the first weight of the third pixel by the second weight of the third pixel to obtain the target weight of the third pixel.

[0091] It can be understood that the sum of the target weight of the second pixel and the target weight of the third pixel is 1; therefore, the electronic device performs difference processing on 1 and the target weight of the third pixel to obtain the target weight of the second pixel.

[0092] For example, for the reference region determined by the motion region of the motion object in the first image, the first weight of the third pixel in the motion region of the motion object in the second image is calculated as w2, and the second weight deghost_weight, then the target weight of the third pixel is w2*deghost_weight, and the target weight of the second pixel in the reference region in the first image is w1=1-w2*deghost_weight.

[0093] Similarly, for the reference region determined by the motion region of the motion object in the second image, the first weight of the third pixel in the motion region of the motion object in the first image is calculated as w1, and the second weight deghost_weight, then the target weight of the third pixel is w1*deghost_weight, and the target weight of the second pixel in the reference region in the second image is w2=1-w1*deghost_weight.

[0094] Optionally, for the non-motion region in the image, the first weight is maintained unchanged.

[0095] It can be understood that the electronic device obtains the first weight of each pixel in each non-motion region and the target weight of each pixel in the motion region in each image, and generates a weight map corresponding to each image; each weight in the weight map is the first weight or the target weight of the corresponding pixel in the image.

[0096] Optionally, the electronic device multiplies the pixel value of each pixel in each image by the weight in the corresponding weight map to obtain an intermediate pixel value of each pixel in each image; and adds the intermediate pixel values to obtain a target pixel, and the target pixels constitute a target image.

[0097] In this embodiment, the electronic device can accurately determine the first weight of each pixel in each image based on the brightness of each pixel in each image, and further determine the target weight of each pixel in each motion region; then, based on the target weight of each pixel in the motion region and the first weight of each pixel in the non-motion region of each image, a weight map corresponding to each image is generated, and at least two adjacent images are fused based on the weight map of each image, so that the target image can be accurately fused.

[0098] Further, the electronic device adjusts each pixel in the motion region based on the reference region, so as to obtain more accurate target weights and more accurately fuse to obtain a target image with higher quality and clarity.

[0099] In one embodiment, another image processing method is also provided, which is applied to an electronic device and includes the following steps:

[0100] Step A1, performing motion pixel identification on at least two adjacent images taken to determine pixels where motion objects are located in each image; the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold.

[0101] Step A2, determining each motion mask region in each image based on the pixels where the motion objects are located.

[0102] Step A3, if adjacent motion mask regions in each image are connected, the adjacent motion mask regions are fused into a motion region of a motion object; if the adjacent motion mask regions in each image are not connected, each motion mask region is taken as a motion region of a motion object respectively; each set of motion regions of motion objects includes the motion region of the motion object in each image.

[0103] Step A4, for each set of motion regions of motion objects, determining a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image.

[0104] Step A5, if the statistical parameter of each first pixel in the motion region of the first image is less than or equal to a preset statistical threshold, the motion region of the moving object in the first image is determined as the reference region; if the statistical parameter of each first pixel in the motion region of the first image is greater than the preset statistical threshold, the motion region of the moving object in the second image is determined as the reference region.

[0105] Step A6, based on the brightness of each pixel in each image, a first weight of each pixel in each image is determined.

[0106] Step A7, for each group of motion regions of moving objects, a pixel difference between the second pixel in the reference region and the corresponding third pixel in the motion region other than the reference region is determined, and a second weight of the third pixel is determined based on the pixel difference.

[0107] Step A8, the first weight is adjusted based on the second weight of the third pixel to obtain a target weight of the third pixel.

[0108] Step A9, based on the target weight of the third pixel, the first weight of the second pixel is adjusted to obtain a target weight of the second pixel.

[0109] Step A10, based on the target weight of each pixel in the motion region and the first weight of each pixel in the non-motion region of each image, a weight map corresponding to each image is generated.

[0110] Step A11, based on the weight map of each image, at least two adjacent images are fused to obtain a target image.

[0111] It should be understood that although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0112] Based on the same inventive concept, the embodiment of the present application further provides an image processing device for implementing the image processing method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations of the image processing method described above, which will not be repeated here.

[0113] In one embodiment, as shown in Figure 3 An image processing device is provided, comprising a motion region identification module 302, a reference region determination module 304, and a fusion module 306, wherein:

[0114] The motion region identification module 302 is configured to identify motion regions of at least one group of moving objects in at least two adjacent images, wherein the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and each motion region of the at least one group of moving objects includes a motion region of a moving object in each image.

[0115] The reference region determination module 304 is configured to determine, for each motion region of the at least one group of moving objects, a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image, and determine a reference region from each motion region of the at least one group of moving objects based on statistical parameters of the first pixel in the motion region of the first image.

[0116] The fusion module 306 is configured to fuse the at least two adjacent images to obtain a target image, and each motion region of the at least one group of moving objects is fused based on the corresponding reference region.

[0117] The image processing device described above, the electronic device identifies motion regions of at least one group of moving objects in at least two adjacent images, wherein the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and each motion region of the at least one group of moving objects includes a motion region of a moving object in each image. For each motion region of the at least one group of moving objects, a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image is determined, and a reference region of the at least one group of moving objects is accurately determined from each motion region of the at least one group of moving objects based on statistical parameters of the first pixel in the motion region of the first image. Then, each motion region of the at least one group of moving objects is fused based on the corresponding reference region, and a target image with higher signal-to-noise ratio, higher image quality and higher clarity can be obtained by fusion.

[0118] In an embodiment, the motion region identification module 302 is further configured to identify motion pixels in the at least two adjacent captured images, and determine each motion mask region in each image.

[0119] In an embodiment, the motion region identification module 302 is further configured to identify pixels of each motion object in each image based on the pixels of the motion object, and determine each motion mask region in each image based on the pixels of each motion object.

[0120] In an embodiment, the motion region identification module 302 is further configured to fuse adjacent motion mask regions into a motion region of a motion object if the adjacent motion mask regions are connected in each image, or respectively take each motion mask region as a motion region of a motion object if the adjacent motion mask regions are not connected in each image.

[0121] In an embodiment, the fusion module 306 is further configured to determine a first weight of each pixel in each image based on brightness of each pixel in each image, adjust the first weight of each pixel in each motion region of each group of motion objects based on the corresponding reference region to obtain a target weight of each pixel in each motion region, generate a weight map corresponding to each image based on the target weight of each pixel in the motion region of each image and the first weight of each pixel in the non-motion region, and fuse the at least two adjacent images based on the weight map of each image to obtain a target image.

[0122] In an embodiment, the fusion module 306 is further configured to determine a pixel difference between a second pixel in the reference region and a corresponding third pixel in a motion region other than the reference region for each motion region of each group of motion objects, and determine a second weight of the third pixel based on the pixel difference, adjust the first weight based on the second weight of the third pixel to obtain a target weight of the third pixel, and adjust the first weight of the second pixel based on the target weight of the third pixel to obtain a target weight of the second pixel.

[0123] In an embodiment, the reference region determination module 304 is further configured to determine the motion region of the motion object in the first image as the reference region if a statistical parameter of each first pixel in the motion region of the first image is less than or equal to a preset statistical threshold, or determine the motion region of the motion object in the second image as the reference region if the statistical parameter of each first pixel in the motion region of the first image is greater than the preset statistical threshold.

[0124] The various modules in the image processing apparatus described above can be implemented in whole or in part by software, hardware, and combinations thereof. The various modules described above can be embedded in hardware or independent of the processor in the electronic device, or stored in the memory in the electronic device in the form of software so as to be invoked and executed by the processor to perform the operations corresponding to the various modules.

[0125] In one embodiment, an electronic device, which can be a terminal, has an internal structure diagram as shown in Figure 4 The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the electronic device is configured to exchange information between the processor and external devices. The communication interface of the electronic device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement an image processing method. The display unit of the electronic device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0126] Those skilled in the art can understand that Figure 4 The structure shown in the above description is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] The embodiments of the present application also provide a computer readable storage medium. One or more non-volatile computer readable storage media containing computer executable instructions, when the computer executable instructions are executed by one or more processors, cause the processor to perform the steps of the image processing method.

[0128] The embodiments of the present application also provide a computer program product containing instructions, when the computer program product is run on a computer, cause the computer to perform the image processing method.

[0129] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of the country and region.

[0130] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. In the embodiments provided in the present application, any reference to memory, database or other medium can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0131] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0132] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: performing motion region identification on at least two adjacent images taken by a camera to determine motion regions of at least one group of moving objects; the at least two adjacent images comprise a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and the motion region of each group of moving objects comprises a motion region of the moving object in each image; for each motion region of each group of moving objects, a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image is determined, and a reference region is determined from each motion region of the moving object based on a statistical parameter of each first pixel in the motion region of the first image; the statistical parameter is used to represent overexposure of the motion region of the first image; the at least two adjacent images are fused to obtain a target image; and each motion region of each group of moving objects is fused based on the corresponding reference region.

2. The method of claim 1, wherein, The method comprises the following steps: performing motion pixel identification on at least two adjacent images taken by a camera to determine each motion mask region in each image; performing clustering processing on each motion mask region in each image to obtain motion regions of at least one group of moving objects.

3. The method of claim 2, wherein, The method comprises the following steps: performing motion pixel identification on at least two adjacent images taken by a camera to determine pixels where each moving object is located in each image; based on the pixels where each moving object is located, each motion mask region in each image is determined.

4. The method of claim 3, wherein, The method comprises the following steps: if adjacent motion mask regions in each image are connected, the adjacent motion mask regions are fused into a motion region of a moving object; if adjacent motion mask regions in each image are not connected, each motion mask region is taken as a motion region of a moving object respectively.

5. The method of claim 1, wherein, The method comprises the following steps: based on the brightness of each pixel in each image, a first weight of each pixel in each image is determined; for each motion region of each group of moving objects, the first weight of each pixel in each motion region is adjusted based on the corresponding reference region to obtain a target weight of each pixel in each motion region; based on the target weight of each pixel in the motion region of each image and the first weight of each pixel in the non-motion region of each image, a weight map corresponding to each image is generated; based on the weight map of each image, the at least two adjacent images are fused to obtain a target image.

6. The method of claim 5, wherein, The method comprises the following steps: for each motion region of each group of moving objects, the first weight of each pixel in each motion region is adjusted based on the corresponding reference region to obtain a target weight of each pixel in each motion region. determining a pixel difference between the second pixel in the reference region and a corresponding third pixel in a motion region other than the reference region for each motion region of the motion object, and determining a second weight of the third pixel based on the pixel difference; adjusting the first weight based on the second weight of the third pixel to obtain a target weight of the third pixel; adjusting the first weight of the second pixel based on the target weight of the third pixel to obtain a target weight of the second pixel.

7. The method according to any one of claims 1 to 6, characterized in that, The determining of the reference region from the motion region of each motion object includes: if the statistical parameter of each first pixel in the motion region of the first image is less than or equal to a preset statistical threshold, determining the motion region of the motion object in the first image as the reference region; if the statistical parameter of each first pixel in the motion region of the first image is greater than the preset statistical threshold, determining the motion region of the motion object in the second image as the reference region.

8. An image processing apparatus characterized by comprising: The method includes: a motion region identification module configured to identify motion regions of at least one group of motion objects from at least two adjacent images captured by a camera; the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold and a second image with an exposure parameter less than the preset exposure threshold, and the motion region of each group of motion objects includes a motion region of the motion object in each image; a reference region determination module configured to determine, for each motion region of the motion objects, a first pixel with a pixel value greater than a preset pixel threshold in the motion region of the first image, and determine a reference region from the motion region of each motion object based on a statistical parameter of each first pixel in the motion region of the first image; the statistical parameter is used to represent an overexposure condition of the motion region of the first image; a fusion module configured to fuse the at least two adjacent images to obtain a target image, and fuse each motion region of the motion objects based on a corresponding reference region.

9. An electronic device comprising a memory and a processor, said memory having stored therein a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to perform the steps of the image processing method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1 to 7.

11. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, causes the processor to perform the steps of the method of any one of claims 1 to 7.

Images