Image processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: performing motion area identification on at least two shot adjacent images, and determining a motion area of at least one group of motion objects; the at least two adjacent images comprise a first image of which the exposure parameter is greater than a preset exposure threshold value and a second image of which the exposure parameter is smaller than the preset exposure threshold value; for the motion area of each group of motion objects, determining first pixels with pixel values greater than a preset pixel threshold in the motion area of the first image, and determining a reference area from each motion area of the motion objects based on statistical parameters of each first pixel in the motion area of the first image; fusing at least two adjacent images to obtain a target image; and the motion areas of each group of motion objects are fused by taking the corresponding reference area as a reference. By adopting the method, the image quality and definition of the fused image can be improved.

Description

Technical Field

[0001] The present application relates to the field of imaging technology, and in particular to an image processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] Exposure fusion is a technology used to generate High Dynamic Range (HDR) images. It combines multiple images with different exposure levels, fusing their brightness and detail information to produce an image with a wider brightness range.

[0003] Traditional image processing methods usually use the same camera settings to shoot a series of images with different exposure levels, usually including low-exposure, standard-exposure, and high-exposure images, and then fuse multiple images to obtain a high dynamic range image. However, due to the existence of moving objects, it is necessary to specify an exposure image as a reference for exposure fusion when shooting images.

[0004] However, in traditional image processing methods, the specified reference image is usually fixed, and there is a problem that the signal-to-noise ratio of the fused image is low, resulting in a decrease in global image quality. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, electronic device, computer-readable storage medium and computer program product, which can more accurately determine the reference area for fusion of motion areas, improve the signal-to-noise ratio of the fused image, and thereby improve the image quality and clarity of the fused image.

[0006] In a first aspect, the present application provides an image processing method. The method comprises:

[0007] Performing motion region recognition on at least two adjacent images captured to determine at least one group of motion regions of a moving object; the at least two adjacent images include a first image having an exposure parameter greater than a preset exposure threshold, and a second image having an exposure parameter less than the preset exposure threshold, and the motion regions of each group of moving objects include the motion regions of the moving objects in each image;

[0008] For each group of motion regions of the motion objects, determining a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determining a reference region from each motion region of the motion objects based on statistical parameters of each first pixel in the motion region of the first image;

[0009] The at least two adjacent images are fused to obtain a target image; and the motion regions of each group of motion objects are fused based on the corresponding reference region.

[0010] In a second aspect, the present application also provides an image processing device. The device comprises:

[0011] A motion region identification module, configured to perform motion region identification on at least two adjacent images captured, and determine at least one group of motion regions of a moving object; the at least two adjacent images include a first image having an exposure parameter greater than a preset exposure threshold, and a second image having an exposure parameter less than the preset exposure threshold, and the motion regions of each group of moving objects include the motion regions of the moving objects in each image;

[0012] a reference region determining module, configured to determine, for each group of motion regions of the motion objects, a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determine a reference region from each motion region of the motion objects based on statistical parameters of each first pixel in the motion region of the first image;

[0013] The fusion module is used to fuse the at least two adjacent images to obtain a target image; the motion areas of each group of motion objects are fused based on the corresponding reference area.

[0014] In a third aspect, the present application further provides an electronic device. The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0015] Performing motion region recognition on at least two adjacent images captured to determine at least one group of motion regions of a moving object; the at least two adjacent images include a first image having an exposure parameter greater than a preset exposure threshold, and a second image having an exposure parameter less than the preset exposure threshold, and the motion regions of each group of moving objects include the motion regions of the moving objects in each image;

[0016] For each group of motion regions of the motion objects, determining a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determining a reference region from each motion region of the motion objects based on statistical parameters of each first pixel in the motion region of the first image;

[0017] The at least two adjacent images are fused to obtain a target image; and the motion regions of each group of motion objects are fused based on the corresponding reference region.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0019] Performing motion region recognition on at least two adjacent images captured to determine at least one group of motion regions of a moving object; the at least two adjacent images include a first image having an exposure parameter greater than a preset exposure threshold, and a second image having an exposure parameter less than the preset exposure threshold, and the motion regions of each group of moving objects include the motion regions of the moving objects in each image;

[0020] For each group of motion regions of the motion objects, determining a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determining a reference region from each motion region of the motion objects based on statistical parameters of each first pixel in the motion region of the first image;

[0021] The at least two adjacent images are fused to obtain a target image; and the motion regions of each group of motion objects are fused based on the corresponding reference region.

[0022] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0023] Performing motion region recognition on at least two adjacent images captured to determine at least one group of motion regions of a moving object; the at least two adjacent images include a first image having an exposure parameter greater than a preset exposure threshold, and a second image having an exposure parameter less than the preset exposure threshold, and the motion regions of each group of moving objects include the motion regions of the moving objects in each image;

[0024] For each group of motion regions of the motion objects, determining a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determining a reference region from each motion region of the motion objects based on statistical parameters of each first pixel in the motion region of the first image;

[0025] The at least two adjacent images are fused to obtain a target image; and the motion regions of each group of motion objects are fused based on the corresponding reference region.

[0026] The above-mentioned image processing method, device, electronic device, computer-readable storage medium and computer program product, the electronic device performs motion area recognition on at least two adjacent images taken, determines the motion area of ​​at least one group of moving objects, the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold, and a second image with an exposure parameter less than the preset exposure threshold, the motion area of ​​each group of moving objects includes the motion area of ​​the moving object in each image; for each group of moving objects, the first pixel in the motion area of ​​the first image with a pixel value greater than the preset pixel threshold is determined, and based on the statistical parameters of each first pixel in the motion area of ​​the first image, the reference area of ​​the group of moving objects can be accurately determined from the various motion areas of the moving objects; then, the motion areas of each group of moving objects are fused based on the corresponding reference area, and then the target image with higher signal-to-noise ratio, higher image quality and higher clarity can be fused. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 is a flowchart of an image processing method in one embodiment;

[0029] Figure 2 A schematic diagram of clustering a motion mask region in one embodiment;

[0030] Figure 3 is a structural block diagram of an image processing device in one embodiment;

[0031] Figure 4 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In one embodiment, Figure 1As shown, an image processing method is provided. This embodiment uses the method applied to an electronic device as an example. The electronic device can be a terminal or a server. It can be understood that the method can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be but not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, smart cars, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0034] In this embodiment, the image processing method includes the following steps:

[0035] Step S102, performing motion area recognition on at least two adjacent images captured to determine at least one group of motion areas of moving objects; the at least two adjacent images include a first image whose exposure parameter is greater than a preset exposure threshold, and a second image whose exposure parameter is less than the preset exposure threshold, and the motion areas of each group of moving objects include the motion areas of the moving objects in each image.

[0036] The moving object may be a person, an animal, an object or other object, etc. The exposure parameters may include exposure time, aperture, etc. The preset exposure threshold may be set as required.

[0037] It can be understood that the greater the exposure parameter, the greater the brightness of the image; the exposure parameter of the first image is greater than the preset exposure threshold, that is, the first image is a long exposure image; the exposure parameter of the second image is less than the preset exposure threshold, that is, the second image is a short exposure image. The number of the first images can be one or more, and the number of the second images can also be one or more.

[0038] Each group of moving object moving regions includes the moving region of the same moving object in each image.

[0039] Optionally, the electronic device includes a camera module; at least two adjacent images are captured by the camera module; at least two adjacent images are registered to obtain at least two registered adjacent images; motion area recognition is performed on the at least two registered adjacent images to determine the motion area of ​​at least one group of moving objects.

[0040] The electronic device performs global registration on at least two adjacent images. It is understandable that the global registration can ensure that at least two adjacent images after registration have local motion, while the globally moving parts are roughly aligned.

[0041] In other optional embodiments, the electronic device may also perform local registration on at least two adjacent images.

[0042] Step S104, for each group of motion areas of the motion objects, determine the first pixel in the motion area of ​​the first image whose pixel value is greater than a preset pixel threshold, and determine the reference area from the motion areas of the motion objects based on statistical parameters of the first pixels in the motion area of ​​the first image.

[0043] The preset pixel threshold can be set as needed. If there is a first pixel with a pixel value greater than the preset pixel threshold in the motion area of ​​the first image, it means that the brightness of the first pixel is high and the first pixel is an overexposed point; if the pixel value in the motion area of ​​the first image is less than or equal to the preset pixel threshold, it means that the brightness of the pixel is low.

[0044] The statistical parameter is a parameter obtained by statistics and is used to characterize the overexposure of the motion region of the first image. The larger the statistical parameter is, the greater the overexposure ratio of the motion region of the first image is.

[0045] Optionally, the statistical parameter may be the number of first pixels in the motion region of the first image, the area of ​​the first pixels in the motion region of the first image, the ratio of the number of first pixels to the total number of pixels in the motion region of the first image, and so on, but is not limited thereto.

[0046] The reference region is a region used as a reference when the motion regions of a group of moving objects are fused.

[0047] Optionally, for each group of moving areas of moving objects, the pixel value of each pixel in the moving area of ​​the moving object of each first image is detected, and the first pixel in the moving area of ​​the moving object of each first image whose pixel value is greater than a preset pixel threshold is determined; for the moving area of ​​the moving object of each first image, each first pixel is counted to obtain a statistical parameter; based on the statistical parameters of each first pixel in the moving area of ​​the first image, a reference area is determined from each moving area of ​​the moving object.

[0048] Optionally, for each group of motion areas of motion objects, if the statistical parameters in the motion areas of each first image are greater than a preset statistical threshold, the motion area of ​​the motion object in the second image is determined as the reference area; if there is a statistical parameter in the motion area of ​​the first image that is less than or equal to the preset statistical threshold, the motion area of ​​the motion object in the first image is determined as the reference area.

[0049] Step S106, fusing at least two adjacent images to obtain a target image; the motion regions of each group of moving objects are fused based on the corresponding reference region.

[0050] Optionally, for each group of moving areas of moving objects, the corresponding reference area is used as a reference to fuse the various moving areas of the moving objects to obtain a target moving area; the non-moving areas in at least two adjacent images are fused to obtain a target non-moving area; the target moving area and the target non-moving area are spliced ​​to obtain a target image.

[0051] The target image may be a high dynamic range image (HDR image, High Dynamic Range Imaging).

[0052] Optionally, the electronic device forms a reference image with a reference area of ​​the motion area and a non-motion area of ​​each group of motion objects; and fuses at least two adjacent images based on the reference image to obtain a target image.

[0053] In the above-mentioned image processing method, the electronic device identifies the motion area of ​​at least two adjacent images taken, determines the motion area of ​​at least one group of moving objects, the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold, and a second image with an exposure parameter less than the preset exposure threshold, and the motion area of ​​each group of moving objects includes the motion area of ​​the moving object in each image; for each group of moving objects, the first pixel in the motion area of ​​the first image whose pixel value is greater than the preset pixel threshold is determined, and based on the statistical parameters of each first pixel in the motion area of ​​the first image, the reference area of ​​the group of moving objects can be accurately determined from the various motion areas of the moving objects; then, the motion areas of each group of moving objects are fused based on the corresponding reference area, and then the target image with higher signal-to-noise ratio, higher image quality and higher clarity can be fused.

[0054] By adopting the above-mentioned image processing method, combined with the motion area, and based on the statistical parameters of each first pixel in the motion area of ​​the first image, the reference area can be more accurately determined from each motion area of ​​the moving object, thereby minimizing the problem of image quality degradation caused by the use of a dark frame reference, ensuring image quality to a certain extent and suppressing ghosting at the same time.

[0055] In one embodiment, motion region recognition is performed on at least two adjacent images captured to determine motion regions of at least one group of moving objects, including: motion pixel recognition is performed on at least two adjacent images captured to determine each motion mask region in each image; and clustering is performed on each motion mask region in each image to obtain at least one group of motion regions of moving objects.

[0056] The motion mask area (MASK) is used to block the processed image (all or part) with a selected image, graphic or object to control the area or process of image processing. At least two adjacent images can be RAW images. RAW images are linear threshold images, which are the original image data collected by the image sensor. The value of the RAW image itself is proportional to the light intensity, so it is said to be linear.

[0057] Optionally, the electronic device uses a preset motion pixel detection algorithm to perform motion pixel recognition on at least two adjacent images captured, determines the motion pixels in each image, and uses the area where each motion pixel is located as the motion mask area.

[0058] Optionally, motion pixel recognition is performed on at least two adjacent images captured to determine each motion mask area in each image, including: motion pixel recognition is performed on at least two adjacent images captured to determine the pixels where the moving object is located in each image; and based on the pixels where the moving objects are located, each motion mask area in each image is determined.

[0059] Among them, the motion pixels refer to the pixels where the moving objects are located in the image.

[0060] Optionally, the electronic device detects the image difference (temporal diff) between pixels at corresponding positions of at least two adjacent images; if the image difference satisfies a preset image difference condition, it indicates that the pixel is a pixel where a moving object is located; if the image difference does not satisfy the preset image difference condition, it indicates that the pixel is a pixel in a non-moving area.

[0061] Optionally, the preset image difference condition may be temporal diff>σ*c, where c is a coefficient for determining a motion model, and σ is a noise standard deviation obtained from a measured noise model. Both the motion model and the noise model may be obtained by pre-training.

[0062] Optionally, the electronic device uses a noise model to estimate the noise level of at least two adjacent images captured, determines non-noise pixels, identifies motion pixels for the non-noise pixels, and determines pixels where the moving object is located in each image.

[0063] It is understandable that the motion image difference between noise pixels and non-noise pixels is often not at the same level. The noise can be judged based on the preset noise judgment conditions. For example, using the normal distribution, the probability that the motion image difference of non-noise pixels is within 3σ is very low. The threshold c can be set to 3 to distinguish noise pixels from non-noise pixels.

[0064] It can be understood that at least two adjacent images are RAW images, which are linear threshold images. The x-gain method can be used to align the brightness of the at least two adjacent images, and then the motion region recognition is performed on the at least two adjacent images after the brightness alignment. Wherein, gain is the exposure ratio, and the premise of motion region recognition is that the brightness between the images is roughly the same, and the brightness alignment can be performed by multiplying the exposure ratio.

[0065] Optionally, after determining the pixels where the moving objects are located in each image, the electronic device performs a smoothing operation on the pixels where the moving objects are located to obtain each motion mask region (motion mask) in each image. The smoothing operation may include smoothing, dilation, erosion and other operations.

[0066] Optionally, the electronic device performs an erosion operation on each motion mask region to obtain an eroded motion mask region, and performs clustering processing on the eroded motion mask region to obtain at least one group of motion regions of the moving object. It is understandable that the electronic device performs an erosion operation on each motion mask region to remove scattered points similar to noise.

[0067] Optionally, clustering is performed on each motion mask region in each image to obtain at least one group of motion regions of the moving object, including: if adjacent motion mask regions in each image are connected, the adjacent motion mask regions are merged into the motion region of the moving object; if adjacent motion mask regions in each image are not connected, each motion mask region is used as the motion region of the moving object.

[0068] Optionally, the electronic device determines whether adjacent motion mask areas in each image are connected, and marks each motion mask area; each connected motion mask area is marked with the same marking signal; and the motion mask areas with the same marking signal are connected as a motion area of ​​a moving object.

[0069] It can be understood that each unconnected motion mask region is marked with different marker signals, which can ensure that different moving objects are distinguished.

[0070] Optionally, in each image, if the distance between adjacent motion mask regions is less than a preset distance threshold, it is determined that the adjacent motion mask regions are connected; if the distance between adjacent motion mask regions is greater than or equal to the preset distance threshold, it is determined that the adjacent motion mask regions are not connected. The preset distance threshold can be set as needed.

[0071] like Figure 2 As shown, the electronic device performs clustering processing on each motion mask area in the image to obtain three groups of motion areas of the moving objects.

[0072] The distance between adjacent motion mask areas may be the distance between the center positions of adjacent motion mask areas, the distance between the centroid positions of adjacent motion mask areas, or the distance between other positions of adjacent motion mask areas, which is not limited here.

[0073] In this embodiment, the electronic device performs motion pixel recognition on at least two adjacent images captured, determines each motion mask area in each image, and then performs clustering processing on each motion mask area in each image, so as to accurately obtain at least one group of motion areas of moving objects.

[0074] Furthermore, the electronic device can accurately determine the motion mask area in the image by performing motion pixel recognition on the image. The electronic device can accurately determine the motion area of ​​the moving object by determining whether adjacent motion mask areas are connected.

[0075] In one embodiment, a reference area is determined from each motion area of ​​a moving object based on statistical parameters of each first pixel in the motion area of ​​a first image, including: if the statistical parameters of each first pixel in the motion area of ​​the first image are less than or equal to a preset statistical threshold, the motion area of ​​the moving object in the first image is determined as the reference area; if the statistical parameters of each first pixel in the motion area of ​​the first image are greater than the preset statistical threshold, the motion area of ​​the moving object in the second image is determined as the reference area.

[0076] It can be understood that if the statistical parameters of each first pixel in the motion area of ​​the first image are less than or equal to the preset statistical threshold, indicating that the motion area of ​​the first image is not overexposed, the motion area of ​​the moving object in the first image with a larger exposure parameter is determined as the reference area; if the statistical parameters of each first pixel in the motion area of ​​the first image are greater than the preset statistical threshold, indicating that the motion area of ​​the first image is overexposed, the motion area of ​​the moving object in the second image with a smaller exposure parameter is determined as the reference area.

[0077] In this embodiment, the electronic device can determine whether the moving area of ​​the first image is overexposed based on the statistical parameters in the moving area in the first image, thereby determining the reference area more accurately; under the premise of no image ghosting and loss of dynamics, the moving area in the first image with a larger exposure parameter is used as the reference area as much as possible, which can improve the signal-to-noise ratio of the fused target image.

[0078] In one embodiment, at least two adjacent images are fused to obtain a target image, including: determining a first weight of each pixel in each image based on the brightness of each pixel in each image; for each group of motion areas of moving objects, adjusting the first weight of each pixel in each motion area based on the corresponding reference area to obtain a target weight of each pixel in each motion area; generating a weight map corresponding to each image based on the target weight of each pixel in the motion area of ​​each image and the first weight of each pixel in the non-motion area; and fusing at least two adjacent images based on the weight map of each image to obtain a target image.

[0079] The first weight is the weight calculated for the first time, and is also the weight for fusion of the non-motion area (fusion weight). The target weight is the weight for the final fusion of the motion area.

[0080] Optionally, the electronic device detects the pixel value of each pixel in each image and uses the pixel value as the brightness of the pixel; the brightness of each pixel is input into a preset first weight formula to determine the first weight of the pixel.

[0081] Among them, the first weight design method can adopt the Gaussian function method, as follows:

[0082]

[0083] Among them, weight fuslon is the first weight of the pixel, σ is a preset parameter and has nothing to do with the noise itself, x is the brightness of the pixel, mid is the preset optimal exposure brightness, and e is the Exp function (exponent).

[0084] Optionally, the first weight of each pixel in each motion area is adjusted based on the corresponding reference area to obtain a target weight of each pixel in each motion area, including: for each group of motion areas of the moving objects, determining the pixel difference between the second pixel in the reference area and the corresponding third pixel in the motion area other than the reference area, and determining the second weight of the third pixel based on the pixel difference; adjusting the first weight based on the second weight of the third pixel to obtain the target weight of the third pixel; adjusting the first weight of the second pixel based on the target weight of the third pixel to obtain the target weight of the second pixel.

[0085] For each group of moving objects in the moving area, the pixels in the base area are the second pixels, and the pixels in the moving area other than the base area are the third pixels. The pixel difference between the second pixel and the corresponding third pixel is determined; the pixel difference is input into a preset second weight formula to obtain a second weight of the third pixel.

[0086] Among them, the second weight formula is as follows:

[0087]

[0088] Among them, weight deghost It is 三 The second weight of the pixel, e is the Exp function (exponent), diff is the pixel difference between the second pixel and the corresponding third pixel after brightness alignment, offse and σ are pre-set parameters, which have nothing to do with the noise itself and need to be adjusted in advance to obtain a good de-ghosting effect.

[0089] Optionally, after determining the first weight of each pixel in each image based on the brightness of each pixel in each image, the electronic device normalizes the first weights of the pixels at corresponding positions in each image; for each group of motion regions of the motion objects, the electronic device adjusts the normalized first weight of each pixel in each motion region based on the corresponding reference region to obtain the second weight of each pixel in each motion region. The sum of the normalized first weights of the pixels at corresponding positions in each image is 1.

[0090] Optionally, the electronic device multiplies the first weight of the third pixel by the second weight of the third pixel to obtain a target weight of the third pixel.

[0091] It can be understood that the sum of the target weight of the second pixel and the target weight of the third pixel is 1; therefore, the electronic device performs difference processing on 1 and the target weight of the third pixel to obtain the target weight of the second pixel.

[0092] Exemplarily, for the moving area of ​​the moving object in the first image determined as the reference area, the first weight of the third pixel in the moving area of ​​the moving object in the second image is calculated to be w2, and the second weight deghost_weight, then the target weight of the third pixel is w2*deghost_weight, and the target weight of the second pixel in the reference area in the first image is w1=1-w2*deghost_weight.

[0093] Similarly, for the moving area of ​​the moving object in the second image determined as the reference area, the first weight of the third pixel in the moving area of ​​the moving object in the first image is calculated to be w1, and the second weight deghost_weight, then the target weight of the third pixel is w1*deghost_weight, and the target weight of the second pixel in the reference area in the second image is w2=1-w1*deghost_weight.

[0094] Optionally, for a non-moving area in the image, the first weight is maintained unchanged.

[0095] It can be understood that the electronic device obtains the first weight of the pixels in each non-moving area in each image, and the target weight of the pixels in the moving area, and can generate a weight map corresponding to each image; each weight in the weight map is the first weight or target weight of the corresponding pixel in the image.

[0096] Optionally, the electronic device multiplies the pixel value of each pixel in each image by the weight in the corresponding weight map to obtain an intermediate pixel value of each pixel in each image; adds each intermediate pixel value to obtain a target pixel, and constructs a target image with each target pixel.

[0097] In this embodiment, the electronic device can accurately determine the first weight of each pixel in each image based on the brightness of each pixel in each image, and further determine the target weight of each pixel in each motion area; then, based on the target weight of each pixel in the motion area of ​​each image and the first weight of each pixel in the non-motion area, a weight map corresponding to each image is generated, and based on the weight map of each image, at least two adjacent images are fused to accurately obtain the target image.

[0098] Furthermore, the electronic device adjusts each pixel in the motion area based on the reference area, thereby obtaining a more accurate target weight and performing more accurate fusion to obtain a target image with higher quality and clarity.

[0099] In one embodiment, another image processing method is provided, which is applied to an electronic device and includes the following steps:

[0100] Step A1, performing motion pixel recognition on at least two adjacent images captured to determine the pixels where the moving object is located in each image; the at least two adjacent images include a first image whose exposure parameter is greater than a preset exposure threshold, and a second image whose exposure parameter is less than the preset exposure threshold.

[0101] Step A2: determining each motion mask area in each image based on the pixels where each moving object is located.

[0102] Step A3: if adjacent motion mask regions in each image are connected, the adjacent motion mask regions are merged into the motion region of the moving object; if adjacent motion mask regions in each image are not connected, each motion mask region is used as the motion region of the moving object; each group of motion regions of the moving object includes the motion region of the moving object in each image.

[0103] Step A4: for each group of moving areas of moving objects, determine the first pixel in the moving area of ​​the first image whose pixel value is greater than a preset pixel threshold.

[0104] Step A5, if the statistical parameters of each first pixel in the motion area of ​​the first image are less than or equal to the preset statistical threshold, the motion area of ​​the moving object in the first image is determined as the reference area; if the statistical parameters of each first pixel in the motion area of ​​the first image are greater than the preset statistical threshold, the motion area of ​​the moving object in the second image is determined as the reference area.

[0105] Step A6, determining a first weight of each pixel in each image based on the brightness of each pixel in each image.

[0106] Step A7, for each group of motion regions of the moving object, determining a pixel difference between a second pixel in the reference region and a corresponding third pixel in the motion region other than the reference region, and determining a second weight of the third pixel based on the pixel difference.

[0107] Step A8: adjusting the first weight based on the second weight of the third pixel to obtain a target weight of the third pixel.

[0108] Step A9: Based on the target weight of the third pixel, adjust the first weight of the second pixel to obtain the target weight of the second pixel.

[0109] Step A10, generating a weight map corresponding to each image based on the target weight of each pixel in the motion area of ​​each image and the first weight of each pixel in the non-motion area.

[0110] Step A11, based on the weight map of each image, at least two adjacent images are fused to obtain a target image.

[0111] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0112] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0113] In one embodiment, Figure 3 As shown, an image processing device is provided, including: a motion region identification module 302, a reference region determination module 304 and a fusion module 306, wherein:

[0114] The motion region identification module 302 is used to perform motion region identification on at least two adjacent images captured to determine at least one group of motion regions of moving objects; the at least two adjacent images include a first image whose exposure parameter is greater than a preset exposure threshold, and a second image whose exposure parameter is less than the preset exposure threshold, and the motion region of each group of moving objects includes the motion region of the moving object in each image.

[0115] The reference area determination module 304 is used to determine, for each group of motion areas of the motion objects, the first pixel in the motion area of ​​the first image whose pixel value is greater than a preset pixel threshold, and determine the reference area from the various motion areas of the motion objects based on statistical parameters of the various first pixels in the motion area of ​​the first image.

[0116] The fusion module 306 is used to fuse at least two adjacent images to obtain a target image; the motion regions of each group of motion objects are fused based on the corresponding reference region.

[0117] In the above-mentioned image processing device, the electronic device performs motion area recognition on at least two adjacent images taken, determines the motion area of ​​at least one group of moving objects, the at least two adjacent images include a first image with an exposure parameter greater than a preset exposure threshold, and a second image with an exposure parameter less than the preset exposure threshold, and the motion area of ​​each group of moving objects includes the motion area of ​​the moving object in each image; for each group of moving objects, the first pixel in the motion area of ​​the first image whose pixel value is greater than the preset pixel threshold is determined, and based on the statistical parameters of each first pixel in the motion area of ​​the first image, the reference area of ​​the group of moving objects can be accurately determined from the various motion areas of the moving objects; then, the motion areas of each group of moving objects are fused based on the corresponding reference area, and then the target image with higher signal-to-noise ratio, higher image quality and higher clarity can be fused.

[0118] In one embodiment, the motion area identification module 302 is also used to perform motion pixel identification on at least two adjacent images captured to determine each motion mask area in each image; cluster each motion mask area in each image to obtain at least one group of motion areas of moving objects.

[0119] In one embodiment, the motion region identification module 302 is further used to perform motion pixel identification on at least two adjacent images captured to determine the pixels where the moving object is located in each image; and determine each motion mask region in each image based on the pixels where each moving object is located.

[0120] In one embodiment, the motion region identification module 302 is further used to merge adjacent motion mask regions into the motion region of the moving object if the adjacent motion mask regions in each image are connected; and to use each motion mask region as the motion region of the moving object if the adjacent motion mask regions in each image are not connected.

[0121] In one embodiment, the fusion module 306 is also used to determine the first weight of each pixel in each image based on the brightness of each pixel in each image; for each group of moving objects, the first weight of each pixel in each moving area is adjusted based on the corresponding reference area to obtain the target weight of each pixel in each moving area; based on the target weight of each pixel in the moving area of ​​each image and the first weight of each pixel in the non-moving area, a weight map corresponding to each image is generated; based on the weight map of each image, at least two adjacent images are fused to obtain the target image.

[0122] In one embodiment, the fusion module 306 is also used to determine, for each group of moving objects in a moving area, a pixel difference between a second pixel in a reference area and a corresponding third pixel in a moving area other than the reference area, and determine a second weight of the third pixel based on the pixel difference; adjust the first weight based on the second weight of the third pixel to obtain a target weight of the third pixel; and adjust the first weight of the second pixel based on the target weight of the third pixel to obtain a target weight of the second pixel.

[0123] In one embodiment, the above-mentioned reference area determination module 304 is also used to determine the motion area of ​​the moving object in the first image as the reference area if the statistical parameters of each first pixel in the motion area of ​​the first image are less than or equal to a preset statistical threshold; if the statistical parameters of each first pixel in the motion area of ​​the first image are greater than the preset statistical threshold, then determine the motion area of ​​the moving object in the second image as the reference area.

[0124] Each module in the above-mentioned image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in an electronic device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0125] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0126] Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] The embodiment of the present application further provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, enable the processors to perform the steps of the image processing method.

[0128] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the image processing method.

[0129] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0131] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: include: Performing motion region recognition on at least two adjacent images captured to determine at least one group of motion regions of moving objects; The at least two adjacent images include a first image having an exposure parameter greater than a preset exposure threshold, and a second image having an exposure parameter less than the preset exposure threshold, and the motion region of each group of moving objects includes the motion region of the moving object in each image; For each group of motion regions of the motion objects, determining a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determining a reference region from each motion region of the motion objects based on statistical parameters of each first pixel in the motion region of the first image; The at least two adjacent images are fused to obtain a target image; and the motion regions of each group of motion objects are fused based on the corresponding reference region.

2. The method according to claim 1, characterized in that The step of performing motion region recognition on at least two adjacent images to determine at least one group of motion regions of moving objects includes: Performing motion pixel recognition on at least two adjacent images captured to determine each motion mask area in each image; Clustering is performed on each motion mask region in each image to obtain at least one group of motion regions of the moving object.

3. The method according to claim 2, characterized in that The step of performing motion pixel recognition on at least two adjacent images to determine each motion mask area in each image includes: Performing motion pixel recognition on at least two adjacent images captured to determine the pixels where the moving object is located in each image; Based on the pixels where each moving object is located, each motion mask area in each image is determined.

4. The method according to claim 3, characterized in that The clustering process is performed on each motion mask region in each image to obtain at least one group of motion regions of the moving object, including: If adjacent motion mask regions in each image are connected, the adjacent motion mask regions are merged into a motion region of the moving object; If adjacent motion mask regions in each image are not connected, each motion mask region is used as a motion region of a moving object.

5. The method according to claim 1, characterized in that The step of fusing the at least two adjacent images to obtain a target image comprises: Determining a first weight for each pixel in each image based on the brightness of each pixel in each image; For each group of moving regions of moving objects, adjusting the first weight of each pixel in each moving region based on the corresponding reference region to obtain a target weight of each pixel in each moving region; Generate a weight map corresponding to each image based on the target weight of each pixel in the motion region of each image and the first weight of each pixel in the non-motion region; Based on the weight map of each image, at least two adjacent images are fused to obtain a target image.

6. The method according to claim 5, characterized in that The method of adjusting the first weight of each pixel in each moving area based on the corresponding reference area for each group of moving areas of the moving object to obtain the target weight of each pixel in each moving area includes: For each group of motion regions of the moving object, determining a pixel difference between a second pixel in the reference region and a corresponding third pixel in the motion region other than the reference region, and determining a second weight of the third pixel based on the pixel difference; Adjusting the first weight based on the second weight of the third pixel to obtain a target weight of the third pixel; Based on the target weight of the third pixel, the first weight of the second pixel is adjusted to obtain the target weight of the second pixel.

7. The method according to any one of claims 1 to 6, characterized in that: The determining of the reference area from each motion area of ​​the moving object based on the statistical parameters of each first pixel in the motion area of ​​the first image comprises: If a statistical parameter of each first pixel in the motion region of the first image is less than or equal to a preset statistical threshold, determining the motion region of the moving object in the first image as a reference region; If the statistical parameter of each first pixel in the motion region of the first image is greater than a preset statistical threshold, the motion region of the moving object in the second image is determined as a reference region.

8. An image processing device, characterized in that: include: A motion region recognition module, used to perform motion region recognition on at least two adjacent images captured, and determine a motion region of at least one group of moving objects; The at least two adjacent images include a first image having an exposure parameter greater than a preset exposure threshold, and a second image having an exposure parameter less than the preset exposure threshold, and the motion region of each group of moving objects includes the motion region of the moving object in each image; a reference region determining module, configured to determine, for each group of motion regions of the motion objects, a first pixel in the motion region of the first image whose pixel value is greater than a preset pixel threshold, and determine a reference region from each motion region of the motion objects based on statistical parameters of each first pixel in the motion region of the first image; The fusion module is used to fuse the at least two adjacent images to obtain a target image; the motion areas of each group of motion objects are fused based on the corresponding reference area.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images