A low-light video enhancement method and system based on dark channel and motion estimation

Abstract

The application belongs to the technical field of computer vision and image processing, and particularly relates to a low-illumination video enhancement method and system based on dark channel and motion estimation. The method comprises the following steps: acquiring an original low-illumination video, wherein the original low-illumination video comprises a plurality of original image groups with the same size; performing image pixel inversion on all video frames in the original image groups to obtain a pseudo-fog image group; performing improved adaptive multi-scale minimum filtering to obtain a dark channel image group; performing absolute error and block constraint on adjacent image frame pixel blocks, performing transmittance matching or transmittance block calculation to obtain the transmittance of each image frame of the dark channel image group; stretching the transmittance of each image frame to obtain the defogging result of each image frame of the dark channel image group; and performing image pixel inversion on the defogging result of each image frame to obtain an enhanced image video of the original low-illumination video. The application has good video enhancement effect, small operation cost and good real-time enhancement capability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of digital video image processing, and particularly relates to a low-illumination video enhancement method and system based on a dark channel and motion estimation. BACKGROUND

[0002] In the 21st century, the medium for people to obtain external information in daily production and life has a large proportion of video information. Video acquisition devices such as security monitoring and mobile phone cameras can obtain video information with high clarity, high contrast and rich details in well-lit scenes, which can meet people's requirements for video quality. However, in the night, mine and other scenes with insufficient light, the imaging quality of video acquisition devices will decrease to different degrees, which causes great trouble in regional security, underground operation and other fields. Since people have strict real-time requirements for video monitoring and remote video operation information, most of the current low-illumination image enhancement methods cannot meet the real-time requirements of video enhancement. In a low-illumination environment, it is a severe challenge to realize video acquisition and real-time enhancement.

[0003] At present, many domestic and foreign scholars have proposed various low-illumination image enhancement technologies, but in the field of video enhancement, these technologies face more challenges. Compared with static images, video enhancement not only requires more than dozens of times of computing resources, but also is limited because it is difficult to obtain real paired data sets under normal illumination and low illumination in the same scene. Most existing image enhancement technologies rely on artificially synthesized data sets for training, which leads to poor performance when processing actual low-illumination images or videos, and often produces a lot of noise. Therefore, although some low-illumination image enhancement algorithms based on deep learning perform well, due to the lack of real data sets suitable for video enhancement and high computing cost, these low-illumination image enhancement algorithms are difficult to be directly applied to video processing.

[0004] In addition, low-illumination image enhancement methods such as unsupervised learning and zero-shot learning have a certain flexibility, but they are difficult to meet the real-time processing requirements of video enhancement due to high computational complexity. Traditional image enhancement methods based on histogram equalization can effectively improve the contrast of images, but may also cause color distortion and detail loss. Although the method based on Retinex theory can improve the uneven illumination, its performance is highly dependent on the selected model function, and may show over-enhancement or insufficient enhancement for scenes with different illumination intensities, which is difficult to cope with the multi-scene situation of video enhancement. SUMMARY

[0005] Therefore, in order to solve the problems of the prior art in the field of low-illumination video enhancement, the adaptive multi-scale minimum filtering mechanism is introduced into the acquisition of the dark channel of the pseudo-fog image. The fixed filtering window can obtain the dark channel value of the main scene, but when the light and dark changes obviously and the distance of the scene changes suddenly, the assumption of constant transmittance of the filtering window is broken, and the result of minimum filtering cannot represent the dark channel value of the edge. The dark channel value of the edge position is essential for the refinement of the transmittance, and the adaptive multi-scale minimum filtering mechanism can realize the adaptive selection of the filtering window by judging whether the filtering window is at the junction of the light and dark and the distance of the scene changes suddenly, so that the transmittance map of the pseudo-fog image can be obtained more accurately, thereby avoiding phenomena such as halos and blocking effects. For the estimation of the atmospheric light value, the selection of the atmospheric light value will be greatly affected by the time and space of the collection device, and it is unreasonable to use the same scale of atmospheric light value estimation method. Therefore, for the estimation of the atmospheric light value, the atmospheric light value estimation method is used to select the smooth area of the pseudo-fog image and exclude the interference of noise and other factors, which can prevent the white, bright scene or noise in a certain area of the collected image from being used as the atmospheric light value, thereby improving the contrast of the enhanced image, highlighting the edge information, reducing the halo effect, relieving the color distortion problem, and improving the visual perception of the image. In the same image group, the time and space change of the image is usually very small, so the atmospheric light value of the initial frame is used in the same image group, and the transmittance is matched with the best value of the adjacent frame through the constraint of motion estimation. The absolute error constraint sensitive to illumination error is introduced to judge the block matching degree of the adjacent frame, and the fast matching of the transmittance is performed, thereby greatly improving the real-time performance of the present application. Finally, in order to avoid the phenomenon of over-enhancement, a smoothing factor is introduced in the algorithm output stage to focus on the enhancement of the region of interest, so that the obtained video image is more visually smooth.

[0006] The present application aims to provide a low-illumination video enhancement method and system based on dark channel and motion estimation, which can effectively improve the brightness and processing speed of video images.

[0007] In the first aspect of the present application, a low-illumination video fast enhancement method based on dark channel prior and motion estimation is provided, which comprises:

[0008] An original low-illumination video is obtained, which comprises a plurality of original image groups of the same size, and each original image group comprises a plurality of continuous low-illumination video frames;

[0009] The image pixel inversion is performed on all video frames in the original image group to obtain a pseudo-fog image group;

[0010] performing improved adaptive multi-scale minimum value filtering on the pseudo-fog image group to obtain a dark channel image group;

[0011] performing transmittance block calculation on an initial image frame in the dark channel image group to obtain transmittance of the initial image frame in the dark channel image group;

[0012] performing absolute error and block constraint on pixel blocks of adjacent image frames in the dark channel image group, performing transmittance matching or transmittance block calculation to obtain transmittance of each adjacent image frame in the dark channel image group;

[0013] stretching the transmittance of each image frame in the dark channel image group to obtain a defogging result of each image frame in the dark channel image group;

[0014] performing image pixel inversion on the defogging result of each image frame in the dark channel image group to obtain an enhanced image video of the original low-illumination video.

[0015] In a second aspect of the present application, the present application further provides a low-illumination video fast enhancement system based on dark channel prior and motion estimation, the system comprising:

[0016] a video acquisition unit configured to acquire an original low-illumination video, the original low-illumination video comprising a plurality of original image groups of the same size, each original image group comprising a plurality of continuous low-illumination video frames;

[0017] a first processing unit configured to perform image pixel inversion on all video frames in the original image group to obtain a pseudo-fog image group;

[0018] a second processing unit configured to perform improved adaptive multi-scale minimum value filtering on the pseudo-fog image group to obtain a dark channel image group;

[0019] a third processing unit configured to perform transmittance block calculation on an initial image frame in the dark channel image group to obtain transmittance of the initial image frame in the dark channel image group;

[0020] a fourth processing unit configured to perform absolute error and block constraint on pixel blocks of adjacent image frames in the dark channel image group, performing transmittance matching or transmittance block calculation to obtain transmittance of each adjacent image frame in the dark channel image group;

[0021] a fifth processing unit configured to stretch the transmittance of each image frame in the dark channel image group to obtain a defogging result of each image frame in the dark channel image group;

[0022] a sixth processing unit configured to perform image pixel inversion on the defogging result of each image frame in the dark channel image group to obtain an enhanced image video of the original low-illumination video.

[0023] The beneficial effects of the present application are as follows:

[0024] The present application improves the adaptive multi-scale minimum filter, so that the scene edge information is maintained while obtaining the dark channel image, and the halo phenomenon and the blocking effect of object boundary at the position of image brightness highlight are avoided; the present application optimizes the estimation of the atmospheric light value, so as to improve the accuracy of the transmittance calculation, selects the atmospheric light value in the smooth area of the image pixel value, avoids the distortion of the enhanced image caused by the influence of the image highlight area and noise on the estimation of the atmospheric light value, and improves the generalization ability of the algorithm; the present application realizes the transmittance optimal value matching through the motion estimation of the video front and rear frames, reduces the transmittance calculation amount of the video enhancement process, and improves the video image enhancement efficiency. The present application has the effects of high contrast of enhanced video picture, uniform overall brightness, real-time enhancement process and the like. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a low-illumination video enhancement method flowchart based on dark channel and motion estimation in the present application;

[0026] Figure 2 is an improved adaptive multi-scale minimum filter flowchart in the present application;

[0027] Figure 3 is a transmittance block calculation flowchart in the present application;

[0028] Figure 4 is a flowchart for performing transmittance matching or transmittance block calculation in the present application;

[0029] Figure 5 is a low-illumination video enhancement system structure diagram based on dark channel and motion estimation in the present application;

[0030] Figure 6 is a dark channel effect comparison diagram of the present application and the comparison method on the video image;

[0031] Figure 7 is an atmospheric light value selection area comparison diagram of the present application and the comparison method on the video image;

[0032] Figure 8 is an enhanced effect comparison diagram of the present application and the comparison method on the video frame. DETAILED DESCRIPTION

[0033] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort are within the scope of the present application.

[0034] The low-illumination video enhancement method and system based on dark channel and motion estimation provided by the embodiments of the present application will be described in detail below through specific embodiments.

[0035] It can be understood that the low-illumination video enhancement method based on dark channel and motion estimation in the embodiments of the present application can be implemented by an electronic device, and the electronic device can be a smart phone, a tablet computer, a wearable electronic device, a vehicle-mounted device, a gateway, etc., which is not specifically limited here.

[0036] Please refer to Figure 1 , which shows a low-illumination video enhancement method based on dark channel and motion estimation provided by the embodiments of the present application, and the method specifically includes:

[0037] S1: obtaining an original low-illumination video, the original low-illumination video including a plurality of original image groups of the same size, each original image group including a plurality of continuous low-illumination video frames;

[0038] The low-illumination video can be obtained by a video acquisition device in a case of insufficient illumination or by controlling the ambient brightness to reach a low-illumination environment, or a low-illumination video dataset can be selected, etc. The present application does not limit the picture content of the low-illumination video, for example, the original video can be furniture, a human object, or a natural environment, etc. which is shot indoors.

[0039] For example, the original low-illumination video is divided into a plurality of original image groups with a group of pictures (GOP) size of 10 in the embodiment, each original image group is composed of 10 continuous low-illumination video frames, the first frame is an I frame, and the remaining 9 frames are P frames. The present application needs to calculate and obtain the atmospheric light value, the dark channel value, and the transmittance of the I frame, but only needs to obtain the dark channel value of the P frame, and the transmittance is obtained by optimal value matching or through transmittance block operation. The atmospheric light value of the I frame is used as the input for the same GOP.

[0040] S2: performing image pixel inversion on all video frames in the original image group to obtain a pseudo-fog image group;

[0041] In the embodiment of the present application, the image pixel operation acts on the pixel level, which means subtracting 255 from the RGB value of the pixel. For example, the original RGB value of a pixel is (100, 150, 200), and the inverted RGB value is (155, 105, 55). This inversion operation will invert the color and brightness of the image, producing a negative film-like effect, whereby the original image group can be changed into a pseudo-fog image group, and the pseudo-fog image obtained by inversion has high similarity with the fog-containing image.

[0042] S3: performing improved adaptive multi-scale minimum value filtering on the pseudo-fog image group to obtain a dark channel image group;

[0043] In some embodiments, as shown in FIG. 3, the improved adaptive multi-scale minimum value filtering on the pseudo-fog image group comprises: Figure 2

[0044] S31: obtaining the maximum value, the minimum value and the average value of the pixel value of the pseudo-fog image group in an initial filtering window;

[0045] S32: if the difference between the maximum value and the minimum value in the initial filtering window is greater than the difference between the average value and the minimum value by a preset multiple, the dark channel value of the current filtering window is the minimum value of the pixels in the initial filtering window;

[0046] S33: if the difference between the maximum value and the minimum value in the initial filtering window is less than or equal to the difference between the average value and the minimum value by a preset multiple, minimum value filtering is performed in the current filtering window in a multi-scale small window manner;

[0047] S34: if the difference between the maximum value and the minimum value in the multi-scale small window is greater than the difference between the average value and the minimum value by a preset multiple, the dark channel value in the current multi-scale small window is the minimum value of the pixels in the multi-scale small window;

[0048] S35: if the difference between the maximum value and the minimum value in the multi-scale small window is less than or equal to the difference between the average value and the minimum value by a preset multiple, the dark channel value in the current multi-scale small window is the average value of the pixels in the multi-scale small window.

[0049] For example, the embodiment performs coarse filtering on the pseudo-fog image, sets the initial filtering window size to W_size=9 and the preset multiple to 10 / 7, obtains the maximum value, the minimum value and the average value of the pixel value in each initial window, and takes the pseudo-fog image group and the maximum value, the minimum value and the average value as the input of the multi-scale minimum value filtering algorithm to obtain a dark channel image group. The adaptive multi-scale minimum value filter is specifically represented as follows:

[0050]

[0051] wherein, W​s is the filter window size, I max , I min , I avg are the maximum, minimum and average pixel values ​​within the window, respectively.

[0052] Specifically, the filtering steps are as follows:

[0053] (1) Calculate the maximum value max, minimum value min and average value avg of the pixels in the initial window;

[0054] (2) When the pixel value in the initial window satisfies (I max -I min )×0.7>I avg -I min When the relationship is , the current filter window does not need to be refined, and the minimum value filtering method is used to obtain the dark channel value;

[0055] (3) When the pixel value in the initial window satisfies (I max -I min )×0.7≤I avg -I min When the relationship is established, a multi-scale small window is used in the current window for more detailed filtering. After the first step in the small window, if (I max -I min )×0.7≤I avg -I min If the relationship is correct, the average value in the small window is used instead of the minimum value as the result.

[0056] Among them, the multi-scale small window is a plurality of small windows of different scales that are smaller than the initial filtering window, wherein each small window of different scales is an integer multiple of the minimum window scale, and the minimum window scale is 2; for example, if the initial filtering window is 16×16, then each small window of different scales can be 8×8, 4×4, 2×2, and so on.

[0057] It can be understood that the embodiment of the present invention adopts an adaptive multi-scale minimum filtering mechanism, which determines whether the filter window is at the junction of light and dark, and near and far changes of the scene by judging the amplitude of the change of pixel values ​​within the filter window, so as to realize adaptive selection of the filter window, and can obtain the transmittance map of the pseudo-fog image more accurately, thereby avoiding halo, block effect and other phenomena.

[0058] S4: performing transmittance block calculation on the initial image frame in the dark channel image group to obtain the transmittance of the initial image frame in the dark channel image group; in this embodiment of the present invention, Figure 3 As shown, the transmittance block calculation process includes:

[0059] S41, determine the image gradient of the target image frame in the dark channel image group by using Laplace operator;

[0060] When the initial image frame in the dark channel image group is processed, the target image frame is the initial image frame, and when other image frames in the dark channel image group are processed, the target image frame is the corresponding other image frame.

[0061] In some embodiments, when the initial image frame in the dark channel image group is processed by Laplace, the second derivatives of each pixel point of the initial image frame in the x direction and the y direction are solved respectively, the changes of the initial image frame in the horizontal direction and the vertical direction are obtained, and the image gradient of each pixel point of the initial image frame obtained by using the Laplace operator is obtained.

[0062] S42, determine the pixel smooth area of the target image frame by using image smooth area threshold constraint;

[0063] In some embodiments, for the image gradient of each pixel point of the initial image frame obtained in step S41, if the image gradient of a pixel point is lower than the image smooth area threshold, the pixel point can be considered as a smooth pixel point, and the pixel smooth area of the initial image frame can be obtained by combining these smooth pixel points.

[0064] S43, determine the pixel point set of the pixel smooth area by using image smooth area key pixel constraint;

[0065] In some embodiments, the image smooth area key pixel constraint is the constraint of the pixel points with high ranking pixel values in the pixel smooth area; for the pixel smooth area of the initial image frame obtained in step S42, the pixel values of all pixel points in the pixel smooth area are ranked, and the pixel points with high ranking pixel values, for example, the first 0.1% of the pixel points, can be selected as the pixel point set of the pixel smooth area.

[0066] S44, determine the atmospheric light value of the target image frame by processing the pixel point set of the pixel smooth area by using the median filter function;

[0067] In some embodiments, for the pixel point set of the pixel smooth area obtained in step S43, the median filter function is used for processing, and the middle value of the sorted pixel values in the window is selected as the atmospheric light value of the initial image frame.

[0068] This embodiment selects smooth areas of the image and uses median filtering to eliminate interference from factors such as noise, which can prevent the capture of white, brighter scenes or noise in a certain area of ​​the image as atmospheric light values. This can improve the contrast of the enhanced image, highlight edge information, reduce the halo effect, alleviate color distortion problems, and improve the visual experience of the image.

[0069] S45 . Based on an atmospheric physical scattering model and according to the atmospheric light value of the target image frame, calculate the transmittance of the target image frame in the dark channel image group.

[0070] According to the atmospheric physics scattering model, that is:

[0071] E(d,λ)=E0(y)e -β(λ)d +E ∞ (λ)(1-e -β(λ)d )

[0072] I(x)=J(x)T(x)+A(1-T(x))

[0073] It is generally believed that the wavelength of incident light is constant and the suspended particles in the air are evenly distributed. Therefore, I(x) is used instead of E(d,λ) to represent the foggy image, J(x) is used instead of E0(λ) to represent the fog-free image to be restored, and T(x) is used instead of e -β(λ)d Indicates the transmittance of light, the total intensity of ambient light E ∞ (λ) is expressed in terms of the atmospheric light value A.

[0074] The dark channel solution formula is as follows:

[0075]

[0076] According to the dark channel prior theory, we know that:

[0077] J dark (x)→0

[0078]

[0079] Where A is the atmospheric light value, is the pixel value of the image, Ω(x) is the filter window area, and ω is a constant set to avoid the situation where T(x) is zero, which is generally selected between 0.85 and 0.95.

[0080] S5: performing absolute error and block constraint on corresponding pixel blocks of adjacent image frames of the dark channel image group, performing transmittance matching or transmittance block calculation, and obtaining the transmittance of each adjacent image frame of the dark channel image group;

[0081] In the embodiments of the present application, it is necessary to judge whether the pixel blocks corresponding to adjacent image frames of the dark channel image group satisfy the absolute error sum and block constraint condition, so as to select the transmittance matching or calculate the corresponding transmittance according to the transmittance block. When the pixel values of the pixel blocks corresponding to adjacent image frames are within the constraint error, the transmittance matching is directly performed, otherwise the transmittance block calculation is performed. As shown in FIG. 2, it specifically includes: Figure 4

[0082] S51, if the pixel value absolute error sum between the pixel blocks corresponding to the current video frame and the previous video frame of the dark channel image group is less than or equal to a preset threshold value, the transmittance of the pixel block of the current video frame is matched, and the transmittance of the pixel block of the current video frame is the transmittance of the pixel block at the corresponding position of the previous video frame;

[0083] In some embodiments, the preset threshold value is a constant, which can be determined according to the original video content, quality and pixel block size, for example, selected as the preset threshold value within [10, 30].

[0084] It should be noted that the pixel value absolute error sum between the pixel blocks corresponding to the current video frame and the previous video frame is obtained by summing the absolute difference values of the pixel values of the corresponding pixel blocks of the two video frame images. For each pixel block, the pixel absolute error sum between the pixel blocks corresponding to the current video frame and the previous video frame is calculated. In order to reduce the influence of noise, a threshold value can be set. Only when the difference value exceeds the threshold value, it is considered that the pixel in the region changes.

[0085] If the pixel value absolute error sum between the pixel blocks corresponding to the current video frame and the previous video frame is less than or equal to the preset threshold value, the transmittance of the pixel block of the previous video frame can be directly used as the transmittance of the pixel block of the current video frame. In this way, the best value matching of adjacent frames is performed through the constraint of motion estimation, the absolute error constraint sensitive to illumination error is introduced to judge the block matching degree of adjacent frames, the block matching of transmittance is performed, so that the real-time performance of the present application can be greatly improved.

[0086] S52, if the pixel value absolute error sum between the pixel blocks corresponding to the current video frame and the previous video frame of the dark channel image group is greater than the preset threshold value, the transmittance of the pixel block of the current video frame is calculated by the transmittance block, and the transmittance of the current video frame is calculated by the transmittance block.

[0087] ​In some embodiments, if the sum of absolute errors of pixel values between the pixel blocks corresponding to the current video frame and the previous video frame is greater than a preset threshold, it indicates that the transmittance of the pixel blocks corresponding to the current video frame and the previous video frame cannot be matched, and the transmittance block calculation method can be used to obtain the transmittance, wherein the transmittance block calculation method can use the method described in steps S41-S45, at this time, the target image frame is the current image frame, and the transmittance of the current video frame can be obtained by using the steps S41-S45 on the current image frame.

[0088] It can be understood that, in use, other constraint conditions can be selected for motion estimation constraint according to the content, quality and other requirements of the video image, which are not listed one by one here.

[0089] S6: stretching the transmittance of each image frame of the dark channel image group to obtain the defogging result of each image frame of the dark channel image group;

[0090] In some embodiments, the transmittance of each image frame of the dark channel image group is nonlinearly stretched to obtain the defogging result of each image frame of the dark channel image group; if the transmittance of the target image frame of the dark channel image group is in the first range interval, the corresponding nonlinear stretching smoothing factor is twice the transmittance of the target image frame, and if the transmittance of the target image frame of the dark channel image group is in the second range interval, the corresponding nonlinear stretching smoothing factor is a fixed value 1.

[0091] S7: performing image pixel inversion on the defogging result of each image frame of the dark channel image group to obtain an enhanced image video of the original low-illumination video.

[0092] Specifically, the defogging process expression is as follows:

[0093]

[0094] wherein, is the defogging result of the pseudo-fog image, is the pseudo-fog image, A is the atmospheric light value of the pseudo-fog image group, P(x) is the smoothing factor of the nonlinear stretching of the transmittance, T d is the lower limit of the transmittance, generally in the range of 0.05-0.15. Finally, the defogging result is inverted to obtain the final enhanced image, as follows:

[0095]

[0096] By using the low-illumination video fast enhancement method based on dark channel prior and motion estimation, the original low-illumination video can be enhanced to obtain an enhanced image video of the original low-illumination video.

[0097] The embodiment of the present application also provides a low-illumination video fast enhancement system based on dark channel prior and motion estimation. Figure 5 The device comprises:

[0098] A video acquisition unit 100 is configured to acquire an original low-illumination video, wherein the original low-illumination video comprises a plurality of original image groups of the same size, and each original image group comprises a plurality of continuous low-illumination video frames;

[0099] A first processing unit 101 is configured to perform image pixel inversion on all video frames in the original image group to obtain a pseudo-fog image group;

[0100] A second processing unit 102 is configured to perform improved adaptive multi-scale minimum filtering on the pseudo-fog image group to obtain a dark channel image group;

[0101] A third processing unit 103 is configured to perform transmittance block calculation on an initial image frame in the dark channel image group to obtain the transmittance of the initial image frame in the dark channel image group;

[0102] A fourth processing unit 104 is configured to perform absolute error and block constraint on a pixel block of adjacent image frames in the dark channel image group, perform transmittance matching or transmittance block calculation to obtain the transmittance of each adjacent image frame in the dark channel image group;

[0103] A fifth processing unit 105 is configured to stretch the transmittance of each image frame in the dark channel image group to obtain the defogging result of each image frame in the dark channel image group;

[0104] A sixth processing unit 106 is configured to perform image pixel inversion on the defogging result of each image frame in the dark channel image group to obtain an enhanced image video of the original low-illumination video.

[0105] After obtaining the enhanced result of each GOP, a final video enhancement result can be synthesized, and the transmittance map obtained by the present application and the DCP method is compared as shown in FIG. 5, wherein (a) is an original image, (b) is the transmittance map obtained by the DCP method, and (c) is the transmittance map obtained by the present application. Figure 6 Figure 7 The atmospheric light value region selected by the present application and the DCP method is compared as shown in FIG. 6, wherein (a) is an original image, the red region in (b) is the atmospheric light value region selected by the DCP method, and the red region in (c) is the atmospheric light value region selected by the present application. Figure 8 ​As shown in the figure, (a) is the original image, (b) is the image obtained after enhancement by the DCP method, and (c) is the enhanced image obtained by the application; by comparison, it can be obviously seen that the transmission map obtained by the application is better refined, the selected atmospheric light value is not easily affected by light and special points of white pigment, the obtained video image is clearer, has fewer noise points and less distortion phenomenon, and has obvious improvement in contrast, brightness, details and the like, and is more in line with the subjective visual effect of human beings.

[0106] Those skilled in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by programs instructing the relevant hardware, and the programs can be stored in a computer readable storage medium, which can include ROM, RAM, magnetic disks or optical disks, etc.

[0107] The above-mentioned embodiments further specifically describe the purposes, technical solutions and advantages of the application, and it should be understood that the above-mentioned embodiments are only preferred embodiments of the application and are not used to limit the application, and any modification, equivalent replacement, improvement, etc. made to the application within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. A low-light video fast enhancement method based on dark channel prior and motion estimation, characterized in that, The method comprises the following steps: obtaining an original low-illumination video, the original low-illumination video comprising a plurality of original image groups of the same size, each original image group comprising a plurality of continuous low-illumination video frames; performing image pixel inversion on all video frames in the original image group to obtain a pseudo-fog image group; performing improved adaptive multi-scale minimum value filtering on the pseudo-fog image group to obtain a dark channel image group; the improved adaptive multi-scale minimum value filtering on the pseudo-fog image group comprises: obtaining the maximum value, the minimum value and the average value of the pixel values in the initial filtering window of the pseudo-fog image group; if the difference between the maximum value and the minimum value in the initial filtering window is greater than the difference between the average value and the minimum value by a preset multiple, the dark channel value of the current filtering window is the minimum pixel value of the initial filtering window; if the difference between the maximum value and the minimum value in the initial filtering window is less than or equal to the difference between the average value and the minimum value by a preset multiple, minimum value filtering is performed in the current filtering window in a multi-scale small window manner; if the difference between the maximum value and the minimum value in the multi-scale small window is greater than the difference between the average value and the minimum value by a preset multiple, the dark channel value in the current multi-scale small window is the minimum pixel value of the multi-scale small window; if the difference between the maximum value and the minimum value in the multi-scale small window is less than or equal to the difference between the average value and the minimum value by a preset multiple, the dark channel value in the current multi-scale small window is the average pixel value of the multi-scale small window; the multi-scale small window is a plurality of small windows of different scales smaller than the initial filtering window, wherein each small window of a different scale is an integer multiple of a minimum window scale, and the minimum window scale is 2; performing transmittance block calculation on an initial image frame in the dark channel image group to obtain the transmittance of the initial image frame of the dark channel image group; the process of the transmittance block calculation comprises: determining the image gradient of a target image frame in the dark channel image group by using a Laplacian operator; determining the pixel smooth area of the target image frame by using an image smooth area threshold constraint; determining the pixel point set of the pixel smooth area by using an image smooth area key pixel constraint; determining the atmospheric light value of the target image frame by using a median filter function to process the pixel point set of the pixel smooth area; calculating the transmittance of the target image frame in the dark channel image group based on an atmospheric physical scattering model according to the atmospheric light value of the target image frame; performing transmittance matching or transmittance block calculation on the adjacent image frame pixel blocks of the dark channel image group by using absolute error and block constraints to obtain the transmittance of each adjacent image frame of the dark channel image group; stretching the transmittance of each image frame of the dark channel image group to obtain the defogging result of each image frame of the dark channel image group; performing image pixel inversion on the defogging result of each image frame of the dark channel image group to obtain an enhanced image video of the original low-illumination video. 2.The low-light video fast enhancement method based on dark channel prior and motion estimation according to claim 1, characterized in that, The image smooth area key pixel constraint is a constraint on the pixel points with a high ranking in the pixel smooth area. 3.The low-light video fast enhancement method based on dark channel prior and motion estimation of claim 1, wherein, The absolute error sum and block constraint are performed on adjacent image frame pixel blocks of the dark channel image group, transmittance matching or transmittance block calculation is performed, and transmittances of each adjacent image frame of the dark channel image group are obtained. If an absolute error sum of pixel values between a current video frame and a previous video frame of the dark channel image group is less than or equal to a preset threshold, transmittance matching is performed on a transmittance of the current video frame, and the transmittance of the current video frame is a transmittance of a pixel block at a corresponding position of the previous video frame. If the absolute error sum of pixel values between the current video frame and the previous video frame of the dark channel image group is greater than the preset threshold, transmittance block calculation is performed on the transmittance of the current video frame, and the transmittance of the current video frame is obtained by the transmittance block calculation.

4. The low-light video fast enhancement method based on dark channel prior and motion estimation according to claim 3, characterized in that, The preset threshold is determined by content, quality and a pixel block size of the original low-illumination video.

5. The low-light video fast enhancement method based on dark channel prior and motion estimation according to claim 1, characterized in that, The transmittances of each image frame of the dark channel image group are stretched to obtain a defogging result of each image frame of the dark channel image group, including: the transmittances of each image frame of the dark channel image group are nonlinearly stretched to obtain the defogging result of each image frame of the dark channel image group; if the transmittance of a target image frame of the dark channel image group is in a first range interval, a smoothing factor of corresponding nonlinear stretching is twice the transmittance of the target image frame; and if the transmittance of the target image frame of the dark channel image group is in a second range interval, the smoothing factor of corresponding nonlinear stretching is a fixed value 1.

6. A low-light video fast enhancement system based on dark channel prior and motion estimation, characterized in that, The system comprises: a video acquisition unit configured to acquire an original low-illumination video, the original low-illumination video comprising a plurality of original image groups of the same size, each original image group comprising a plurality of continuous low-illumination video frames; a first processing unit configured to perform image pixel inversion on all video frames in the original image group to obtain a pseudo-fog image group; a second processing unit configured to perform improved adaptive multi-scale minimum value filtering on the pseudo-fog image group to obtain a dark channel image group; performing improved adaptive multi-scale minimum value filtering on the pseudo-fog image group comprises: obtaining a maximum value, a minimum value and an average value of pixel values in an initial filtering window of the pseudo-fog image group; if a difference between the maximum value and the minimum value in the initial filtering window is greater than a preset multiple of a difference between the average value and the minimum value, a dark channel value of a current filtering window is the minimum value of pixels in the initial filtering window; if the difference between the maximum value and the minimum value in the initial filtering window is less than or equal to the preset multiple of the difference between the average value and the minimum value, minimum value filtering is performed in the current filtering window in a multi-scale small window manner; if a difference between the maximum value and the minimum value in a multi-scale small window is greater than a preset multiple of a difference between an average value and the minimum value, a dark channel value in a current multi-scale small window is a minimum value of pixels in the multi-scale small window; if the difference between the maximum value and the minimum value in the multi-scale small window is less than or equal to the preset multiple of the difference between the average value and the minimum value, the dark channel value in the current multi-scale small window is an average value of pixels in the multi-scale small window. The multi-scale small window is a plurality of small windows of different scales smaller than the initial filtering window, wherein each small window of different scale is an integer multiple of a minimum window scale, and the minimum window scale is 2; The third processing unit is configured to perform transmittance block calculation on an initial image frame in the dark channel image group to obtain transmittance of the initial image frame in the dark channel image group; The transmittance block calculation process comprises: An image gradient of a target image frame in the dark channel image group is determined by using a Laplacian operator; A pixel smooth region of the target image frame is determined by using an image smooth region threshold constraint; A pixel point set of the pixel smooth region is determined by using an image smooth region key pixel constraint; An atmospheric light value of the target image frame is determined by processing the pixel point set of the pixel smooth region by using a median filter function; Transmittance of the target image frame in the dark channel image group is calculated based on an atmospheric physical scattering model and the atmospheric light value of the target image frame; The fourth processing unit is configured to perform absolute error and block constraint on a pixel block of adjacent image frames in the dark channel image group, and perform transmittance matching or transmittance block calculation to obtain transmittance of each adjacent image frame in the dark channel image group; The fifth processing unit is configured to stretch the transmittance of each image frame in the dark channel image group to obtain a defogging result of each image frame in the dark channel image group; The sixth processing unit is configured to perform image pixel inversion on the defogging result of each image frame in the dark channel image group to obtain an enhanced image video of the original low-illumination video.

Images