A method for controlling a UAV based on real night image defogging

Abstract

The application provides a kind of unmanned aerial vehicle control method based on real night image defogging, including the pre-processing of night fog image, the preliminary defogging of image after pre-processing is carried out using dark channel prior method, obtain a plurality of intermediate results.Based on image contrast, chroma, saliency and color contrast, a scoring model is established.Each intermediate result is input into the scoring model for scoring to obtain the final score of the kth intermediate result of the ith frame of night fog image.According to the final score, generate the binary weight of each intermediate result.According to the binary weight, all intermediate results are fused using a multi-scale fusion strategy.The fusion result is post-processed to obtain the final defogging image.The binary weight is used to selectively fuse the maximum information part of each pixel point of each intermediate result.The multi-scale fusion strategy is used to prevent the generation of artifacts while maintaining high contrast, improving the success rate and efficiency of unmanned aerial vehicle in executing night scene tasks.

Description

Technical Field

[0001] This invention relates to the fields of image dehazing and drone technology, and in particular to a drone control method based on real nighttime image dehazing. Background Technology

[0002] Images captured in real-world nighttime haze environments are often affected by various degradation factors, exhibiting issues such as blurred details, color shifts, and low light. However, existing physical model-based methods struggle to adequately describe the complex degradation phenomena of real-world nighttime scenes, as well as the combined effects of these factors. Deep learning-based methods are limited by their lack of effective generalization to nighttime haze scenes, making them difficult to apply directly to real-world nighttime environments. Therefore, there is an urgent need for an algorithm that does not rely on physical models or large amounts of data to effectively improve visibility in real-world nighttime environments.

[0003] In recent years, drones have been widely used in many fields, such as power line inspection, nighttime security patrol and disaster search and rescue. Nighttime haze and smog have a dual impact on drone operations. First, insufficient natural light results in low image brightness. Second, suspended particles scatter the drone's supplementary lighting or ambient light sources, making the image more blurred, leading to decreased contrast and color shift, which seriously affects the drone's performance in nighttime scenarios. (1) Power Line Inspection: In power line inspection, drones need to monitor power lines and equipment at night. Nighttime haze and smog reduce image clarity, making it more difficult to detect defects and faults in power equipment. Dehazing algorithms can improve image visibility, helping operators to more accurately identify potential problems, thereby improving the efficiency and safety of power line inspection. (2) Nighttime Security Patrol: Nighttime security patrol is one of the important applications of drones. In haze and smog, images taken by drones may not be able to clearly identify targets due to low visibility, which poses a challenge to security patrol tasks. By applying nighttime dehazing algorithms, image visibility can be enhanced, enabling operators to more effectively monitor and identify potential threats, ensuring the effectiveness of nighttime security patrols. (3) Disaster Search and Rescue: In nighttime search and rescue operations following a disaster, drones can quickly reach the scene and provide real-time images. However, nighttime fog and haze can severely affect image quality, hindering rescuers from assessing the affected area and locating targets. Defogging algorithms can improve image clarity, helping rescue teams more accurately identify trapped individuals and dangerous areas, thereby increasing the success rate and efficiency of disaster search and rescue.

[0004] The limitations of existing defogging technologies for drone operations in nighttime foggy environments are mainly reflected in two aspects: hardware and software. Hardware aspects: (1) Limited sensor performance: In foggy weather, air particles scatter and absorb light, resulting in reduced contrast and color distortion of drone camera images; insufficient light at night further exacerbates the problem and affects the defogging effect. (2) Insufficient endurance: Some defogging technologies increase energy consumption, while drones have limited endurance. The mission time is extended in nighttime foggy environments, and insufficient endurance may limit the operation range and time. (3) Payload limitation: To improve the defogging effect, it is necessary to add hardware such as filters and lighting devices, but drones have limited payload, and increasing weight may affect flight performance and stability. Software aspects: (1) Insufficient algorithm adaptability: Existing defogging algorithms are mostly based on daytime foggy models, which are poorly adaptable to complex nighttime lighting and foggy characteristics, resulting in blurry and noisy images after defogging, affecting target recognition. (2) Computational resource limitation: Advanced defogging algorithms require high computing power, but the computing resources configured on drones are limited, and the speed is slow when processing large amounts of image data in real time, making it difficult to meet the needs of real-time applications. (3) Data fusion difficulties: In the nighttime haze environment, it is necessary to fuse data from multiple sensors. However, due to the large differences in the characteristics of different sensor data, the existing fusion algorithms are not mature enough and it is difficult to give full play to the advantages of each sensor.

[0005] Therefore, a real nighttime image dehazing algorithm is needed that can comprehensively measure multiple image features of nighttime haze images, generate multiple intermediate results, select the most informative part of each intermediate result for fusion, enhance the texture in the image, optimize the image edges, and adjust the image saturation. Based on this algorithm, a drone can be controlled to perform nighttime scene tasks. Summary of the Invention

[0006] To overcome the problems existing in related technologies, the purpose of this invention is to provide a drone control method based on real nighttime image dehazing. This control method designs a real nighttime image dehazing algorithm. This algorithm can comprehensively measure multiple image features of nighttime haze images and generate multiple intermediate results. The most informative part of each intermediate result is selected for fusion. At the same time, the texture in the image is enhanced, the image edge is optimized, the image saturation is adjusted, and the drone is controlled to perform nighttime scene tasks based on this algorithm.

[0007] A drone control method based on dehazing of real nighttime images includes:

[0008] Acquire nighttime haze images, and preprocess the nighttime haze images to obtain preprocessed images;

[0009] The dark channel prior method is used to perform preliminary dehazing on the preprocessed image, resulting in several intermediate results;

[0010] A scoring model is established based on image contrast, chromaticity, saliency, and color contrast.

[0011] Each intermediate result is input into the scoring model for scoring, and the final score of the k-th intermediate result of the i-th frame of the nighttime haze image is obtained;

[0012] Based on the final score, generate binary weights for each of the intermediate results;

[0013] Based on the binary weights, a multi-scale fusion strategy is used to fuse all the intermediate results to obtain the fusion result;

[0014] The fusion result is post-processed to obtain the final dehazed image;

[0015] The final dehazed image is sent to the drone, which is then controlled to perform nighttime scene tasks.

[0016] In a preferred embodiment of the present invention, the step of preprocessing the nighttime haze image to obtain a preprocessed image includes:

[0017] Different nonlinear correction coefficients are set, and gamma correction is performed on the nighttime haze image using the following formula:

[0018]

[0019] in, Let I(i) be the k-th preprocessed image of the i-th nighttime haze image, and γ be the i-th nighttime haze image. k is the k-th nonlinear correction coefficient.

[0020] In a preferred embodiment of the present invention, the dark channel prior method is used to perform preliminary dehazing on the preprocessed image to obtain multiple intermediate results, including:

[0021] The following formula is used to perform preliminary dehazing on the preprocessed image:

[0022]

[0023] in, Let represent the k-th intermediate result of the i-th frame of the nighttime haze image, and DCP represent the haze removal operation based on the dark channel prior. For the k-th preprocessed image of the i-th frame of the nighttime haze image, γ k is the k-th nonlinear correction coefficient.

[0024] In a preferred embodiment of the present invention, establishing a scoring model based on image contrast, chroma, saliency, and color contrast includes:

[0025] Calculate image contrast, chromaticity, saliency, and color contrast using the following formulas:

[0026]

[0027] in, Let be the image contrast of the kth intermediate result of the i-th frame of the nighttime haze image. Let be the chromaticity of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance of the k-th intermediate result of the i-th frame of the nighttime haze image. The color contrast of the kth intermediate result of the i-th frame of the nighttime haze image; This represents the k-th intermediate result of the i-th frame of the nighttime haze image. express The second derivative in the x-direction, express The second derivative in the y-direction; c represents the color channel, r represents the red channel, g represents the green channel, and b represents the blue channel. This represents the color channel of the k-th intermediate result of the i-th frame of the nighttime haze image; This represents the average of the three color channels of the k-th intermediate result of the i-th frame of the nighttime haze image. This represents the grayscale value of the k-th intermediate result of the i-th frame of the nighttime haze image. This represents the blur value of the kth intermediate result of the i-th frame of the nighttime haze image; This represents the hue of the k-th intermediate result in the HSV space for the i-th frame of the nighttime haze image. Let represent the saturation of the k-th intermediate result of the i-th frame of the nighttime haze image in the HSV space, θ represent the color contrast multiplicative parameter, Φ represent the color contrast additive parameter, and FFT represent the fast Fourier transform.

[0028] A scoring model is established by using the image contrast, chromaticity, saliency, and color contrast as independent variables, and the final score of the kth intermediate result of the i-th frame of the nighttime haze image as the dependent variable.

[0029] In a preferred embodiment of the present invention, the step of inputting each intermediate result into a scoring model for scoring to obtain the final score of the k-th intermediate result of the i-th frame of the nighttime haze image includes:

[0030] Input the kth intermediate result into the scoring model to calculate the corresponding image contrast, chroma, saliency and color contrast;

[0031] Calculate the image contrast weight, chromaticity weight, saliency weight, and color contrast weight;

[0032] The image contrast, chromaticity, saliency, and color contrast are weighted and summed to obtain the final score of the kth intermediate result of the i-th frame of the nighttime haze image.

[0033] In a preferred embodiment of the present invention, the calculation of image contrast weight, chromaticity weight, saliency weight, and color contrast weight includes:

[0034] Calculate the image contrast weight using the following formula:

[0035]

[0036] in, Let be the mean contrast of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the contrast standard deviation of the k-th intermediate result of the i-th frame of the nighttime haze image, and α be the first adjustment parameter. The image contrast weight of the kth intermediate result of the i-th frame of the nighttime haze image;

[0037] Calculate the chromaticity weights using the following formula:

[0038]

[0039] Where exp represents the exponential function, β is the second adjustment parameter, and ε is the third adjustment parameter. Let be the chromaticity of the k-th intermediate result of the i-th frame of the nighttime haze image. The chromaticity weight is the k-th intermediate result of the i-th frame of the nighttime haze image, and max indicates the operation of taking the maximum value;

[0040] The significance weight is calculated using the following formula:

[0041]

[0042] in, Let be the significance mean of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance standard deviation of the k-th intermediate result of the i-th frame of the nighttime haze image. This is the fourth adjustment parameter. The significance weight of the k-th intermediate result of the i-th frame of the nighttime haze image;

[0043] Calculate the color contrast weight using the following formula:

[0044]

[0045] in, This represents the hue of the k-th intermediate result in the HSV space for the i-th frame of the nighttime haze image. This represents the entropy value of the k-th intermediate result of the i-th frame of the nighttime haze image in the HSV space, representing the hue. The color contrast weights are the k-th intermediate results of the i-th frame of the nighttime haze image.

[0046] In a preferred embodiment of the present invention, generating binary weights for each intermediate result based on the final score includes:

[0047] The following formula is used to search for the optimal intermediate result corresponding to each pixel in the nighttime haze image based on the final score:

[0048]

[0049] Where τ(row,col) represents the index of the optimal intermediate result of the pixel in the row-th row and column-th column of the nighttime haze image, K is the total number of intermediate results, k is the sequence number of the intermediate result, argmax represents the sequence number of the intermediate result corresponding to the maximum value of the final search score, and S k (row,col) represents the final score of the pixel in the row-th row and column-th column of the k-th intermediate result;

[0050] The binary weights for each of the intermediate results are generated according to the following formula:

[0051]

[0052] Among them, W τ(row,col) (row,col) represents the binary weight of the optimal intermediate result corresponding to the pixel in the row-th row and column-th column of the nighttime haze image, W. p (row,col) represents the binary weight of the non-optimal intermediate result corresponding to the pixel in row 'row' and column 'col' of the nighttime haze image, and p represents the index of the non-optimal intermediate result of the pixel in row 'row' and column 'col' of the nighttime haze image.

[0053] In a preferred embodiment of the present invention, the step of fusing all the intermediate results using a multi-scale fusion strategy to obtain a fusion result includes:

[0054] The following formula is used to fuse all the intermediate results:

[0055]

[0056] Among them, G l L represents the Gaussian pyramid. l This refers to the Pyramid of Laplace, where Ltotal represents the total number of levels in the pyramid, and l represents the pyramid's index. R represents the k-th intermediate result of the i-th frame of the nighttime haze image.initial (i) represents the fusion result of the i-th frame of nighttime haze image, W k (i) represents the binary weight of the kth intermediate result of the i-th frame of the nighttime haze image.

[0057] In a preferred embodiment of the present invention, the post-processing of the fusion result to obtain the final dehazed image includes:

[0058] Non-subsampled shear wave transform is used to separate the high-frequency and low-frequency components of the brightness channel of the fusion result;

[0059] Non-subsampled shear wave transform is used to separate the high-frequency and low-frequency components of the brightness channel in the nighttime haze image;

[0060] The high-frequency components of the brightness channel of the nighttime haze image are fused with the high-frequency components of the brightness channel of the fusion result to obtain the fused high-frequency features.

[0061] The fused high-frequency features are subjected to an inverse non-subsampled shear plate transform to obtain the enhanced brightness after texture enhancement;

[0062] By combining a smoothing and sharpening filter and a guided filter, the edge portion of the brightness channel of the nighttime haze image is extracted; the edge portion and the brightness after texture enhancement are fused to obtain the edge-optimized brightness.

[0063] The saturation channel of the fusion result is adjusted using nonlocal adaptive saturation to obtain the final dehazed image.

[0064] In a preferred embodiment of the present invention, sending the final dehazed image to the drone and controlling the drone to perform a night scene task includes:

[0065] The drone is controlled to perform power line inspection based on the final defogging image in order to detect defects in the power equipment;

[0066] The drone is controlled to conduct nighttime security patrols based on the final defogging image in order to identify target personnel in the scene;

[0067] The drone is controlled to conduct disaster search and rescue operations based on the final defogging image to identify trapped personnel and dangerous areas in the scene.

[0068] The beneficial effects of this invention are as follows:

[0069] The UAV control method based on real nighttime image dehazing provided by this invention includes acquiring nighttime haze images, preprocessing the nighttime haze images to obtain preprocessed images. After preprocessing, the brightness of low-light areas in the nighttime haze images is improved, or some details in the haze areas are restored. A dark channel prior method is used to perform preliminary dehazing on the preprocessed images, obtaining multiple intermediate results, thereby further improving the clarity of the preprocessed images. A scoring model is established based on image contrast, chroma, saliency, and color contrast. Image contrast, chroma, saliency, and color contrast have high correlation with nighttime haze; combining these four indicators can comprehensively reflect the characteristics of nighttime haze. Each intermediate result is input into the scoring model for scoring, obtaining the final score of the k-th intermediate result of the i-th frame of the nighttime haze image. Based on the final score, binary weights are generated for each intermediate result. Binary weights are assigned to each intermediate result according to the final score, and the maximum information content of each pixel in each intermediate result is selectively fused using binary weights. Based on binary weights, a multi-scale fusion strategy is employed to fuse all intermediate results, yielding a fused image to prevent artifacts and maintain high contrast and detail. Post-processing is then applied to the fused image to obtain the final dehazed image. Post-processing includes texture enhancement, edge optimization, and saturation adjustment, resulting in a high-quality final dehazed image. This final dehazed image is then sent to a drone to perform nighttime scene tasks. Compared to the original nighttime haze image, the visibility and clarity of the final dehazed image are significantly improved, increasing the success rate and efficiency of the drone in performing nighttime scene tasks. Attached Figure Description

[0070] Figure 1 This is a flowchart of the UAV control method based on real nighttime images according to the present invention;

[0071] Figure 2 This is a flowchart of the post-processing of the fusion results according to the present invention;

[0072] Figure 3 This is a comparison chart of the gamma correction results using different gamma correction parameters according to the present invention. Detailed Implementation

[0073] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0074] Example 1

[0075] like Figure 1 As shown, this embodiment provides a drone control method based on dehazing of real nighttime images, including:

[0076] S1: Acquire a nighttime haze image, preprocess the nighttime haze image, and obtain a preprocessed image.

[0077] S2: The dark channel prior method is used to perform preliminary dehazing on the preprocessed image to obtain multiple intermediate results.

[0078] S3: Establish a scoring model based on image contrast, chromaticity, saliency, and color contrast.

[0079] S4: Input each intermediate result into the scoring model for scoring, and obtain the final score of the kth intermediate result of the i-th frame of the nighttime haze image.

[0080] S5: Generate binary weights for each of the intermediate results based on the final score.

[0081] S6: Based on the binary weights, a multi-scale fusion strategy is used to fuse all the intermediate results to obtain the fusion result.

[0082] S7: Post-process the fusion result to obtain the final dehazed image.

[0083] S8: Send the final dehazed image to the drone and control the drone to perform night scene tasks.

[0084] Nighttime haze images reflect real nighttime scenes with haze. These images are affected by various degradation factors, such as blurred details and textures caused by haze, low brightness in areas without lights (i.e., low light phenomenon), and color shift.

[0085] The preprocessing of the nighttime haze image to obtain a preprocessed image includes:

[0086] Different nonlinear correction coefficients are set, and gamma correction is performed on the nighttime haze image using the following formula:

[0087]

[0088] in, Let I(i) be the k-th preprocessed image of the i-th nighttime haze image, and γ be the i-th nighttime haze image. k is the k-th nonlinear correction coefficient.

[0089] To obtain high-quality intermediate images, gamma correction is used for brightness enhancement and detail restoration. Let 'i' represent the frame number. Using k non-linear correction coefficients, gamma correction is performed k times on the i-th frame of the nighttime haze image, resulting in k pre-processed images. These k pre-processed images are for the same scene, but each selectively highlights different image regions within the nighttime haze scene. In this example, k=10, meaning 10 different non-linear correction coefficients are used to perform gamma correction on the i-th frame of the nighttime haze image, resulting in 10 pre-processed images. Figure 3 As shown, Figure 3 The first line in γ k <1 The preprocessed image generated, the second row is γ k >1 is the inverted version of the preprocessed image generated, with the third row being a daytime haze image. Figure 3 Columns 1, 3, and 5 in the histogram represent real-world scene images with haze. Column 2 is the histogram of the real-world scene images in column 1, column 4 is the histogram of the real-world scene images in column 3, and column 6 is the histogram of the real-world scene images in column 5. In the histograms, red represents the pixel value distribution of the R channel, green represents the pixel value distribution of the G channel, and blue represents the pixel value distribution of the B channel. Gamma correction is a global non-linear operation; when γ... k When γ > 1, while it can effectively restore details in hazy areas, it exacerbates the brightness degradation problem in low-light areas. Similarly, when γ k When the value is less than 1, although it can effectively improve the brightness of low-light areas, it cannot restore the details of hazy areas. This invention discovered that γ... k >1 The inverted version of the generated image and γ k The generated result has similar characteristics to daytime haze scenes. Based on this characteristic, the present invention utilizes daytime dehaze operation based on dark channel prior to further improve the clarity of the preprocessed image.

[0090] The dark channel prior method is used to perform preliminary dehazing on the preprocessed image, resulting in several intermediate results, including:

[0091] The following formula is used to perform preliminary dehazing on the preprocessed image:

[0092]

[0093] in, Let represent the k-th intermediate result of the i-th frame of the nighttime haze image, and DCP represent the haze removal operation based on the dark channel prior. For the k-th preprocessed image of the i-th frame of the nighttime haze image, γ k is the k-th nonlinear correction coefficient.

[0094] The dark channel prior dehazing algorithm is based on an atmospheric scattering model, which removes haze from images by estimating atmospheric light and the image's transmission map. The theoretical basis of the dark channel prior dehazing algorithm is that in haze-free images of outdoor non-sky areas, a large number of local pixels have extremely low pixel values ​​in at least one color channel, with pixel values ​​approaching 0.

[0095] The steps of the dark channel prior dehazing algorithm include: (1) Dark channel extraction: First, extract the dark channel of the preprocessed image. The dark channel refers to the region with low pixel value in the preprocessed image. These regions can still maintain a certain visibility in the presence of haze. (2) Atmospheric light estimation: Use the dark channel to estimate the value of atmospheric light. Select the pixel value in the dark channel that is greater than a set threshold as the estimated value of atmospheric light. (3) Transmission map estimation: The transmission map refers to the light transmission rate of each pixel in the preprocessed image through haze. Combine the estimated value of atmospheric light and the dark channel to estimate the transmission map. (4) Calculate the i-th intermediate result according to the following formula:

[0096]

[0097] Where A is the estimated value of atmospheric light. Let t(i) be the k-th intermediate result of the i-th frame of the nighttime haze image, t(i) be the i-th transmittance, and t0 be the preset transmittance. This is the k-th preprocessed image of the i-th nighttime haze image.

[0098] In the multi-scale fusion stage, the present invention first uses gamma correction and dark channel prior method to generate a set of intermediate results with improved visibility and brightness. The intermediate result is an image with the same size as the nighttime haze image.

[0099] This embodiment provides a UAV control method based on real nighttime image dehazing, which includes acquiring nighttime haze images, preprocessing the nighttime haze images to obtain preprocessed images. After preprocessing, the brightness of low-light areas in the nighttime haze images is improved, or some details in the haze areas are restored. A dark channel prior method is used to perform preliminary dehazing on the preprocessed images, obtaining multiple intermediate results, thereby further improving the clarity of the preprocessed images. A scoring model is established based on image contrast, chroma, saliency, and color contrast. Image contrast, chroma, saliency, and color contrast have high correlation with nighttime haze; combining these four indicators can comprehensively reflect the characteristics of nighttime haze. Each intermediate result is input into the scoring model for scoring, obtaining the final score of the k-th intermediate result of the i-th frame of the nighttime haze image. Binary weights are generated for each intermediate result based on the final score. Binary weights are assigned to each intermediate result based on the final score, and the maximum information content of each pixel in each intermediate result is selectively fused using these binary weights. Based on binary weights, a multi-scale fusion strategy is employed to fuse all intermediate results, yielding a fused image to prevent artifacts and maintain high contrast and detail. Post-processing is then applied to the fused image to obtain the final dehazed image. Post-processing includes texture enhancement, edge optimization, and saturation adjustment, resulting in a high-quality final dehazed image. This final dehazed image is then sent to a drone to perform nighttime scene tasks. Compared to the original nighttime haze image, the visibility and clarity of the final dehazed image are significantly improved, increasing the success rate and efficiency of the drone in performing nighttime scene tasks.

[0100] Example 2

[0101] This embodiment provides a drone control method based on dehazing of real nighttime images. This embodiment only describes the differences from Embodiment 1. The method for establishing a scoring model based on image contrast, chroma, saliency, and color contrast includes:

[0102] Calculate image contrast, chromaticity, saliency, and color contrast using the following formulas:

[0103]

[0104] in, Let be the image contrast of the kth intermediate result of the i-th frame of the nighttime haze image. Let be the chromaticity of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance of the k-th intermediate result of the i-th frame of the nighttime haze image. The color contrast of the kth intermediate result of the i-th frame of the nighttime haze image; This represents the k-th intermediate result of the i-th frame of the nighttime haze image. express The second derivative in the x-direction, express The second derivative in the y-direction; c represents the color channel, r represents the red channel, g represents the green channel, and b represents the blue channel. This represents the color channel of the k-th intermediate result of the i-th frame of the nighttime haze image; This represents the average of the three color channels of the k-th intermediate result of the i-th frame of the nighttime haze image. This represents the grayscale value of the k-th intermediate result of the i-th frame of the nighttime haze image. This represents the blur value of the kth intermediate result of the i-th frame of the nighttime haze image; This represents the hue of the k-th intermediate result in the HSV space for the i-th frame of the nighttime haze image. θ represents the saturation of the k-th intermediate result of the i-th frame of the nighttime haze image in the HSV space; θ represents the color contrast multiplicative parameter, Φ represents the color contrast additive parameter, FFT represents the Fast Fourier Transform, and . in this invention represents the multiplication operation;

[0105] The hue range is 0-360°, but the actual intermediate hue is generally smaller than this range, for example, 40-170°. That is, the hue of the intermediate result is not symmetrical. Therefore, Fast Fourier Transform is used to process the hue after multiplication and addition operations. Fast Fourier Transform decomposes the processed hue into a series of sine and cosine functions of different frequencies. Compared with using only cosine functions, this invention uses Fast Fourier Transform, which can adjust the hue more finely in different frequency bands and is applicable to hues that do not have symmetry.

[0106] A scoring model is established by using the image contrast, chromaticity, saliency, and color contrast as independent variables, and the final score of the kth intermediate result of the i-th frame of the nighttime haze image as the dependent variable.

[0107] To selectively fuse the most informative parts of each pixel location, this invention utilizes four haze-related features—image contrast, chroma, saliency, and color contrast—to establish a scoring mechanism for filtering intermediate results. Image contrast reflects the image edges of the intermediate results in the x and y directions. The second derivative has better response characteristics to fine details such as thin lines and isolated points. The second derivative produces a double-edge effect in areas with large gradients in the intermediate results; the sign of the second derivative determines whether the edge transition of the intermediate results is from bright to dark or from dark to bright. Chroma reflects the distribution of pixel values ​​in the R, G, and B channels of the intermediate results, i.e., the differences between different image regions of the intermediate results. The higher the chroma value, the more uneven the distribution of pixel values ​​in the three color channels. Saliency is determined by first blurring the intermediate results, such as through low-pass filtering, to remove high-frequency components like image noise. Then, by subtraction and L2 calculation, it detects whether there are regions with high or low pixel values ​​in the intermediate results. If such regions exist, the intermediate results are considered more saliency-rich. Color contrast is achieved by performing a fast Fourier transform on the hue of the intermediate result, and then combining the saturation and grayscale values ​​of the intermediate result to make an overall evaluation of the color of the intermediate result.

[0108] The k-th intermediate result of the i-th frame of the nighttime haze image is blurred to remove high-frequency components, such as image noise. This invention employs a 5×5 separable binomial convolution kernel and sets a high-frequency cutoff value to remove components with frequencies higher than the high-frequency cutoff value from the k-th intermediate result of the i-th frame of the nighttime haze image, thus performing low-pass filtering on the k-th intermediate result of the i-th frame of the nighttime haze image.

[0109] The weights of the binomial convolution kernel are Set the high-frequency cutoff value to It is the kth intermediate result of the i-th frame of the nighttime haze image. The image is obtained by converting a color image to a grayscale image. In this embodiment, the color contrast multiplicative parameter θ is set to 2 and the color contrast additive parameter Φ is set to 250°.

[0110] The step of inputting each intermediate result into the scoring model for scoring to obtain the final score of the k-th intermediate result of the i-th frame of the nighttime haze image includes:

[0111] S41: Input the kth intermediate result into the scoring model to calculate the corresponding image contrast, chromaticity, saliency, and color contrast.

[0112] S42: Calculate the image contrast weight, chromaticity weight, saliency weight, and color contrast weight.

[0113] S43: The image contrast, chromaticity, saliency, and color contrast are weighted and summed to obtain the final score of the kth intermediate result of the i-th frame of the nighttime haze image.

[0114] Image contrast refers to the different brightness levels between the brightest white and darkest black areas of an image. The greater the difference between the minimum and maximum pixel values, the wider the range of pixel value differences. Chroma is expressed in angles, ranging from 0 to 360°, with red at 0°, green at 120°, and blue at 240°. Chroma determines the appearance and characteristics of a color. and The significance of the kth intermediate result of the i-th frame of the nighttime haze image is measured by the difference between the values ​​of the hue and the grayscale values. The color contrast of the kth intermediate result of the i-th frame of the nighttime haze image is calculated by performing a fast Fourier transform using the hue in the HSV color space and combining it with the saturation and grayscale values ​​in the HSV color space.

[0115] The calculation of image contrast weight, chromaticity weight, saliency weight, and color contrast weight includes:

[0116] Calculate the image contrast weight using the following formula:

[0117]

[0118] in, Let be the mean contrast of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the contrast standard deviation of the k-th intermediate result of the i-th frame of the nighttime haze image, and α be the first adjustment parameter. The image contrast weight of the kth intermediate result of the i-th frame of the nighttime haze image;

[0119] Calculate the chromaticity weights using the following formula:

[0120]

[0121] Where exp represents the exponential function, β is the second adjustment parameter, and ε is the third adjustment parameter. Let be the chromaticity of the k-th intermediate result of the i-th frame of the nighttime haze image. Let represent the chromaticity weight of the k-th intermediate result of the i-th frame of the nighttime haze image, where max indicates that the maximum value is taken during the operation. The maximum value of the chromaticity of the kth intermediate result of the i-th frame of the nighttime haze image;

[0122] The significance weight is calculated using the following formula:

[0123]

[0124] in, Let be the significance mean of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance standard deviation of the k-th intermediate result of the i-th frame of the nighttime haze image. This is the fourth adjustment parameter. The significance weight of the k-th intermediate result of the i-th frame of the nighttime haze image;

[0125] Calculate the color contrast weight using the following formula:

[0126]

[0127] in, This represents the hue of the k-th intermediate result in the HSV space for the i-th frame of the nighttime haze image. This represents the entropy value of the k-th intermediate result of the i-th frame of the nighttime haze image in the HSV space, representing the hue. The color contrast weights are the k-th intermediate results of the i-th frame of the nighttime haze image.

[0128] This invention employs a weighted summation method, multiplying the contrast weight by the contrast, the chroma weight by the chroma, the saliency weight by the saliency, and the color contrast weight by the color contrast. Finally, the four products are summed to obtain the final score of the kth intermediate result of the i-th frame of the nighttime haze image.

[0129] The contrast weight uses the sigmoid function, which has the advantages of smoothness and easy differentiation. The sigmoid function is linear near zero. When the object affected by the sigmoid function is too large or too small, it tends to level off, meaning the amplitude of change decreases significantly, thus preventing the value from being too small or too large and constraining the contrast weight within a certain range. In this embodiment, the first adjustment parameter α is set to 3. First, the image contrast of the k-th intermediate result is normalized, and then substituted into the sigmoid function to prevent the image contrast weight from being too small or too large, and to ensure that the image contrast weight has monotonically increasing and linear characteristics.

[0130] The chromaticity weight is determined by dividing the chromaticity of each pixel in the kth intermediate result of the i-th frame of the nighttime haze image by the maximum chromaticity of the kth intermediate result of the i-th frame of the nighttime haze image, and combining the chromaticity ratio and an exponential function to constrain the chromaticity. In this embodiment, the second adjustment parameter β is set to 4, and the third adjustment parameter ε is a small positive value, such as 0.01, to prevent the denominator from being zero during the calculation of the chromaticity weight. The independent variable of the exponential function of the chromaticity weight is negative, which gives the chromaticity weight a non-linear decay characteristic. That is, when the chromaticity of the kth intermediate result is fixed, due to... The value of is between 0 and 1, therefore, The larger, The larger.

[0131] The significance mean and significance standard deviation are used to constrain the significance of the k-th intermediate result of the i-th frame of the nighttime haze image. When the significance of the k-th intermediate result of the i-th frame of the nighttime haze image is large, the corresponding significance weight is decreased; when the significance of the k-th intermediate result of the i-th frame of the nighttime haze image is small, the corresponding significance weight is increased. In this embodiment, the fourth adjustment parameter... Set to 1.5. The denominator of the significance weight is the sum of the significance mean and the significance standard deviation. The standard deviation reflects the dispersion of the data, and the mean reflects whether the data distribution is even. By combining the mean and standard deviation, we can assign a greater significance weight to intermediate results with more concentrated significance and a more uniform significance distribution for each pixel, thereby making full use of the significance of the corresponding intermediate results.

[0132] The color contrast weight is calculated by first normalizing the color contrast, which is the color contrast of each point in the kth intermediate result of the i-th frame of the nighttime haze image, divided by the maximum color contrast of the kth intermediate result of the i-th frame of the nighttime haze image. A higher entropy value indicates a higher degree of system disorder. In this embodiment, a higher hue entropy value corresponds to a lower color contrast weight, thus reducing the color contrast of intermediate results with higher disorder. The color contrast weight is obtained by performing division and subtraction operations on the entropy value, and then multiplying it by the ratio of color contrast. A higher hue entropy value and a lower color contrast result in a lower color contrast weight. Combining the hue entropy value and color contrast allows full utilization of the color contrast of intermediate results with lower hue disorder and more uniform color contrast distribution.

[0133] The final score of the kth intermediate result of the i-th frame of the nighttime haze image is calculated using the following formula:

[0134]

[0135] Among them, S k (i) represents the final score of the k-th intermediate result of the i-th frame of the nighttime haze image. The image contrast weight of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the chromaticity weight of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance weight of the k-th intermediate result of the i-th frame of the nighttime haze image. The color contrast weights are the k-th intermediate results of the i-th frame of the nighttime haze image. Let be the image contrast of the kth intermediate result of the i-th frame of the nighttime haze image. Let be the chromaticity of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance of the k-th intermediate result of the i-th frame of the nighttime haze image. is the color contrast of the kth intermediate result of the i-th frame of the nighttime haze image.

[0136] This embodiment inputs each intermediate result into a scoring model for scoring, obtaining the final score of the k-th intermediate result of the i-th frame of the nighttime haze image. This includes inputting the k-th intermediate result into the scoring model and calculating the corresponding image contrast, chroma, saliency, and color contrast. Image contrast weights, chroma weights, saliency weights, and color contrast weights are calculated, and these weighted sums are applied to obtain the final score of the k-th intermediate result of the i-th frame of the nighttime haze image. Image contrast is based on the second derivatives in the x and y directions of each intermediate result; chroma is based on the pixel values ​​of each color channel of each intermediate result; saliency is measured by the difference between the blur value of the intermediate result and the average value of the three color channels; and color contrast is calculated by performing a Fast Fourier Transform on the hue of the intermediate result and then combining it with saturation and grayscale values. By using a weighted summation method, image contrast, chroma, saliency, and color contrast can be constrained simultaneously. The weight of indicators with larger values ​​is reduced. For example, when the weight of contrast is large, the calculation method of this invention can generate a smaller weight to limit the image contrast, chroma, saliency, and color contrast to a certain range, avoiding one or more of them being too large or too small, so that the final score can reflect the image quality of each intermediate result as a whole.

[0137] Example 3

[0138] This embodiment provides a drone control method based on dehazing of real nighttime images. This embodiment only describes the differences from Embodiment 1. The step of generating binary weights for each intermediate result based on the final score includes:

[0139] The following formula is used to search for the optimal intermediate result corresponding to each pixel in the nighttime haze image based on the final score:

[0140]

[0141] Where τ(row,col) represents the index of the optimal intermediate result of the pixel in the row-th row and column-th column of the nighttime haze image, K is the total number of intermediate results, k is the sequence number of the intermediate result, argmax represents the sequence number of the intermediate result corresponding to the maximum value of the final search score, and S k (row,col) represents the final score of the pixel in the row-th row and column-th column of the k-th intermediate result;

[0142] The binary weights for each of the intermediate results are generated according to the following formula:

[0143]

[0144] Among them, W τ(row,col) (row,col) represents the binary weight of the optimal intermediate result corresponding to the pixel in the row-th row and column-th column of the nighttime haze image, W. p (row,col) represents the binary weight of the non-optimal intermediate result corresponding to the pixel in row 'row' and column 'col' of the nighttime haze image, and p represents the index of the non-optimal intermediate result of the pixel in row 'row' and column 'col' of the nighttime haze image.

[0145] The nighttime haze image and each intermediate result have the same size. Each pixel in the nighttime haze image is determined by one of the K intermediate results. This invention first calculates the final score for each intermediate result. The final score of an intermediate result is a matrix with the same size as the intermediate result. Then, the final score of each pixel in each intermediate result is calculated. Each pixel in the nighttime haze image is selected sequentially: first, the pixel in the 1st row and 1st column is selected; then, the pixel in the 1st row and 2nd column is selected; finally, the pixel in the Mth row and Nth column is selected, where M is the total number of rows and N is the total number of columns in the nighttime haze image. From the K intermediate results, the maximum final score in the row-1, column-1 is selected, and its corresponding binary weight is set to 1. The binary weights of the other intermediate results are set to 0. For example, for the pixel in the 1st row and 5th column of a nighttime haze image, the final score of the 1st row and 5th column of the second intermediate result is the largest. Therefore, the binary weight corresponding to the 1st row and 5th column of the second intermediate result is set to 1, and the binary weight of the 1st row and 5th column of the other intermediate results is set to 0.

[0146] This invention generates a binary weight map for each intermediate result to facilitate searching for intermediate results with a binary weight of 1 for each pixel in a nighttime haze image. A binary weighting method is used, combining K intermediate results while ensuring the sparsity of the binary weight map; that is, the majority of pixels in each intermediate result have a binary weight of 0, and only a small number of pixels have a binary weight of 1.

[0147] This invention fuses K intermediate results based on a binary weighted graph to fully utilize the information within these K results. Naive fusion can lead to artifacts and a failure to preserve contrast and image details. Therefore, this invention employs a multi-scale strategy to fuse the K intermediate results:

[0148] The multi-scale fusion strategy is used to fuse all the intermediate results to obtain the fused result, including:

[0149] The following formula is used to fuse all the intermediate results:

[0150]

[0151] Among them, G l L represents the Gaussian pyramid. l This refers to the Pyramid of Laplace, where Ltotal represents the total number of levels in the pyramid, and l represents the pyramid's index. R represents the k-th intermediate result of the i-th frame of the nighttime haze image. initial (i) represents the fusion result of the i-th frame of nighttime haze image, W k (i) represents the binary weight of the kth intermediate result of the i-th frame of the nighttime haze image, where K is the total number of intermediate results, and . represents a multiplication operation.

[0152] Gaussian pyramids are a multi-scale image representation method that generates a series of images with progressively decreasing resolution by applying Gaussian blur and downsampling to each intermediate result. Gaussian blur uses a Gaussian function to convolve the image; the Gaussian function is a normal distribution function with smooth and continuous properties, and can be used to smooth pixel values ​​in the image through weighted averaging.

[0153] The Laplacian pyramid is a multi-scale image decomposition method based on the Gaussian pyramid. It generates low-frequency information from the image using the Gaussian pyramid and then extracts high-frequency information by utilizing the differences between layers of the Gaussian pyramid. The Laplacian pyramid decomposes an image into a series of sub-bands at different scales, each corresponding to the image's detailed information at a specific scale. The Laplacian pyramid can selectively fuse images based on their importance at each scale. Reconstructing the image using the fused Laplacian pyramid and the Gaussian pyramid yields high-quality fusion results.

[0154] The multi-scale fusion method of this application consists of L layers. At each layer scale, a Gaussian pyramid is used for the k-th intermediate result of the i-th frame of the night haze image, and a Laplacian pyramid is used for the binary weight of the non-optimal intermediate result corresponding to the i-th pixel in the night haze image. Combining the Gaussian pyramid and the Laplacian pyramid can improve the fusion effect of image features at different scales.

[0155] Calculate the total number of layers of the pyramid using the following formula:

[0156]

[0157] Where Ltotal is the total number of layers in the pyramid, M is the total number of rows in the nighttime haze image, N is the total number of columns in the nighttime haze image, ln is the logarithmic function with the natural constant e as the base, min is the minimum value function, and [] represents the floor function.

[0158] The multi-scale fusion strategy described in this embodiment for fusing all intermediate results to obtain a fused result includes fusing all intermediate results using the following formula: Among them, G l L represents the Gaussian pyramid. l This refers to the Pyramid of Laplace, where Ltotal represents the total number of levels in the pyramid, and l represents the pyramid's index. R represents the k-th intermediate result of the i-th frame of the nighttime haze image. initial (i) represents the fusion result of the i-th frame of nighttime haze image, W k (i) represents the binary weight of the kth intermediate result of the i-th frame of the nighttime haze image. The multi-scale fusion algorithm of this embodiment can perform fusion at multiple levels, while fully utilizing the most informative parts of the k intermediate results, thereby avoiding artifacts in the fusion result and improving the contrast and image details of the fusion result.

[0159] Example 4

[0160] like Figure 2 As shown, this embodiment provides a drone control method based on dehazing of real nighttime images. This embodiment only describes the differences from Embodiment 1. The post-processing of the fusion result to obtain the final dehazed image includes:

[0161] S71: Non-subsampled shear wave transform is used to separate the high-frequency and low-frequency components of the brightness channel of the fusion result.

[0162] S72: Use non-subsampled shear wave transform to separate the high-frequency and low-frequency components of the brightness channel of the nighttime haze image.

[0163] S73: The residual of the high-frequency component of the brightness channel of the nighttime haze image and the high-frequency component of the brightness channel of the fusion result are fused to obtain the fused high-frequency feature.

[0164] S74: Perform a non-subsampled shear plate inverse transform on the fused high-frequency features to obtain the enhanced brightness.

[0165] S75: Combine the smoothing and sharpening filter and the guided filter to extract the edge portion of the brightness channel of the nighttime haze image; fuse the edge portion and the brightness after texture enhancement to obtain the edge-optimized brightness.

[0166] S76: Perform nonlocal adaptive saturation adjustment on the saturation channel of the fusion result to obtain the final dehazed image.

[0167] Non-subsampled shear wave transform is employed to perform multi-scale and multi-directional analysis of the fusion results. Compared to traditional wavelet transform, non-subsampled shear wave transform offers greater flexibility in directionality, enabling it to better capture edge information in the fusion results. Furthermore, since non-subsampled shear wave transform does not use a downsampling operation, it reduces the occurrence of pseudo-Gibbs phenomena.

[0168] This embodiment employs non-subsampled shear wave transform (NSBFT) to separate components at different levels, thereby removing the glow effect from the image. NSBFT has low computational complexity and high efficiency. Since texture typically exists in high-frequency components of an image, NSBFT is applied to the fusion result and the nighttime haze image separately according to the following formula:

[0169]

[0170] Where N represents the non-subsampled shear wave transform, This represents the result of performing a non-downsampled shear wave transform on the brightness channel of the fusion result of the i-th frame of the nighttime haze image. The high-frequency components of the brightness channel represent the fusion result of the i-th frame of the nighttime haze image. N(I) represents the low-frequency component of the brightness channel of the fusion result of the i-th frame of the nighttime haze image; V (i) represents the result of performing a non-downsampled shear wave transform on the brightness channel of the i-th frame of the nighttime haze image. This represents the high-frequency components of the brightness channel in the i-th frame of the nighttime haze image. This represents the low-frequency components of the brightness channel in the i-th frame of the nighttime haze image. I represents the luminance channel of the fusion result of the i-th frame of the nighttime haze image. V (i) represents the brightness channel of the i-th frame of the nighttime haze image.

[0171] To enhance the texture of the image, the high-frequency components of the fusion result of the i-th frame of the nighttime haze image and the residuals of the high-frequency components of the i-th frame of the nighttime haze image are fused according to the following formula:

[0172]

[0173] in, Let λ be the fused high-frequency features of the i-th frame of the nighttime haze image, and e be the linear enhancement factor. N (i) represents the residual of the high-frequency components of the i-th frame of the nighttime haze image. This represents the high-frequency components of the i-th frame of the nighttime haze image. This represents the mean of the high-frequency components of the i-th frame of the nighttime haze image.

[0174] The fused high-frequency features are affected by image noise; therefore, this embodiment uses a mean-based soft thresholding strategy for noise suppression.

[0175]

[0176] Among them, u N (i) represents the fused high-frequency features of the i-th frame of the nighttime haze image. The mean value of and fix is ​​the soft threshold adjustment parameter. In this embodiment, fix is ​​0.05. This embodiment uses the soft threshold adjustment parameter fix and u as an example. N (i) is multiplied by a soft threshold, and the portion of the fused high-frequency features of the i-th frame nighttime haze image that is less than the soft threshold is set to 0.

[0177] After processing, the enhanced brightness is obtained by performing a non-subsampled clipping inverse transform according to the following formula:

[0178]

[0179] in, N represents the brightness of the i-th frame of the nighttime haze image after texture enhancement. -1 This represents the inverse transform of the non-subsampled shear wave.

[0180] To preserve the structural edges of the image, this invention optimizes the brightness edges. A sharpening filter is used to sharpen the brightness channel of the i-th frame of the nighttime haze image, and a guided filter is used to smooth the brightness channel of the i-th frame of the nighttime haze image. The difference between the sharpening result and the smoothing result is used as the brightness after edge optimization.

[0181] Calculate the edge-optimized brightness using the following formula:

[0182]

[0183] Where ln is the logarithmic transform with the natural constant as the base, SSIF represents the sharpening filter, GIF represents the guided filter, and I V (i) represents the brightness channel of the i-th frame of the nighttime haze image. This represents the brightness of the i-th frame of the nighttime haze image after edge optimization.

[0184] In Equation (20), adding 1 to the independent variable of the logarithmic transformation makes the independent variable of the logarithmic transformation greater than 1, and the result of the logarithmic transformation greater than 0. The logarithmic transformation amplifies subtle changes in the edges while reducing the impact of large intensity differences, making it easier to detect weaker image edges, thus improving the overall image edge detection effect. In addition, smoothing the brightness channel of the i-th frame of the nighttime haze image using guided filtering helps to effectively suppress noise while maintaining the image edge brightness of the i-th frame of the nighttime haze image after edge optimization.

[0185] The following formula is used to perform guided filtering on the brightness of the i-th frame of the nighttime haze image after edge optimization:

[0186]

[0187] Here, GIF represents guided filtering. This represents the edge-optimized brightness of the i-th frame of the nighttime haze image. This represents the edge portion of the brightness channel in the i-th frame of the nighttime haze image.

[0188] The brightness of the edge portion and the enhanced texture are fused using the following formula:

[0189]

[0190] in, This represents the edge-optimized brightness of the i-th frame of the nighttime haze image. Let represent the brightness of the i-th frame of the nighttime haze image after texture enhancement, exp represent the exponential function, and . represent the multiplication operation.

[0191] Wavelength-dependent scattering and interference from artificial light sources cause severe saturation decay in nighttime haze images, resulting in visually faded colors and reduced color discernibility. To address this issue, this invention proposes a nonlocal adaptive saturation adjustment algorithm that dynamically and precisely adjusts saturation by utilizing color similarity between nonlocal regions.

[0192] The saturation channel of the fusion result is adjusted nonlocally and adaptively according to the following formula:

[0193]

[0194] Among them, R S (i) represents the fusion result R of the i-th frame of nighttime haze image. initial (i) saturation channel, R′ S (i) represents the adjusted saturation channel of the i-th frame of the nighttime haze image, u s (m) is R S(i) The average value of the m-th region, e s (i) represents the residual saturation of the i-th frame of the nighttime haze image, e s (i) By R S (i) Subtract u s (m) is obtained. ζ is the fifth adjustment parameter, ω is the sixth adjustment parameter, and exp represents the exponential function. Preferably, the adjusted saturation channel R′ of the i-th frame of the nighttime haze image is calculated using formula (23). S (i) after which, R′ S (i) Perform guided filtering to adjust R′ S (i) Further optimization will be carried out.

[0195] This embodiment performs post-processing on the fusion results obtained from multi-scale fusion. The post-processing includes texture enhancement, edge optimization, and saturation adjustment to improve the image quality in various aspects and effectively alleviate various degradation problems such as uneven lighting, texture blurring, edge blurring, haze, and color distortion.

[0196] Example 5

[0197] This embodiment provides a drone control method based on dehazing of real nighttime images. This embodiment only describes the differences from Embodiment 1. The step of sending the final dehazed image to the drone and controlling the drone to perform nighttime scene tasks includes:

[0198] S81: Control the drone to perform power line inspection based on the final defogging image to detect defects in the power equipment.

[0199] S82: Control the drone to conduct nighttime security patrols based on the final defogging image to identify target personnel in the scene.

[0200] S83: Control the drone to conduct disaster search and rescue based on the final defogging image to identify trapped personnel and dangerous areas in the scene.

[0201] Using the real-night image dehazing algorithm from any one of Examples 1-4, haze can be removed from nighttime haze images, thereby improving image contrast, clarity, and visibility. An algorithm platform is mounted on a server, running the real-night image dehazing algorithm of this invention. Alternatively, the processor of a drone can execute the real-night image dehazing algorithm of this invention, enabling more accurate and faster identification of faults in power lines and equipment in nighttime scenes, thereby improving the efficiency and safety of power inspections.

[0202] In nighttime security patrol scenarios, the algorithm platform on the drone processor or server performs defogging and image quality enhancement operations on nighttime haze images, thereby improving the visibility of the final defogging image. This makes it easier for subsequent target recognition algorithms to identify potential threats in the final defogging image, ensuring the effectiveness of nighttime security patrols.

[0203] In disaster search and rescue scenarios, such as nighttime search and rescue operations after earthquakes or floods, drones can quickly reach the scene and acquire images in real time. However, nighttime fog and haze can severely affect the image quality captured by drones, potentially leading to inaccurate assessments of disaster areas by rescuers and significant deviations in target location. By running the real-night image defogging algorithm of this invention on the drone's processor or server, the clarity of nighttime fog and haze images acquired by drones can be improved, helping rescue teams more accurately identify trapped individuals and dangerous areas, thereby increasing the success rate and efficiency of disaster search and rescue.

[0204] This embodiment applies a real-night image dehazing algorithm to UAVs performing nighttime scene tasks, including power line inspection, nighttime security patrols, and disaster search and rescue. The algorithm platform on the UAV processor or server runs the real-night image dehazing algorithm of this invention to remove haze from nighttime fog images, improving image quality and thus increasing the accuracy of identifying power equipment defects, target personnel, trapped personnel, and dangerous areas in the scene.

[0205] This embodiment also provides a computer device, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor in this computer design provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communication with external terminals via a network connection.

[0206] This embodiment also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements a drone control method based on defogging of real nighttime images. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0207] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0208] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A drone control method based on dehazing of real nighttime images, characterized in that, include: Acquire nighttime haze images, and preprocess the nighttime haze images to obtain preprocessed images; The dark channel prior method is used to perform preliminary dehazing on the preprocessed image, resulting in several intermediate results; A scoring model is established based on image contrast, chromaticity, saliency, and color contrast. Each intermediate result is input into the scoring model for scoring, and the final score of the k-th intermediate result of the i-th frame of the nighttime haze image is obtained; Based on the final score, generate binary weights for each of the intermediate results; Based on the binary weights, a multi-scale fusion strategy is used to fuse all the intermediate results to obtain the fusion result; The fusion result is post-processed to obtain the final dehazed image; The final dehazed image is sent to the drone, which is then controlled to perform nighttime scene tasks.

2. The UAV control method based on dehazing of real nighttime images according to claim 1, characterized in that, The preprocessing of the nighttime haze image to obtain a preprocessed image includes: Different nonlinear correction coefficients are set, and gamma correction is performed on the nighttime haze image using the following formula: in, Let I(i) be the k-th preprocessed image of the i-th nighttime haze image, and γ be the i-th nighttime haze image. k is the k-th nonlinear correction coefficient.

3. The UAV control method based on real nighttime image dehazing according to claim 1, characterized in that, The dark channel prior method is used to perform preliminary dehazing on the preprocessed image, resulting in several intermediate results, including: The following formula is used to perform preliminary dehazing on the preprocessed image: in, Let represent the k-th intermediate result of the i-th frame of the nighttime haze image, and DCP represent the haze removal operation based on the dark channel prior. For the k-th preprocessed image of the i-th frame of the nighttime haze image, γ k is the k-th nonlinear correction coefficient.

4. The UAV control method based on real nighttime image dehazing according to claim 1, characterized in that, The scoring model, based on image contrast, chroma, saliency, and color contrast, includes: Calculate image contrast, chromaticity, saliency, and color contrast using the following formulas: in, Let be the image contrast of the kth intermediate result of the i-th frame of the nighttime haze image. Let be the chromaticity of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance of the k-th intermediate result of the i-th frame of the nighttime haze image. The color contrast of the kth intermediate result of the i-th frame of the nighttime haze image; This represents the k-th intermediate result of the i-th frame of the nighttime haze image. express The second derivative in the x-direction, express The second derivative in the y-direction; c represents the color channel, r represents the red channel, g represents the green channel, and b represents the blue channel. This represents the color channel of the k-th intermediate result of the i-th frame of the nighttime haze image; This represents the average of the three color channels of the k-th intermediate result of the i-th frame of the nighttime haze image. This represents the grayscale value of the k-th intermediate result of the i-th frame of the nighttime haze image. This represents the blur value of the kth intermediate result of the i-th frame of the nighttime haze image; This represents the hue of the k-th intermediate result in the HSV space for the i-th frame of the nighttime haze image. Let represent the saturation of the k-th intermediate result of the i-th frame of the nighttime haze image in the HSV space, θ represent the color contrast multiplicative parameter, Φ represent the color contrast additive parameter, and FFT represent the fast Fourier transform. A scoring model is established by using the image contrast, chromaticity, saliency, and color contrast as independent variables, and the final score of the kth intermediate result of the i-th frame of the nighttime haze image as the dependent variable.

5. The UAV control method based on real nighttime image defogging according to claim 4, characterized in that, The step of inputting each intermediate result into the scoring model for scoring to obtain the final score of the k-th intermediate result of the i-th frame of the nighttime haze image includes: Input the kth intermediate result into the scoring model to calculate the corresponding image contrast, chroma, saliency and color contrast; Calculate the image contrast weight, chromaticity weight, saliency weight, and color contrast weight; The image contrast, chromaticity, saliency, and color contrast are weighted and summed to obtain the final score of the kth intermediate result of the i-th frame of the nighttime haze image.

6. The UAV control method based on real nighttime image dehazing according to claim 5, characterized in that, The calculation of image contrast weight, chromaticity weight, saliency weight, and color contrast weight includes: Calculate the image contrast weight using the following formula: in, Let be the mean contrast of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the contrast standard deviation of the k-th intermediate result of the i-th frame of the nighttime haze image, and α be the first adjustment parameter. The image contrast weight of the kth intermediate result of the i-th frame of the nighttime haze image; Calculate the chromaticity weights using the following formula: Where exp represents the exponential function, β is the second adjustment parameter, and ε is the third adjustment parameter. Let be the chromaticity of the k-th intermediate result of the i-th frame of the nighttime haze image. The chromaticity weight is the k-th intermediate result of the i-th frame of the nighttime haze image, and max indicates the operation of taking the maximum value; The significance weight is calculated using the following formula: in, Let be the significance mean of the k-th intermediate result of the i-th frame of the nighttime haze image. Let be the significance standard deviation of the k-th intermediate result of the i-th frame of the nighttime haze image. This is the fourth adjustment parameter. The significance weight of the k-th intermediate result of the i-th frame of the nighttime haze image; Calculate the color contrast weight using the following formula: in, This represents the hue of the k-th intermediate result in the HSV space for the i-th frame of the nighttime haze image. This represents the entropy value of the k-th intermediate result of the i-th frame of the nighttime haze image in the HSV space, representing the hue. The color contrast weights are the k-th intermediate results of the i-th frame of the nighttime haze image.

7. The UAV control method based on real nighttime image dehazing according to claim 1, characterized in that, The step of generating binary weights for each intermediate result based on the final score includes: The following formula is used to search for the optimal intermediate result corresponding to each pixel in the nighttime haze image based on the final score: Where τ(row,col) represents the index of the optimal intermediate result of the pixel in the row-th row and column-th column of the nighttime haze image, K is the total number of intermediate results, k is the sequence number of the intermediate result, argmax represents the sequence number of the intermediate result corresponding to the maximum value of the final search score, and S k (row,col) represents the final score of the pixel in the row-th row and column-th column of the k-th intermediate result; The binary weights for each of the intermediate results are generated according to the following formula: Among them, W τ(row,col) (row,col) represents the binary weight of the optimal intermediate result corresponding to the pixel in the row-th row and column-th column of the nighttime haze image, W. p (row,col) represents the binary weight of the non-optimal intermediate result corresponding to the pixel in row 'row' and column 'col' of the nighttime haze image, and p represents the index of the non-optimal intermediate result of the pixel in row 'row' and column 'col' of the nighttime haze image.

8. The UAV control method based on real nighttime image dehazing according to claim 7, characterized in that, The multi-scale fusion strategy is used to fuse all the intermediate results to obtain the fused result, including: The following formula is used to fuse all the intermediate results: Among them, G l L represents the Gaussian pyramid. l This refers to the Pyramid of Laplace, where Ltotal represents the total number of levels in the pyramid, and l represents the pyramid's index. R represents the k-th intermediate result of the i-th frame of the nighttime haze image. initial (i) represents the fusion result of the i-th frame of nighttime haze image, W k (i) represents the binary weight of the kth intermediate result of the i-th frame of the nighttime haze image.

9. The UAV control method based on real nighttime image dehazing according to claim 1, characterized in that, The post-processing of the fusion result to obtain the final dehazed image includes: Non-subsampled shear wave transform is used to separate the high-frequency and low-frequency components of the brightness channel of the fusion result; Non-subsampled shear wave transform is used to separate the high-frequency and low-frequency components of the brightness channel in the nighttime haze image; The high-frequency components of the brightness channel of the nighttime haze image are fused with the high-frequency components of the brightness channel of the fusion result to obtain the fused high-frequency features. The fused high-frequency features are subjected to an inverse non-subsampled shear plate transform to obtain the enhanced brightness after texture enhancement; By combining a smoothing and sharpening filter and a guided filter, the edge portion of the brightness channel of the nighttime haze image is extracted; the edge portion and the brightness after texture enhancement are fused to obtain the edge-optimized brightness. The saturation channel of the fusion result is adjusted using nonlocal adaptive saturation to obtain the final dehazed image.

10. The UAV control method based on real nighttime image dehazing according to claim 1, characterized in that, Sending the final dehazed image to the drone and controlling the drone to perform nighttime scene tasks includes: The drone is controlled to perform power line inspection based on the final defogging image in order to detect defects in the power equipment; The drone is controlled to conduct nighttime security patrols based on the final defogging image in order to identify target personnel in the scene; The drone is controlled to conduct disaster search and rescue operations based on the final defogging image to identify trapped personnel and dangerous areas in the scene.

Images