An image dehazing method based on curve adjustment

Abstract

The present invention discloses an image defogging method based on curve adjustment, which is applied to the field of image processing technology, including: normalizing the image pixel values ​​before and after the image is defogged, and constructing a representation function of the adjustment curve using the cubic spline interpolation method; based on sample point continuity, first-order derivative continuity, second-order derivative continuity and boundary conditions, constraining conditions are constructed, and the representation function is solved, four preset sample points including two curve endpoints are substituted into the solution result, and the cubic spline interpolation function of the adjustment curve is obtained; wherein, the horizontal coordinates of the other two preset sample points except the two curve endpoints are known, and the vertical coordinates are unknown; the vertical coordinates of the other two preset sample points except the two curve endpoints in the four preset sample points in the input image are obtained by using a densely connected neural network, and the corresponding adjustment curve is obtained by substituting into the cubic spline interpolation function, and the input image is defogged. The present invention effectively improves the image defogging accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image defogging method based on curve adjustment. Background Art

[0002] In the field of image processing, the presence of fog often leads to reduced image contrast and reduced information, which affects subsequent image processing and analysis, resulting in data waste. Existing defogging methods are mainly divided into two categories. One is an end-to-end method based on deep learning, which can directly achieve the output from foggy image input to fog-free image by building a convolutional neural network without the need for additional image prior knowledge. The other is a method based on physical models, the most common of which is the atmospheric scattering model, which requires estimating the atmospheric light value and transmittance, and then restoring the fog-free image based on the model.

[0003] Among them, although the end-to-end learning method can directly learn the complex mapping transformation from fog to fog-free through the network and restore the fog-free image, due to the lack of prior knowledge of the image, the image effect after defogging is often not ideal, such as the image quality is reduced and the defogging is not comprehensive enough; and the method based on the atmospheric scattering model can remove fog and improve image quality to a certain extent, but in the case of heavy fog, it will introduce artifacts or over-process some parts of the image, making the image unnatural and unclear.

[0004] Therefore, how to provide an image defogging method based on curve adjustment that can cope with a variety of complex defogging environments and effectively improve the image defogging accuracy is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] In view of this, the present invention proposes an image defogging method based on curve adjustment.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] An image defogging method based on curve adjustment, comprising:

[0008] Step 1: Normalize the image pixel values ​​before and after image defogging, and use the cubic spline interpolation method to construct the representation function of the adjustment curve;

[0009] Step 2: Based on the continuity of sample points, the continuity of first-order derivatives, the continuity of second-order derivatives and boundary conditions, the constraint conditions of the representation function are constructed, and the representation function is solved. The four preset sample points including the two curve endpoints are substituted into the solution result to obtain the cubic spline interpolation function of the adjustment curve; among the four preset sample points, the horizontal coordinates of the other two preset sample points except the two curve endpoints are known, but the vertical coordinates are unknown;

[0010] Step 3: Use a densely connected neural network to obtain the ordinates of the other two preset sample points in the input image except the two curve endpoints, substitute them into the cubic spline interpolation function to obtain the corresponding adjustment curve, and defog the input image.

[0011] Optionally, in step 1, a cubic spline interpolation method is used to construct a representation function of the adjustment curve, specifically:

[0012] The pixel values ​​of the normalized image before and after defogging are used as the horizontal and vertical coordinates to construct the sample set ;in, , are the two endpoints of the adjustment curve;

[0013] Subinterval The cubic spline function is as follows:

[0014]

[0015] The sample set has subintervals, then the corresponding A cubic spline function, The cubic spline functions together constitute the representation function of the adjustment curve; each cubic spline function contains 4 unknowns, which means that there are An unknown number.

[0016] Optionally, in step 2, based on the continuity of sample points, the continuity of first-order derivatives, the continuity of second-order derivatives and boundary conditions, constraints representing the function are constructed, specifically:

[0017] Based on the continuity of sample points, the constraints of the representation function are constructed, specifically:

[0018] Except for the two endpoints of the adjustment curve, all All sample points satisfy , then construct represents the constraints of the function, and the two endpoints satisfy , construct two constraints representing the function, then continuously construct The constraints of the function are represented by

[0019] Based on the continuity of the first-order derivative, the constraints representing the function are constructed, specifically:

[0020] Except for the two endpoints of the adjustment curve, all The first-order derivatives of the sample points are continuous, and the first-order derivatives of the left and right cubic spline functions are equal at the current sample point, which can be expressed as: , then based on the first-order derivative, we can construct The constraints of the function are represented by

[0021] Based on the continuity of the second-order derivative, the constraints representing the function are constructed, specifically:

[0022] Except for the two endpoints of the adjustment curve, all The second-order derivatives of the sample points are continuous, and the second-order derivatives of the left and right cubic spline functions are equal at the current sample point, which can be expressed as: , then based on the second-order derivative, we can construct The constraints of the function are represented by

[0023] Based on the boundary conditions, the constraints of the representation function are constructed, specifically:

[0024] The second-order derivatives of the two endpoints of the adjustment curve are 0, expressed as: , then the boundary conditions construct a total of 2 constraints representing the function;

[0025] Based on the continuity of sample points, first-order derivatives, second-order derivatives and boundary conditions, we construct The constraints of the representation function are the same as the number of unknowns in the representation function, and meet the solution conditions of the unknowns in the representation function.

[0026] Optionally, in step 2, the representation function is solved based on the constraints of the representation function, specifically:

[0027] Find the first and second derivatives of the cubic spline function as follows:

[0028]

[0029] use Indicates the step length, using represents the second-order derivative;

[0030] Depend on ,get ;

[0031] Depend on ,get ;

[0032] Depend on ,get ;

[0033] Depend on ,get ;

[0034] Will Both are expressed as second-order derivatives , and substitute it into In, get ;

[0035] Will Substitution ,get ;

[0036] Add boundary conditions , get a is an unknown number, including The linear system of equations is as follows:

[0037]

[0038] Optionally, in step 2, the horizontal coordinates of the four preset sample points are , based on the range of values ​​that does not affect the pixel value before and after the image adjustment, the horizontal coordinates of the two curve endpoints are obtained The corresponding vertical coordinates are 0 and 1 respectively.

[0039] Optionally, in step 2, four preset sample points including two curve endpoints are substituted into the solution result to obtain a cubic spline interpolation function of the adjustment curve, as follows:

[0040]

[0041] in, is the pixel value after normalization of the image; is the function corresponding to the adjustment curve; is the pixel value after curve adjustment; The horizontal axis is The ordinate corresponding to the sample point; The horizontal axis is The vertical coordinate corresponding to the sample point.

[0042] Optionally, in step 3, based on a densely connected neural network, specifically:

[0043] The input image passes through four convolutional layers with a convolution kernel of 3*3 and a stride of 1 in sequence; the input of the first convolutional layer is the input image, the input of the second convolutional layer is the output of the first convolutional layer, the input of the third convolutional layer is the output of the first and second convolutional layers, and the input of the fourth convolutional layer is the output of the first, second, and third convolutional layers;

[0044] At the same time, the input image passes through an average pooling layer with a pooling kernel of 5*5 and a step size of 1 to obtain the average pooled features, which are then concatenated with the output of the fourth convolutional layer and input into the downsampling module to obtain the downsampled and flattened features.

[0045] The features output by the downsampling module are sequentially input into two fully connected layers to obtain the ordinates of the other two preset sample points except the two curve endpoints, which are then substituted into the cubic spline interpolation function to obtain the corresponding adjustment curve, and the dehazed output image is obtained by combining the input image; wherein, each convolutional layer and fully connected layer is followed by an activation layer; when the adjustment curve is used to dehaze the input image, the curves of the three RGB channels are adjusted.

[0046] Optional, downsampling module, specifically:

[0047] The feature map input passes through the convolution layer with a convolution kernel of 3*3 and a step size of 2, the average pooling layer with a pooling kernel of 2*2, the convolution layer with a convolution kernel of 3*3 and a step size of 2, and the adaptive average pooling layer, and the output feature map is expanded into a vector with a length equal to the number of channels;

[0048] Among them, the convolution layer, average pooling layer, and each layer in the convolution layer will compress the feature map output of the previous layer to half of the input; the adaptive average pooling layer will compress the input feature map to 1*1.

[0049] Optional training losses for densely connected neural networks, including L1 loss and perceptual loss;

[0050] L1 loss, as follows:

[0051]

[0052] in, is the number of training samples; is the number of pixels in a single image; is the pixel value of the i-th pixel in the k-th training image; is the pixel value of the i-th pixel of the k-th training image output by the model;

[0053] Perceptual loss is as follows:

[0054]

[0055] in, The features extracted by inputting the image into the classic pre-trained network; is the total number of layers selected from the pre-trained model; is the output of the cth layer in the layer selected after inputting the kth training image into the pre-trained network; is the output of the cth layer in the layer selected after the kth training image output by the model is input into the pre-trained network;

[0056] The training loss of a densely connected neural network is as follows:

[0057]

[0058] in, is a hyperparameter that controls the relative weights of different losses.

[0059] Optionally, it also includes: setting the second fully connected layer to output a vector of length 3, and then inputting the features output by the downsampling module into the two fully connected layers in sequence to obtain the vertical coordinates of the other two preset sample points except the two curve endpoints, as follows:

[0060]

[0061] in, is the i-th element in the output vector of length 3, is the length of the corresponding subinterval; the ordinates of the other two preset sample points except the two curve endpoints are and .

[0062] It can be seen from the above technical scheme that compared with the prior art, the present invention proposes an image defogging method based on curve adjustment. Defogging the image based on the curve can enhance the contrast, stretch the color scale, and keep the image clear while obtaining a good defogging effect, and the junction of the foreground and background will not change suddenly or blur. The representation function of the adjustment curve is constructed by the cubic spline interpolation method, and the constraint conditions of the representation function are constructed based on the continuity of sample points, the continuity of first-order derivatives, the continuity of second-order derivatives and boundary conditions. The representation function is solved, and the four preset sample points including the two curve endpoints are substituted into the solution result to obtain the cubic spline interpolation function of the adjustment curve with flexible interpolation points. The adaptive estimation processing of the ordinate of the preset sample points of the input image is realized by a densely connected neural network, which can be applied to a variety of complex defogging environments and effectively improve the image defogging accuracy. By making the second fully connected layer in the densely connected neural network output a vector of length 3, the curve for adjusting the image is monotonically increasing, ensuring that the dark areas in the image before and after defogging are always darker than the bright areas, maintaining the consistency of the relative brightness of the image, and the picture is not easily distorted. By making the same curve adjustment to the three RGB channels of the input image, the possibility of color cast after image dehazing is reduced, making the color more natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0064] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0065] Figure 2 Schematic diagram of the densely connected neural network structure of the present invention.

[0066] Figure 3 It is a schematic diagram of the structure of the downsampling module of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Embodiment 1:

[0069] Embodiment 1 of the present invention discloses an image defogging method based on curve adjustment, such as Figure 1 As shown, including:

[0070] Step 1: Normalize the image pixel values ​​before and after defogging, and use the cubic spline interpolation method to construct the representation function of the adjustment curve. For foggy images, the overall contrast is low and the color level distribution is concentrated. After defogging, the contrast of the picture will be greatly improved, and the color level distribution will become dispersed. Curve adjustment is an effective way to enhance contrast and stretch color levels. The most common is the S-shaped curve, which can compress the bright and dark parts, stretch the grayscale in the middle, and make the information in the image clearer. However, the actual foggy environment is diverse, and the corresponding adjustment curves are also variable. Therefore, the image pixel values ​​before and after defogging are curve fitted. Among them, in addition to the RGB picture format, the defogging image can also be in RAW domain or YUV and other image formats.

[0071] The cubic spline interpolation method is used to construct the representation function of the adjustment curve, specifically:

[0072] The pixel values ​​of the normalized image before and after defogging are used as the horizontal and vertical coordinates to construct the sample set ;in, , are the two endpoints of the adjustment curve; the complexity brought about by the increase in the number of sample points is proportional to the accuracy and flexibility.

[0073] In each subinterval, the interpolation function is a cubic equation, and each sample point is traversed by the interpolation function of the subinterval. The curve obtained by the cubic spline interpolation method has excellent properties and is The curve is smooth, and the first-order derivative and the second-order derivative are continuous, and the transition of the whole curve is smooth and natural. Therefore, only a limited number of sample points are needed to flexibly represent a smooth adjustment curve. The cubic spline function is as follows:

[0074]

[0075] in, The sample set has subintervals, then the corresponding A cubic spline function, The cubic spline functions together constitute the representation function of the adjustment curve; each cubic spline function contains 4 unknowns , then it means that the function exists To solve these unknowns, we need to construct constraints.

[0076] Step 2: Based on the continuity of sample points, continuity of first-order derivatives, continuity of second-order derivatives and boundary conditions, construct the constraints of the representation function, and solve the representation function. Substitute the four preset sample points including the two curve endpoints into the solution result to obtain the cubic spline interpolation function of the adjustment curve; among them, the horizontal coordinates of the other two preset sample points except the two curve endpoints are known, but the vertical coordinates are unknown.

[0077] Based on the continuity of sample points, continuity of first-order derivatives, continuity of second-order derivatives and boundary conditions, the constraints representing the function are constructed, specifically:

[0078] Based on the continuity of sample points, the constraints of the representation function are constructed, specifically:

[0079] Except for the two endpoints of the adjustment curve, all All sample points satisfy , then construct represents the constraints of the function, and the two endpoints satisfy , construct two constraints representing the function, then continuously construct The constraints of the function are represented by

[0080] Based on the continuity of the first-order derivative, the constraints representing the function are constructed, specifically:

[0081] Except for the two endpoints of the adjustment curve, all The first-order derivatives of the sample points are continuous, and the first-order derivatives of the left and right cubic spline functions are equal at the current sample point, which can be expressed as: , then based on the first-order derivative, we can construct The constraints of the function are represented by

[0082] Based on the continuity of the second-order derivative, the constraints representing the function are constructed, specifically:

[0083] Except for the two endpoints of the adjustment curve, all The second-order derivatives of the sample points are continuous, and the second-order derivatives of the left and right cubic spline functions are equal at the current sample point, which can be expressed as: , then based on the second-order derivative, we can construct The constraints of the function are represented by

[0084] Based on the boundary conditions, the constraints of the representation function are constructed, specifically:

[0085] The second-order derivatives of the two endpoints of the adjustment curve are 0, expressed as: , then the boundary conditions construct a total of 2 constraints representing the function;

[0086] Based on the continuity of sample points, first-order derivatives, second-order derivatives and boundary conditions, we construct The constraints of the representation function are the same as the number of unknowns in the representation function, and meet the solution conditions of the unknowns in the representation function.

[0087] Based on the constraints of the representation function, the representation function is solved as follows:

[0088] Find the first and second derivatives of the cubic spline function as follows:

[0089]

[0090] use Indicates the step length, using represents the second-order derivative;

[0091] Depend on ,get ;

[0092] Depend on ,get ;

[0093] Depend on ,get ;

[0094] Depend on ,get ;

[0095] Will Both are expressed as second-order derivatives , and substitute it into In, get ;

[0096] Will Substitution ,get ;

[0097] Add boundary conditions , get a is an unknown number, including The linear system of equations is as follows:

[0098]

[0099] However, although cubic spline interpolation has excellent properties, it involves a high degree of complexity and many parameters. The solution process will be complicated without constraints. In order to reduce parameters and simplify calculations, based on the current image adjustment application scenario, without loss of generality, four preset sample points are set for the required image adjustment curve. The horizontal coordinates of the four preset sample points are , based on the range of values ​​that does not affect the pixel value before and after the image adjustment, the horizontal coordinates of the two curve endpoints are obtained The corresponding vertical coordinates are 0 and 1 respectively.

[0100] Substitute the four preset sample points including the two curve endpoints into the solution result to obtain the cubic spline interpolation function of the adjustment curve, as follows:

[0101]

[0102] in, is the pixel value after normalization of the image; is the function corresponding to the adjustment curve; is the pixel value after curve adjustment; The horizontal axis is The ordinate corresponding to the sample point; The horizontal axis is At this point, it is only necessary to obtain the ordinate values ​​corresponding to the two preset sample points to obtain a complete adjustment curve without going through a complicated interpolation solution process.

[0103] Step 3: Use a densely connected neural network to obtain the ordinates of the other two preset sample points in the input image except the two curve endpoints, substitute them into the cubic spline interpolation function to obtain the corresponding adjustment curve, and defog the input image.

[0104] Based on densely connected neural networks, such as Figure 2 As shown, specifically:

[0105] Figure 2In the figure, the thick horizontal arrows represent convolutional layer or fully connected layer operations, the thick vertical arrows represent average pooling layer operations, and the thin arrows only represent the transmission path.

[0106] The input image (the pixel value range of the input image needs to be normalized to [0,1]) passes through four convolution layers with a convolution kernel of 3*3 and a step size of 1 in sequence; wherein the input of the first convolution layer is the input image, the input of the second convolution layer is the output of the first convolution layer, the input of the third convolution layer is the output of the first convolution layer and the second convolution layer, and the input of the fourth convolution layer is the output of the first convolution layer, the second convolution layer and the third convolution layer; compared with the common structure in which the convolution layer only takes the output of the previous layer as input, the dense connection used in the present invention can more fully obtain the information features learned by each layer of convolution, which is helpful to obtain more accurate numerical estimation;

[0107] At the same time, the input image passes through an average pooling layer with a pooling kernel of 5*5 and a step size of 1 to obtain the average pooled features, which are then concatenated with the output of the fourth convolutional layer as a further supplement to the image information and input into the downsampling module to obtain the downsampled and flattened features;

[0108] The features output by the downsampling module are input into two fully connected layers in sequence to obtain the ordinates of the other two preset sample points except the two curve endpoints, which are substituted into the cubic spline interpolation function to obtain the corresponding adjustment curve. The dehazed output image is obtained by combining the input image (the pixel value range of the output image after curve adjustment needs to be clipped to [0,1]); each convolution layer and fully connected layer is followed by an activation layer; when the adjustment curve is used to dehaze the input image, the curve adjustment is performed on the three RGB channels to reduce the possibility of color cast.

[0109] Among them, the network size of each layer and the size of the convolution kernel can be adjusted according to actual conditions.

[0110] Downsampling modules, such as Figure 3 As shown, specifically:

[0111] The feature map input passes through the convolution layer with a convolution kernel of 3*3 and a step size of 2, the average pooling layer with a pooling kernel of 2*2, the convolution layer with a convolution kernel of 3*3 and a step size of 2, and the adaptive average pooling layer, and the output feature map is expanded into a vector with a length equal to the number of channels;

[0112] Among them, each layer in the convolution layer, average pooling layer, and convolution layer will compress the feature map output of the previous layer to half of the input, which plays the role of downsampling; the adaptive average pooling layer will compress the input feature map to 1*1 to facilitate the input of the fully connected layer after the downsampling module.

[0113] Training losses for densely connected neural networks, including L1 loss and perceptual loss;

[0114] The L1 loss function calculates the mean absolute error of the pixel values ​​of two images. The smaller the L1 loss, the closer the two images are. The L1 loss is as follows:

[0115]

[0116] in, is the number of training samples; is the number of pixels in a single image; is the pixel value of the i-th pixel in the k-th training image; is the pixel value of the i-th pixel of the k-th training image output by the model;

[0117] Perceptual loss is a loss function based on neural network features. It measures the similarity of images by comparing the difference between the output image and the groundtruth in high-level feature maps. It can better capture the semantic information of the image and make the output image of the model more consistent with human visual perception. Perceptual loss is as follows:

[0118]

[0119] in, The features extracted by inputting the image into a classic pre-trained network (such as VGG or ResNet) are compared. In order to more fully capture and compare the feature information of the two images, the outputs of multiple layers in the pre-trained network are selected for feature comparison. is the total number of layers selected from the pre-trained model. Here, the classic network VGG19 is used, and the feature output layers obtained are 2, 7, 12, 21, and 30 layers; is the output of the cth layer in the layer selected after inputting the kth training image into the pre-trained network; is the output of the cth layer in the layer selected after the kth training image output by the model is input into the pre-trained network;

[0120] The training loss of a densely connected neural network is as follows:

[0121]

[0122] in, It is a hyperparameter used to control the relative weights of different losses. During training, the network parameters are continuously updated and iterated according to the loss value until the loss converges to a stable range.

[0123] Although two values ​​need to be estimated in order to restore the curve, a vector of length 3 will be output after the second fully connected layer in the network. According to the process of curve construction, the range of the two values ​​that need to be estimated should be limited to [0,1]. This can actually be achieved directly by following the sigmoid activation layer after the fully connected layer. But at the same time, the curve adjustment should not affect the relative brightness of the original image, so the curve should be monotonically increasing, that is, the ordinate corresponding to each interpolation point should be monotonically increasing. If sigmoid is added directly after the fully connected layer, this restriction cannot be achieved. The approach of the present invention is to let the second fully connected layer output a vector of length 3. The reason is that this adjustment curve has four interpolation points, and their ordinates should be increasing, that is, the interval [0,1] will be divided into three sub-intervals, and the three values ​​output after the fully connected layer correspond to the lengths of these three sub-intervals. However, it is difficult for the sum of these three values ​​to be exactly equal to 1, and a normalization operation needs to be performed. Therefore, it also includes: setting the second fully connected layer to output a vector of length 3, then inputting the features output by the downsampling module into the two fully connected layers in sequence, and obtaining the ordinates of the other two preset sample points except the two curve endpoints, as follows:

[0124]

[0125] in, is the i-th element in the output vector of length 3, is the length of the corresponding subinterval; the ordinates of the other two preset sample points except the two curve endpoints are (The horizontal axis is The ordinate of the sample point) and (The horizontal axis is The vertical coordinate corresponding to the sample point).

[0126] The embodiment of the present invention discloses an image defogging method based on curve adjustment. Defogging an image based on a curve can enhance contrast, stretch color levels, and keep the image clear while obtaining a good defogging effect, and the junction of the foreground and background will not change suddenly or blur. The representation function of the adjustment curve is constructed by the cubic spline interpolation method, and the constraint conditions of the representation function are constructed based on the continuity of sample points, the continuity of first-order derivatives, the continuity of second-order derivatives and boundary conditions. The representation function is solved, and four preset sample points including two curve endpoints are substituted into the solution result to obtain the cubic spline interpolation function of the adjustment curve with flexible interpolation points. The adaptive estimation processing of the ordinate of the preset sample points of the input image is realized by a densely connected neural network, which can be applied to a variety of complex defogging environments and effectively improve the image defogging accuracy. By making the second fully connected layer in the densely connected neural network output a vector with a length of 3, the curve for adjusting the image is monotonically increasing, ensuring that the dark areas in the image before and after defogging are always darker than the bright areas, maintaining the consistency of the relative brightness of the image, and the picture is not easily distorted. By making the same curve adjustment to the three RGB channels of the input image, the possibility of color cast after image dehazing is reduced, making the color more natural.

[0127] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0128] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image defogging method based on curve adjustment, characterized in that: include: Step 1: Normalize the image pixel values ​​before and after image defogging, and use the cubic spline interpolation method to construct the representation function of the adjustment curve; Step 2: Based on the continuity of sample points, the continuity of first-order derivatives, the continuity of second-order derivatives and boundary conditions, the constraint conditions of the representation function are constructed, and the representation function is solved, and four preset sample points including two curve endpoints are substituted into the solution result to obtain the cubic spline interpolation function of the adjustment curve; wherein, among the four preset sample points, the horizontal coordinates of the other two preset sample points except the two curve endpoints are known, and the vertical coordinates are unknown; Step 3: using a densely connected neural network to obtain the ordinates of the other two preset sample points in the four preset sample points in the input image except the two curve endpoints, substituting them into the cubic spline interpolation function to obtain the corresponding adjustment curve, and defogging the input image; In step 1, the cubic spline interpolation method is used to construct the representation function of the adjustment curve, specifically: The pixel values ​​of the normalized image before and after defogging are used as the horizontal and vertical coordinates to construct a sample set {(x1, y1), (x2, y2), ..., (x n+1 ,y n+1 )}; where x1 = a, x n+1 =b, which are the two end points of the adjustment curve; Subinterval (x i , x i+1 ) is as follows: S i (x)=a i +b i (x-x i )+c i (x-x i ) 2 +d i (x-x i ) 3 ; The sample set has a total of n subintervals, which includes corresponding n cubic spline functions, and the n cubic spline functions together constitute the representation function of the adjustment curve; wherein each of the cubic spline functions includes 4 unknowns, and there are a total of 4n unknowns in the representation function; In step 3, the densely connected neural network is specifically: The input image passes through four convolution layers with a convolution kernel of 3*3 and a stride of 1 in sequence; wherein the input of the first convolution layer is the input image, the input of the second convolution layer is the output of the first convolution layer, the input of the third convolution layer is the output of the first convolution layer and the second convolution layer, and the input of the fourth convolution layer is the output of the first convolution layer, the second convolution layer, and the third convolution layer; At the same time, the input image passes through an average pooling layer with a pooling kernel of 5*5 and a step size of 1 to obtain the average pooled features, which are spliced ​​with the output of the fourth convolutional layer and input into the downsampling module to obtain the downsampled and flattened features; The features output by the downsampling module are sequentially input into two fully connected layers to obtain the ordinates of the other two preset sample points except the two endpoints of the curve, and the ordinates are substituted into the cubic spline interpolation function to obtain the corresponding adjustment curve, and the defogged output image is obtained in combination with the input image; wherein each of the convolutional layer and the fully connected layer is followed by an activation layer; and the adjustment curve performs curve adjustment on the three RGB channels when defogging the input image.

2. The image defogging method based on curve adjustment according to claim 1, characterized in that: In step 2, based on the continuity of sample points, the continuity of first-order derivatives, the continuity of second-order derivatives and boundary conditions, the constraints of the representation function are constructed, specifically: Based on the continuity of the sample points, the constraint conditions of the representation function are constructed, specifically: Except for the two endpoints of the adjustment curve, all n-1 sample points satisfy S i (x i+1 )=S i+1 (x i+1 )=y i+1 , then 2(n-1) constraints of the representation function are constructed, and the two endpoints satisfy S1(x1)=y1 and S n (x n+1 )=y n+1 , constructing two constraint conditions of the representation function, then continuously constructing 2n constraint conditions of the representation function based on the sample points; Based on the continuity of the first-order derivative, the constraint condition of the representation function is constructed, specifically: Except for the two endpoints of the adjustment curve, the first-order derivatives of all n-1 sample points are continuous, and the first-order derivatives of the left and right cubic spline functions are equal at the current sample point, which is expressed as: S′ i (x i+1 ) = S′ i+1 (x i+1 ), then based on the first-order derivative, (n-1) constraints of the representation function are continuously constructed; Based on the continuity of the second-order derivative, the constraint condition of the representation function is constructed, specifically: Except for the two endpoints of the adjustment curve, the second-order derivatives of all n-1 sample points are continuous, and the second-order derivatives of the left and right cubic spline functions are equal at the current sample point, which is expressed as: S″ i (x i+1 )=S″ i+1 (x i+1 ), then based on the second-order derivative, (n-1) constraints of the representation function are continuously constructed; Based on the boundary conditions, the constraints of the representation function are constructed, specifically: The second-order derivatives of the two endpoints of the adjustment curve are 0, which is expressed as: S″1(x1)=S″ n (x n+1 )=0, then the boundary conditions construct two constraint conditions of the representation function; Based on the continuity of sample points, the continuity of first-order derivatives, the continuity of second-order derivatives and boundary conditions, a total of 4n constraints of the representation function are constructed, which is the same as the number of unknowns in the representation function, and meets the solution conditions of the unknowns in the representation function.

3. The image defogging method based on curve adjustment according to claim 1, characterized in that: In step 2, the horizontal coordinates of the four preset sample points are Based on the value range of the pixel value that is not affected before and after the image adjustment, the ordinates corresponding to the abscissas {0, 1} of the two endpoints of the curve are obtained to be 0 and 1 respectively.

4. The image defogging method based on curve adjustment according to claim 3, characterized in that: In step 2, four preset sample points including two curve endpoints are substituted into the solution result to obtain the cubic spline interpolation function of the adjustment curve, as follows: Where x is the pixel value after normalization of the image; f(·) is the function corresponding to the adjustment curve; f(x) is the pixel value after adjustment by the curve; a is the horizontal coordinate The ordinate of the sample point; b is the abscissa The vertical coordinate corresponding to the sample point.

5. The image defogging method based on curve adjustment according to claim 1, characterized in that: The downsampling module is specifically: The feature map input passes through the convolution layer with a convolution kernel of 3*3 and a step size of 2, the average pooling layer with a pooling kernel of 2*2, the convolution layer with a convolution kernel of 3*3 and a step size of 2, and the adaptive average pooling layer, and the output feature map is expanded into a vector with a length equal to the number of channels; Among them, each of the convolutional layer, average pooling layer, and convolutional layer will compress the feature map output of the previous layer to half of the input; the adaptive average pooling layer will compress the input feature map to 1*1.

6. The image defogging method based on curve adjustment according to claim 1, characterized in that: The training loss of the densely connected neural network includes: L1 loss and perceptual loss; The L1 loss is as follows: Where N is the number of training samples; M is the number of pixels in a single image; y ki is the pixel value of the i-th pixel in the k-th training image; is the pixel value of the i-th pixel of the k-th training image output by the model; The perceptual loss is as follows: Where Ω(·) is the feature extracted by inputting the image into the classic pre-trained network; C is the total number of layers selected from the pre-trained model; Ω(y k ) c is the output of the cth layer in the layer selected after inputting the kth training image into the pre-trained network; is the output of the cth layer in the layer selected after the kth training image output by the model is input into the pre-trained network; The training loss of the densely connected neural network is as follows: L dehaze =L1+λL percep ; Among them, λ is a hyperparameter used to control the relative weights of different losses.

7. The image defogging method based on curve adjustment according to claim 1, characterized in that: Also includes: The second fully connected layer is set to output a vector of length 3, and the features output by the downsampling module are sequentially input into two fully connected layers to obtain the ordinates of the other two preset sample points except the two endpoints of the curve, as follows: Among them, Z i is the i-th element in the output vector of length 3, t i is the length of the corresponding subinterval; the ordinates of the other two preset sample points except the two endpoints of the curve are t1 and t1+t2 respectively.

Images