Image defogging method and device based on layered weight recombination of spatial change rate, medium and product

Abstract

The invention discloses an image defogging method and device based on hierarchical weight recombination of a spatial change rate, a medium and a product, and relates to the field of image processing, and the method comprises the steps: constructing an image defogging model; obtaining a target foggy image; and inputting the target foggy image into the image defogging model to obtain a target defogged image. Wherein the image defogging model adopts a deep convolutional neural network to extract multi-scale features of an input image, adopts a spatial change rate of the input image to adaptively adjust the weight of each scale feature, adopts a residual learning mode to determine a residual according to the multi-scale features and the weight of the input image, and adds the input image and the residual to obtain a defogging result; and obtaining an output image. The image defogging model which is lightweight, high in real-time performance and high in adaptability can be provided, and efficient defogging processing under the severe weather condition is achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image defogging method, device, medium and product based on hierarchical weighted reorganization of spatial variation rate. Background Art

[0002] The safe operation of power transmission lines is crucial to the stability and reliability of the power system. Transmission lines usually pass through complex environments such as mountains, plains, and forests, and are greatly affected by weather, terrain, and environmental factors. In order to ensure the normal operation of the lines and timely discover potential fault hazards, power companies usually deploy monitoring systems in the transmission line channels to regularly capture and transmit images of the lines and their surroundings. However, under severe weather conditions such as haze, rain, fog, and smoke, monitoring images are prone to problems such as reduced contrast and blurred details, which seriously affect the accuracy and effectiveness of intelligent monitoring tasks such as fault detection and foreign object recognition. Therefore, developing efficient defogging technology to improve the quality of monitoring images is of great practical significance for the intelligent inspection and maintenance of power transmission lines.

[0003] Traditional image dehazing technology is mainly based on image processing and enhancement algorithms, such as histogram equalization, contrast enhancement, and dark channel prior. These methods can improve the visibility of images to a certain extent, but they often have problems such as unsatisfactory processing effects, strong parameter dependence, and difficulty in adapting to changing weather and scenes. In addition, the characteristics of power transmission line monitoring images are complex and changeable backgrounds, and the shapes, colors, positions, and haze levels of different objects are quite different. Traditional methods often do not work well when processing these images.

[0004] With the development of artificial intelligence and deep learning technology, dehazing methods based on deep convolutional neural networks have gradually become a hot topic of research. Deep learning methods can automatically learn the dehazing features of images through large-scale data training, without the need to manually set complex prior knowledge, and have stronger robustness and adaptability. In particular, models such as multi-scale residual networks and generative adversarial networks (GAN) perform well in processing images under complex haze conditions, and can effectively improve the contrast and detail quality of images. However, these methods still face challenges in practical applications, such as high model complexity, large computing resource requirements, and low real-time performance, and are not fully suitable for edge device deployment scenarios required in power inspections.

[0005] Therefore, in order to meet the defogging needs of monitoring images of power transmission line channels, it is particularly important to study a lightweight, real-time and highly adaptable defogging technology. Summary of the invention

[0006] The purpose of this application is to provide an image defogging method, device, medium and product based on hierarchical weighted reorganization of spatial change rate, which can provide a lightweight, real-time and highly adaptable image defogging model to achieve efficient defogging processing under severe weather conditions.

[0007] To achieve the above objectives, this application provides the following solutions:

[0008] In a first aspect, the present application provides an image defogging method based on hierarchical weighted reorganization of spatial variation rate, comprising:

[0009] Constructing an image defogging model; the image defogging model uses a deep convolutional neural network to extract multi-scale features of an input image, uses the spatial change rate of the input image to adaptively adjust the weights of each scale feature, uses residual learning to determine the residual according to the multi-scale features and weights of the input image, and adds the input image to the residual to obtain an output image;

[0010] Get the target foggy image;

[0011] The target foggy image is input into the image defogging model to obtain a target defogging image.

[0012] Optionally, construct an image dehazing model, including:

[0013] Build deep learning models;

[0014] Acquire a training data set; the training data set includes: a number of sample foggy images and corresponding sample defogging images;

[0015] The sample foggy image is taken as input, the defogging image estimation value corresponding to the sample foggy image is output, and the mean square error between the defogging image estimation value and the sample defogging image is used as the loss function. The training data set is used to train the deep learning model to obtain an image defogging model.

[0016] Optionally, the image defogging model includes: a spatial change rate calculation module, a feature extraction module, a hierarchical weight reorganization module and a residual learning module;

[0017] The spatial change rate calculation module is used to calculate the spatial change rate of the input image using the Laplace operator;

[0018] The feature extraction module is used to extract multi-scale features of the input image using a deep convolutional neural network;

[0019] The hierarchical weight reorganization module is used to normalize the spatial change rate of the input image, and adaptively adjust the weight of each scale feature according to the normalized result of the spatial change rate;

[0020] The residual learning module is used to determine the residual according to the multi-scale features and weights of the input image by using the residual learning method, and add the input image to the residual to obtain the output image.

[0021] Optionally, the Laplacian operator is used to calculate the spatial rate of change of the input image, and the expression is:

[0022]

[0023] Among them, ΔI is the Laplace operator of the input image I, which is used to represent the spatial change rate. is the second-order partial derivative of the input image I in the x direction, is the second-order partial derivative of the input image I in the y direction.

[0024] Optionally, a deep convolutional neural network is used to extract multi-scale features of the input image, expressed as:

[0025] F1=σ(W1*I+b1);

[0026] F l =σ(W l *F l-1 +b l );

[0027] F={F1,F2,...,F L};

[0028] Among them, F1 is the output feature map of the first layer of convolution, F2 is the output feature map of the second layer of convolution, and F l-1 is the output feature map of the l-1th layer convolution, F l is the output feature map of the lth layer of convolution, L is the total number of convolution layers, I is the input image, W1 is the weight matrix of the first layer of convolution, W l is the weight matrix of the lth layer of convolution, b1 is the bias term of the first layer of convolution, b l is the bias term of the l-th layer convolution, σ(·) is the nonlinear activation function, and * represents the convolution operation.

[0029] Optionally, the weight of each scale feature is adaptively adjusted according to the normalized result of the spatial change rate, and the expression is:

[0030] w l (x,y)=α l S(x,y)+β l (1-S(x,y));

[0031] Among them, w l (x, y) is the adaptive weight function, S(x, y) is the normalized result of the spatial change rate, α l and βl are weight coefficients related to the convolution layer.

[0032] Optionally, residual learning is used to determine the residual according to the multi-scale features and weights of the input image, and the input image is added to the residual to obtain the output image, which is expressed as:

[0033]

[0034] in, is the output image, I is the input image, l is the layer index of the convolution, L is the total number of convolution layers, and w l is the weight of the l-th layer convolution, F l is the output feature map of the lth convolution layer.

[0035] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image dehazing method based on hierarchical weighted reorganization of spatial variation rate.

[0036] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image dehazing method based on hierarchical weighted reorganization of spatial variation rate.

[0037] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the image dehazing method based on hierarchical weighted reorganization of spatial variation rate.

[0038] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0039] The present application provides an image defogging method, device, medium and product based on hierarchical weight reorganization of spatial change rate, which realizes defogging of foggy images through an image defogging model. The image defogging model uses a deep convolutional neural network to extract multi-scale features of the input image, uses the spatial change rate of the input image to adaptively adjust the weights of each scale feature, uses residual learning to determine the residual according to the multi-scale features and weights of the input image, and adds the input image to the residual to obtain the output image. The image defogging model provided by the present application has the characteristics of lightweight, strong real-time performance and high adaptability, and can be applied to power inspection systems. It combines the characteristics of power monitoring images, utilizes the advantages of deep learning, and optimizes the model structure and algorithm to achieve efficient defogging processing under severe weather conditions, improve the intelligence level and application effect of the power inspection system, and thus ensure the safe operation and reliable maintenance of power transmission lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of an image defogging method based on hierarchical weighted reorganization of spatial variation rate provided in this application;

[0042] Figure 2 A diagram showing the main steps of the image defogging method based on hierarchical weighted reorganization of spatial variation rate provided in this application;

[0043] Figure 3 A detailed flowchart of the image defogging method based on hierarchical weighted reorganization of spatial variation rate provided in this application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0046] In an exemplary embodiment, the present application provides an image defogging method based on hierarchical weighted reorganization of spatial change rate, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, Figure 1 As shown, the method includes the following steps 1 to 3.

[0047] Step 1: Build an image dehazing model.

[0048] Among them, the image dehazing model uses a deep convolutional neural network to extract the multi-scale features of the input image, uses the spatial change rate of the input image to adaptively adjust the weights of each scale feature, uses residual learning to determine the residual according to the multi-scale features and weights of the input image, and adds the input image to the residual to obtain the output image.

[0049] Step 2: Obtain the target foggy image.

[0050] Step 3: Input the target foggy image into the image defogging model to obtain a target defogging image.

[0051] Specifically, the image dehazing model includes: a spatial change rate calculation module, a feature extraction module, a hierarchical weight reorganization module and a residual learning module. Among them, the spatial change rate calculation module is used to calculate the spatial change rate of the input image using the Laplace operator. The feature extraction module is used to extract the multi-scale features of the input image using a deep convolutional neural network. The hierarchical weight reorganization module is used to normalize the spatial change rate of the input image, and adaptively adjust the weights of each scale feature according to the normalized result of the spatial change rate. The residual learning module is used to determine the residual according to the multi-scale features and weights of the input image using a residual learning method, and add the input image to the residual to obtain the output image.

[0052] like Figure 2 and Figure 3 As shown, the detailed technical solution of this application is as follows:

[0053] In the field of image processing, the spatial rate of change uses the Laplace operator to measure the intensity and direction of brightness changes in an image. This analysis method can highlight areas in the image where details are blurred and contrast is reduced due to haze, and provide prior information for image defogging. The gradient of an image is used to represent the direction and rate of change in image brightness, and is a commonly used representation of the spatial rate of change of an image. For an input image I(x,y), the two components of the image gradient can be calculated using the first-order partial derivatives:

[0054]

[0055] in:

[0056] G x Indicates the rate of change of image brightness in the x direction.

[0057] G y Indicates the rate of change of image brightness in the y direction.

[0058] The magnitude of the gradient and direction θ can be expressed as:

[0059]

[0060] Gradient Amplitude It can highlight the edges and details of the image where the brightness changes significantly.

[0061] The Laplace operator is a second-order derivative operator that measures the intensity of brightness changes in an image, thereby detecting edges and mutation areas of the image. The Laplace operator ΔI of the input image I(x,y) is defined as:

[0062]

[0063] in:

[0064] It is the second-order partial derivative of the image in the x direction, which represents the change in the rate of change of brightness in the x direction.

[0065] It is the second-order partial derivative of the image in the y direction, which indicates the change in the rate of change of brightness in the y direction.

[0066] When the output of the Laplacian operator at a point is zero, it indicates that there is an edge at that point, while a positive or negative value indicates the trend of brightness change near that point (increase or decrease in brightness). Therefore, it can effectively detect edges and mutation points in images, and provide important prior information for image dehazing. In the dehazing process of haze images, the Laplacian operator can effectively identify blurred areas and areas with reduced contrast caused by haze in the image. By fusing the results of the Laplacian operator with the original image, the details and contrast of the image can be better restored.

[0067] A deep convolutional neural network is used to extract the multi-scale features of the image layer by layer. The convolutional layers and pooling layers in the deep convolutional neural network process the input image step by step, extracting local features such as low-level edges and textures to high-level abstract semantic features.

[0068] The basic operation of convolutional neural networks is convolution. For an input image I, the convolution operation of a convolution kernel (or filter) K at position (i, j) in a convolutional layer can be expressed as:

[0069]

[0070] in:

[0071] I(i+m,j+n) is the pixel value at position (i+m,j+n) of the input image.

[0072] K(m,n) is the value of the convolution kernel K at the (m,n) position, and k is the size of the convolution kernel.

[0073] O(i,j) is the value of the output feature map at position (i,j).

[0074] By performing the above convolution calculation on the input image and the convolution kernel pixel by pixel, the convolution layer extracts local features such as edges and textures.

[0075] In a deep convolutional neural network, each convolution layer can be regarded as a nonlinear feature extractor, and its output is called a feature map. In the low-level convolutional layers of the network (the part close to the input layer), the convolution kernel usually learns some local edge, corner and texture features. These features can be expressed as:

[0076] F1 = σ(W1*I+b1).

[0077] in:

[0078] F1 is the output feature map of the first convolution layer.

[0079] W1 is the weight matrix (convolution kernel) of the first layer of convolution.

[0080] b1 is the bias term of the first convolution layer.

[0081] * indicates a convolution operation.

[0082] σ(·) is a non-linear activation function.

[0083] Low-level convolutional layers usually have smaller receptive fields and are therefore able to capture local features of the image.

[0084] As the network depth increases, the convolution kernels of subsequent convolutional layers learn more and more abstract features. For example, in higher convolutional layers, feature extraction can be expressed as:

[0085] F l =σ(W l *F l-1 +b l ).

[0086] in:

[0087] F l It is the output feature map of the lth convolution layer.

[0088] W l It is the weight matrix (convolution kernel) of the lth layer of convolution.

[0089] F l-1 It is the output feature map of the l-1th convolution layer.

[0090] b l is the bias term of the l-th convolution layer.

[0091] In the deep convolutional layers, the receptive field increases, and more global features and semantic information can be captured, such as the overall shape and category characteristics of the object.

[0092] The layer-by-layer feature extraction process of deep convolutional neural networks can be regarded as multi-scale modeling of image features. In the low-level convolutional layers, the network extracts detailed features (such as edges and textures). In the middle-level convolutional layers, the network gradually combines and aggregates low-level features to extract higher-level geometric features. In the high-level convolutional layers, the network can extract global and abstract semantic features. The integration of multi-scale features can be described as:

[0093] F={F1,F2,...,F L}.

[0094] in:

[0095] F is the feature set of the entire network.

[0096] F1,F2,...,F L They represent the features extracted from the 1st layer to the Lth layer respectively.

[0097] Ultimately, deep convolutional neural networks can form a comprehensive representation of the global and local information of the input image by combining feature maps of different scales. This feature gives deep learning methods significant advantages in tasks such as image classification, object detection, and image dehazing.

[0098] The spatial variation rate analysis can enhance the image dehazing effect by adaptively adjusting the weights of features at different levels of the convolutional neural network. The spatial variation rate is used to guide the network to adjust the weights of low-level and high-level features, thereby enhancing details and restoring global contrast.

[0099] The spatial rate of change is used to measure the intensity of the local brightness change of the image, and is represented by the Laplacian operator. For an input image I(x,y), its Laplacian operator calculates the change in image brightness:

[0100]

[0101] Regions with high rates of change (such as edges and contours) typically have larger gradient or Laplacian values, while regions with low rates of change (such as the sky or haze coverage) have smaller values.

[0102] Assume that a deep convolutional neural network has L convolutional layers, and the output feature map of each layer is F l , where l = 1, 2, ..., L. Low-level convolutional layers (such as F1, F2, ...) usually contain detailed information such as edges and textures, while high-level convolutional layers (such as F L-1 ,F L ) contains more abstract semantic features.

[0103] Define the adaptive weight function w l(x, y), this function adjusts the weights of each layer feature according to the spatial change rate at different positions (x, y). The weight adjustment formula can be defined as:

[0104] w l (x,y)=α l S(x,y)+β l (1-S(x,y)).

[0105] in:

[0106] S(x,y) is the normalized result of the spatial change rate, which is used to measure the degree of change of the position (x,y) and its value range is [0,1].

[0107] α l and β l is the weight coefficient associated with the level.

[0108] To enhance the details and global contrast of the image, the adaptive adjustment rule can be defined as follows:

[0109] For high-change rate areas (such as edges and contours, with large S(x,y) values), the weights of low-level features (such as F1, F2, ...) are increased to highlight detail information, expressed as: w l (x,y)=α l S(x,y), for lower convolutional layers.

[0110] For low-change rate areas (such as the sky and haze-covered areas, where S(x,y) values ​​are small), high-level features (such as F L-1 ,F L ) is increased to restore the global contrast, expressed as: w l (x,y)=β l S(x,y), for high-level convolutional layers.

[0111] The final image feature F(x,y) is obtained by adaptively weighted summing the feature maps of all layers:

[0112]

[0113] This formula combines low-level and high-level features and uses the spatial variation rate to adjust the weights to achieve a balance between detail enhancement and global contrast restoration.

[0114] The residual learning strategy enables the deep convolutional neural network to learn the difference between the dehazed image and the original image more effectively, thereby reducing the difficulty of training and accelerating the convergence of the model. Given an original input image I and its clear image I after dehazing clean , the goal of the dehazing task is to learn a mapping function F(I) such that:

[0115] I clean =F(I).

[0116] In the framework of residual learning, we do not directly learn from I to I clean Instead of mapping , we learn the difference between them:

[0117] R(I)=I clean -I.

[0118] Here, R(I) is the difference (residual) between the dehazed image and the original image.

[0119] The goal of residual learning is to predict the residual R(I) through a convolutional neural network so that the network outputs an estimate of the clear image. (output image), can be obtained by the following formula:

[0120]

[0121] The loss function ζ of the model is defined as the estimated value of the dehazed image Compared with the actual clear image I clean The mean square error between:

[0122]

[0123] in:

[0124] N is the number of training samples.

[0125] i represents the index of each training sample.

[0126] The residual learning model extracts features through multi-layer convolution and generates a clearer dehazed image based on layered weight reorganization. Suppose the output feature of the first layer of the convolutional neural network is F l , the features can be reorganized by adaptive weight adjustment:

[0127]

[0128] in:

[0129] It is the l-th layer feature after weight reorganization.

[0130] w l is the adaptively learned weight.

[0131] The output R(I) after the entire feature reorganization can be expressed as:

[0132]

[0133] In this way, the final output of the dehazed image is It is expressed by the following formula:

[0134]

[0135] Residual learning allows the network to learn only the incremental part of image dehazing (i.e., residual) instead of directly learning the complex mapping relationship from input to output, which can reduce the learning complexity of the network, avoid the gradient vanishing problem, and thus speed up the training convergence of the model. Through residual learning strategies and hierarchical weight reorganization, deep convolutional neural networks can more effectively learn the difference between dehazed images and original images, thereby generating clearer dehazed images.

[0136] The construction process of the above-mentioned image defogging model includes: constructing a deep learning model; obtaining a training data set; the training data set includes: a number of sample foggy images and corresponding sample defogged images; taking the sample foggy image as input, the defogged image estimation value corresponding to the sample foggy image as output, and taking the mean square error between the defogged image estimation value and the sample defogged image as the loss function, using the training data set to train the deep learning model to obtain the image defogging model.

[0137] In this embodiment, a digital power line database is constructed by collecting foggy images taken by several monitoring devices, and 10,000 foggy images under different haze conditions are selected as a training set. Data enhancement methods such as random rotation, random horizontal flipping, random vertical flipping, and random modulation scaling are applied to each image to improve the generalization ability of model training and ensure that the same number of enhanced images are generated for each image. The enhanced images are then included in the data set, which now contains 40,000 images.

[0138] In order to construct a training network for the hierarchical weight reorganization model based on spatial variation rate, the corresponding hyperparameters need to be set. First, the size of the input feature map is adjusted to 640x640, the initial learning rate is set to 0.01, and the SGD optimizer is used, with the momentum coefficient set to 0.937, the weight decay coefficient to 0.0005, the border loss weight to 0.05, and the classification loss weight to 0.5. The learning rate will be adjusted according to the set decay strategy during the training process, the batch size is set to 64, and the number of training iterations is 300. Finally, the weight parameters of the image defogging model based on hierarchical weight reorganization of spatial variation rate are obtained, and the original foggy image is input into the image defogging model with configured weights, and the model produces a clear defogging image.

[0139] This application proposes an image defogging method based on hierarchical weight reorganization of spatial variation rate for severe weather scenes such as haze, rain, fog, smoke, etc. in real life. The main steps include: first, the spatial variation rate uses the Laplacian operator to measure the intensity and direction of brightness changes in the image. This analysis can highlight the areas where details are blurred and contrast is reduced due to haze in the image, and provide prior information for image defogging. Then, a deep convolutional neural network is used to extract multi-scale features of the image layer by layer. The low-level convolutional layer captures the edge and texture information of the image, while the high-level convolutional layer extracts more abstract semantic features. Secondly, according to the results of the spatial variation rate analysis, the weights of different convolutional layers are adaptively adjusted. For high-variance rate areas (such as edges and contours), the weights of low-level features are increased to highlight the detail enhancement of the image; for low-variance rate areas (such as the sky and haze coverage areas), the weights of high-level features are reorganized to restore the global contrast and clarity of the image. Finally, residual learning is used to learn the difference between the dehazed image and the original image to reduce the difficulty of training and speed up the convergence. The model generates a clear dehazed image at the output by learning the features after layered weight reorganization.

[0140] In an exemplary embodiment, the present application further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0141] In an exemplary embodiment, the present application further provides a computer-readable storage medium storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0142] In an exemplary embodiment, the present application further provides a computer program product, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0143] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the corresponding device owner. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant legal provisions.

[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0145] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An image defogging method based on hierarchical weighted recombination of spatial variation rate, characterized in that: include: Constructing an image defogging model; the image defogging model uses a deep convolutional neural network to extract multi-scale features of an input image, uses the spatial change rate of the input image to adaptively adjust the weights of each scale feature, uses residual learning to determine the residual according to the multi-scale features and weights of the input image, and adds the input image to the residual to obtain an output image; Get the target foggy image; The target foggy image is input into the image defogging model to obtain a target defogging image.

2. The image defogging method based on hierarchical weighted recombination of spatial variation rate according to claim 1 is characterized in that: Build an image dehazing model, including: Build deep learning models; Acquire a training data set; the training data set includes: a number of sample foggy images and corresponding sample defogging images; The sample foggy image is taken as input, the defogging image estimation value corresponding to the sample foggy image is output, and the mean square error between the defogging image estimation value and the sample defogging image is used as the loss function. The training data set is used to train the deep learning model to obtain an image defogging model.

3. The image defogging method based on hierarchical weighted recombination of spatial variation rate according to claim 1 is characterized in that: The image defogging model includes: a spatial change rate calculation module, a feature extraction module, a hierarchical weight reorganization module and a residual learning module; The spatial change rate calculation module is used to calculate the spatial change rate of the input image using the Laplace operator; The feature extraction module is used to extract multi-scale features of the input image using a deep convolutional neural network; The hierarchical weight reorganization module is used to normalize the spatial change rate of the input image, and adaptively adjust the weight of each scale feature according to the normalized result of the spatial change rate; The residual learning module is used to determine the residual according to the multi-scale features and weights of the input image by using the residual learning method, and add the input image to the residual to obtain the output image.

4. The image defogging method based on hierarchical weighted recombination of spatial variation rate according to claim 3 is characterized in that: The Laplace operator is used to calculate the spatial rate of change of the input image, and the expression is: Among them, ΔI is the Laplace operator of the input image I, which is used to represent the spatial change rate. is the second-order partial derivative of the input image I in the x direction, is the second-order partial derivative of the input image I in the y direction.

5. The image defogging method based on hierarchical weighted recombination of spatial variation rate according to claim 3 is characterized in that: A deep convolutional neural network is used to extract the multi-scale features of the input image, which is expressed as: F1=σ(W1*I+b1); F l =σ(W l *F l-1 +b l ); F={F1,F2,...,F L }; Among them, F1 is the output feature map of the first layer of convolution, F2 is the output feature map of the second layer of convolution, and F l-1 is the output feature map of the l-1th layer convolution, F l is the output feature map of the lth layer of convolution, L is the total number of convolution layers, I is the input image, W1 is the weight matrix of the first layer of convolution, W l is the weight matrix of the lth layer of convolution, b1 is the bias term of the first layer of convolution, b l is the bias term of the l-th layer convolution, σ(·) is the nonlinear activation function, and * represents the convolution operation.

6. The image defogging method based on hierarchical weighted reorganization of spatial variation rate according to claim 3 is characterized in that: The weights of each scale feature are adaptively adjusted according to the normalized result of the spatial change rate. The expression is: w l (x,y)=α l ·S(x,y)+β l ·(1-S(x,y)); Among them, w l (x, y) is the adaptive weight function, S(x, y) is the normalized result of the spatial change rate, α l and β l are weight coefficients related to the convolution layer.

7. The image defogging method based on hierarchical weighted reorganization of spatial variation rate according to claim 3 is characterized in that: The residual learning method is used to determine the residual according to the multi-scale features and weights of the input image, and the input image is added to the residual to obtain the output image. The expression is: in, is the output image, I is the input image, l is the layer index of the convolution, L is the total number of convolution layers, and w l is the weight of the l-th layer convolution, F l is the output feature map of the lth convolution layer.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image defogging method based on hierarchical weighted reorganization of spatial variation rate as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image defogging method based on hierarchical weighted reorganization of spatial variation rate described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image defogging method based on hierarchical weighted reorganization of spatial variation rate described in any one of claims 1 to 7 is implemented.

Images