Image sharpening method in complex scene
By combining the DPM diffusion model and the U-net neural network, the problems of computational resource consumption and data dependence in image sharpening under complex scenarios are solved, achieving efficient and accurate image sharpening effects, which are suitable for real-time systems and large-scale data analysis.
Patent Information
- Application Number
- CN202411065769.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-19
AI Technical Summary
In complex scenarios, traditional image sharpening methods are ineffective in handling noise and blur, and deep learning models are highly dependent on data and consume a lot of computing resources, making them difficult to use effectively in real-time processing and mobile applications.
By combining the DPM diffusion model with the U-net neural network, image blurring is regarded as a diffusion process. Deep learning is performed using a Markov chain structure, which combines prior knowledge of the image to reduce computational resource consumption and improve the model's generalization ability.
It achieves efficient image sharpening in complex scenarios, reduces reliance on large amounts of labeled data, improves the model's prediction accuracy and computational efficiency, and is suitable for real-time systems and large-scale data analysis.
Smart Images

Figure CN118982486A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of wireless technology, and more particularly to an image clearing method in complex scenes. Background Art
[0002] Image sharpening plays an important role in the field of modern computer vision. Image sharpening is crucial for applications such as real-time monitoring, medical imaging diagnosis, and long-distance shooting. However, in complex scenarios, such as bad weather, low light conditions, or motion blur, traditional methods and deep learning models have limitations. Traditional filters are not effective for noise and blur in complex scenarios, and traditional interpolation methods may cause loss of detail information when increasing image resolution. Deep learning models require a large amount of clearly labeled data for training, and obtaining this data may be difficult in some complex scenarios. In addition, deep learning models usually require a lot of computing resources, which may not be practical for real-time image processing or mobile applications.
[0003] Therefore, a method for image sharpening in complex scenes is proposed. A lightweight deep learning model is designed, and the prior knowledge of the image, such as edge information and texture information, is combined. These models can not only improve the sharpening effect by embedding prior knowledge, but also reduce the consumption of computing resources. They can also use existing large data sets to pre-train the model, and then fine-tune the model in specific scenes. In this way, the reliance on a large amount of labeled data can be reduced while ensuring the sharpening effect, and the generalization ability of the model can be improved. Summary of the invention
[0004] In view of the shortcomings of the above-mentioned technology, the present invention discloses a method for image sharpening in complex scenes. The image sharpening operation is realized by utilizing a U-net neural network training model based on a DPM diffusion model, thereby greatly improving the deblurring capability, having a simple structure and reducing the consumption of computing resources.
[0005] A method for image clarity in complex scenes comprises the following steps:
[0006] Step 1: Image acquisition and data preprocessing, converting image information in complex environments into image data sets;
[0007] In this step, information fusion is performed using image modalities such as RGB images and depth images to improve the ability to understand and process complex environments; denoising and color correction are performed on the data set to reduce noise and improve image quality; large images are divided into regions and processed in parallel to improve the subsequent model computing capabilities;
[0008] Step 2: Set up the inverse model. Image blurring is considered as a diffusion process. The DPM diffusion probability model algorithm is used as the image sharpening model. The DPM forward process gradually adds Gaussian noise to the clear image. The reverse process uses the Markov chain structure to perform deep learning operations, denoise and restore the clean image to achieve blur image sharpening.
[0009] Step 3, model parameter estimation and reverse optimization, reverse diffusion process modeling Markov chain, state transfer process setting transition probability, using U-net neural network model to receive the current state as input, output conditional probability distribution function to achieve model probability parameterization; define negative ELBO lower bound function as loss function to reverse optimize and adjust the neural network; the U-net neural network model includes an encoder, an intermediate layer and a decoder, the encoder includes a series of convolutional layers and pooling layers, the intermediate layer includes multiple convolutional layers, the decoder includes multiple deconvolutional layers and convolutional layers, wherein the decoder convolutional layer output is connected to the pooling layer input, the pooling layer output is connected to the convolutional layer input of the intermediate layer, the convolutional layer output of the intermediate layer is connected to the decoder deconvolutional layer input, the decoder deconvolutional layer output is connected to the decoder convolutional layer input, U-net implements splicing operation in the channel dimension to achieve the connection between the decoder feature map and the corresponding layer of the decoder;
[0010] Step 4: Improve the model and adjust the model parameters. Evaluate the clarity and detail retention of the sharpened image. Select the SSIM structural similarity index to evaluate the model. Use the test set data to evaluate the performance of the model. Evaluate the accuracy of the model by calculating the difference between the predicted probability distribution and the real data. Obtain the reverse process parameters with better performance.
[0011] Step 5: Display and apply the results to verify the clarity effect in different complex scenes. Use the neural network training to complete the model application to take blurred pictures and compare them with the original images.
[0012] As a further embodiment of the present invention, the denoising process of the data set adopts bilateral filtering, which effectively smoothes the image and retains the edge detail information of the image based on the spatial distance between pixels and the similarity between pixel values. The bilateral filtering formula is:
[0013]
[0014] In formula (1), O(x, y) represents the pixel value of the output image, W P (x, y) represents the normalized weight, Ω represents the pixel neighborhood, which covers the surrounding pixels of pixel (x, y), i and j represent the row and column of the pixel coordinates in the image, I(i, j) represents the grayscale value of the pixel located in the i-th row and j-th column in the image, ω s(x, y, i, j) represents the spatial domain weight, which measures the spatial distance between pixels (x, y) and (i, j), σ s Represents the standard deviation in the spatial domain, and the control weight changes with the spatial distance and speed; ω r (x, y, i, j) represents the pixel value domain weight, which measures the similarity of pixel values between pixels (x, y) and (i, j), σ r Represents the standard deviation in the pixel domain, and controls how fast the weight changes with the difference in pixel values.
[0015] As a further embodiment of the present invention, the DPM diffusion model is an extension of a variational autoencoder. The autoencoding process encodes the input x into z, z is used as the input and x is used as the output to implement the decoding process, and the encoding obtains z as the input to continue to execute the next level encoding process and repeat the expansion multiple times to form a Markov chain structure; the Markov chain is a double chain structure, and the encoding process is executed T times from left to right as a forward process, and the decoding process is executed T times from right to left as a reverse process. Each step T of the forward process and the reverse process is only related to the previous step. The specific formula is as follows:
[0016]
[0017] In formula (2), q φ (z t |z t-1 ) represents the encoding process of a step, p θ (z t-1 |z t ) represents a decoding process of one step, p(x,z 1:T ) represents the joint probability distribution of the entire Markov hierarchical autoencoder, z 1:T Indicates from z1 to z T A complete set of hidden variables is formed, p(x) represents the initial probability of the first step, and q(x) represents the probability of the reverse first step; the forward process continuously adds tiny standard Gaussian noise. When T is greater than 150 times, the original image approaches standard Gaussian noise. The linear Gaussian transformation formula is as follows:
[0018]
[0019] In formula (3), x t Represents the mean and x t-1 There is a linear relationship, α t It means selecting a coefficient less than 1 to achieve a mean close to 0 and a variance close to 1. represents a normal distribution with a mean of 0 and a variance of 1; there are no unknown parameters in the cyclic calculation process, and it is not necessary to calculate x at any time through the model learning process t .
[0020] As a further embodiment of the present invention, the pθ (z t-1 |z t ) in p(z t ) Set the standard normal distribution with mean 0 and variance 1, which cannot be calculated. Fit the inverse model through the U-net neural network; the U-net decoder will encode the image x t The mapping representation feature is mapped back to the image space through the transposed convolution, producing x t-1 Predicted value, the prediction process is represented by x t-1 =Decoder(x t ); the transposed convolution formula is as follows:
[0021]
[0022] In formula (4), where represents the pixel value of the output feature map, i and j represent the height and width of the output feature map, : represents the output channel, represents the convolution kernel size, represents the step size, X represents the input feature map, m and n represent the offset of the convolution kernel in the height direction and the width direction, k represents the channel index of the input feature map, W represents the weight of the transposed convolution, B represents the bias term, and its size is; in the transposed convolution, the convolution kernel has a set of weights for each input channel, which is used to perform convolution operations with the corresponding input channel; for each output channel, the convolution kernel will perform convolution operations with each channel of the input feature map. In the calculation of the transposed convolution, k is a loop variable, which is used to traverse each channel of the input feature map and calculate the value of each pixel position in the output feature map;
[0023] The last layer of the U-net decoder uses the softmax function to convert the network output into a probability distribution to obtain p θ (z t-1 |z t ) probability, the function expression is as follows:
[0024]
[0025] In formula (4), z i Represents the i-th element in the input vector, N represents the length of the input vector, the function exponentializes each input element, and the result is divided by the sum of all exponential results to achieve output as a probability distribution.
[0026] As a further embodiment of the present invention, the ELBO is used as a loss function to make the model approach the true posterior distribution during training through variational inference. The ELBO formula is as follows:
[0027] ELBO(θ,φ)=E qφ(z|x) (πlogp θ (x|z))-D KL (qφ (z|x)||p(z)) (6)
[0028] In formula (6), where p θ (x|z) represents the probability of the decoder generating observation data x given the latent variable z, q φ (z|x) represents the potential variable distribution of the encoder given the observed data x; p(z) represents the prior distribution, D KL Represents KL divergence. ELBO calculation includes reconstruction loss and KL divergence loss. Reconstruction loss measures the decoder to reconstruct x t-1 The KL divergence loss measures the difference between the encoder output distribution and the prior distribution; the chain rule is used to back-propagate the ELBO gradient of the network parameters θ and φ to the encoder and decoder parameters, and the neural network parameters are updated by gradient descent. For each parameter θ i The update rules are as follows:
[0029]
[0030] In formula (7), α represents the learning rate, and the hyperparameters are manually set for training the neural network; the neural network training is achieved by repeating forward propagation, calculating loss, back propagation, and updating parameters 100 times.
[0031] Positive beneficial effects
[0032] The DPM diffusion model method is established through U-net neural network training to achieve image clarity processing. The use of neural network parameterized probability distribution can more flexibly capture the complex relationships in the data, can better adapt to the data, and is not limited by the traditional inverse diffusion modeling method's assumption of linear or specific forms of nonlinearity for transition probabilities; by using neural networks to parameterize conditional probability distributions, the model can more accurately predict transitions between states and improve the model's prediction accuracy; after adopting the neural network model, the entire modeling and training process can be learned end-to-end without manually designing features or models; the outstanding advantages are particularly significant when dealing with complex tasks; after the model training is completed, the reasoning process can be carried out quickly, and this method can play a role in applications that require fast response, such as real-time systems or large-scale data analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:
[0034] Figure 1 A schematic diagram of the overall architecture of an image clearing method in a complex scene according to the present invention;
[0035] Figure 2 A schematic diagram of the connection relationship principle architecture of a neural network model in U-Net in a method for image sharpening in complex scenes of the present invention; DETAILED DESCRIPTION
[0036] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0037] like Figure 1-Figure 2 As shown, a method for image sharpening in a complex scene includes the following steps:
[0038] Step 1: Image acquisition and data preprocessing, converting image information in complex environments into image data sets;
[0039] In this step, information fusion is performed using image modalities such as RGB images and depth images to improve the ability to understand and process complex environments; denoising and color correction are performed on the data set to reduce noise and improve image quality; large images are divided into regions and processed in parallel to improve the subsequent model computing capabilities;
[0040] Step 2: Set up the inverse model. Image blurring is considered as a diffusion process. The DPM diffusion probability model algorithm is used as the image sharpening model. The DPM forward process gradually adds Gaussian noise to the clear image. The reverse process uses the Markov chain structure to perform deep learning operations, denoise and restore the clean image to achieve blur image sharpening.
[0041] Step 3, model parameter estimation and reverse optimization, reverse diffusion process modeling Markov chain, state transfer process setting transition probability, using U-net neural network model to receive the current state as input, output conditional probability distribution function to achieve model probability parameterization; define negative ELBO lower bound function as loss function to reverse optimize and adjust the neural network; the U-net neural network model includes an encoder, an intermediate layer and a decoder, the encoder includes a series of convolutional layers and pooling layers, the intermediate layer includes multiple convolutional layers, the decoder includes multiple deconvolutional layers and convolutional layers, wherein the decoder convolutional layer output is connected to the pooling layer input, the pooling layer output is connected to the convolutional layer input of the intermediate layer, the convolutional layer output of the intermediate layer is connected to the decoder deconvolutional layer input, the decoder deconvolutional layer output is connected to the decoder convolutional layer input, U-net implements splicing operation in the channel dimension to achieve the connection between the decoder feature map and the corresponding layer of the decoder;
[0042] Step 4: Improve the model and adjust the model parameters. Evaluate the clarity and detail retention of the sharpened image. Select the SSIM structural similarity index to evaluate the model. Use the test set data to evaluate the performance of the model. Evaluate the accuracy of the model by calculating the difference between the predicted probability distribution and the real data. Obtain the reverse process parameters with better performance.
[0043] Step 5: Display and apply the results to verify the clarity effect in different complex scenes. Use the neural network training to complete the model application to take blurred pictures and compare them with the original images.
[0044] In the present invention, the U-net neural network of the reverse DPM diffusion model is constructed, which includes an encoder, an intermediate layer and a decoder; the key to the structure lies in the connection between the encoder and the decoder, which helps to retain feature information of different scales and effectively transmit information so that the decoder can better restore the original input;
[0045] Encoder, the encoder consists of a series of convolutional layers and pooling layers, which gradually reduce the size of the feature map and extract abstract feature information. At the end of each layer, a feature map corresponding to the decoder is saved as a jump connection. Each jump connection copies the feature map of the encoder and connects it to the corresponding layer of the decoder so that the decoder can use the features extracted by the encoder;
[0046] The middle layer is used to connect the encoder and decoder, and contains some additional convolutional layers to further extract feature information;
[0047] The decoder consists of a series of deconvolution layers or upsampling layers and convolution layers, which are used to gradually restore the size of the feature map to the size of the original input. At each layer, the decoder receives the feature map from the encoder and connects it with the feature map of its corresponding encoder layer to fuse feature information of different scales. The last layer is connected to the softmax function layer to convert the output of the network into a probability distribution.
[0048] In the present invention, the denoising process of the data set adopts bilateral filtering, which effectively smoothes the image and retains the edge detail information of the image based on the spatial distance between pixels and the similarity between pixel values. The bilateral filtering formula is:
[0049]
[0050] In formula (1), O(x, y) represents the pixel value of the output image, W P (x, y) represents the normalized weight, Ω represents the pixel neighborhood, which covers the surrounding pixels of pixel (x, y), i and j represent the row and column of the pixel coordinates in the image, I(i, j) represents the grayscale value of the pixel located in the i-th row and j-th column in the image, ω s(x, y, i, j) represents the spatial domain weight, which measures the spatial distance between pixels (x, y) and (i, j), σ s Represents the standard deviation in the spatial domain, and the control weight changes with the spatial distance and speed; ω r (x, y, i, j) represents the pixel value domain weight, which measures the similarity of pixel values between pixels (x, y) and (i, j), σ r Represents the standard deviation in the pixel domain, and controls how fast the weight changes with the difference in pixel values.
[0051] In the present invention, the DPM diffusion model is an extension of the variational autoencoder. The autoencoding process encodes the input x into z, z is used as the input and x is used as the output to implement the decoding process, and the encoding obtains z as the input to continue to execute the next level encoding process and repeat the expansion multiple times to form a Markov chain structure; the Markov chain is a double chain structure, and the encoding process is executed T times from left to right as the forward process, and the decoding process is executed T times from right to left as the reverse process. Each step T of the forward process and the reverse process is only related to the previous step. The specific formula is as follows:
[0052]
[0053] In formula (2), q φ (z t |z t-1 ) represents the encoding process of a step, p θ (z t-1 |z t ) represents a decoding process of one step, p(x,z 1:T ) represents the joint probability distribution of the entire Markov hierarchical autoencoder, z 1:T Indicates from z1 to z T A complete set of hidden variables is formed, p(x) represents the initial probability of the first step, and q(x) represents the probability of the reverse first step; the forward process continuously adds tiny standard Gaussian noise. When T is greater than 150 times, the original image approaches standard Gaussian noise. The linear Gaussian transformation formula is as follows:
[0054]
[0055] In formula (3), x t Represents the mean and x t-1 There is a linear relationship, α t It means selecting a coefficient less than 1 to achieve a mean close to 0 and a variance close to 1. represents a normal distribution with a mean of 0 and a variance of 1; there are no unknown parameters in the cyclic calculation process, and it is not necessary to calculate x at any time through the model learning process t .
[0056] In the present invention, the p θ(z t-1 |z t ) in p(z t ) Set the standard normal distribution with mean 0 and variance 1, which cannot be calculated. Fit the inverse model through the U-net neural network; the U-net decoder will encode the image x t The mapping representation feature is mapped back to the image space through the transposed convolution, producing x t-1 Predicted value, the prediction process is represented by x t-1 =Decoder(x t ); the transposed convolution formula is as follows:
[0057]
[0058] In formula (4), Y(i,j,:) represents the pixel value of the output feature map, i and j represent the height and width of the output feature map, : represents the output channel, (K H ,K W ) represents the convolution kernel size, (S H ,S W ) represents the step size, X represents the input feature map, m and n represent the offset of the convolution kernel in the height direction and width direction, k represents the channel index of the input feature map, W represents the weight of the transposed convolution, B represents the bias term, and the size is C out ; In the transposed convolution, the convolution kernel has a set of weights for each input channel, which is used to perform convolution operations with the corresponding input channels; for each output channel, the convolution kernel will perform convolution operations with each channel of the input feature map. In the calculation of the transposed convolution, k is a loop variable, which is used to traverse each channel of the input feature map and calculate the value of each pixel position in the output feature map.
[0059] In the present invention, the last layer of the U-net decoder uses a softmax function to convert the network output into a probability distribution to obtain p θ (z t-1 |z t ) probability, the function expression is as follows:
[0060]
[0061] In formula (4), z i Represents the i-th element in the input vector, N represents the length of the input vector, the function exponentializes each input element, and the result is divided by the sum of all exponential results to achieve output as a probability distribution.
[0062] In the present invention, the ELBO is used as a loss function to make the model approach the true posterior distribution during training through variational inference. The ELBO formula is as follows:
[0063] ELBO(θ,φ)=E qφ(z|x) (πlogp θ (x|z))-D KL (q φ (z|x)||p(z)) (6)
[0064] In formula (6), where p θ (x|z) represents the probability of the decoder generating observation data x given the latent variable z, q φ (z|x) represents the potential variable distribution of the encoder given the observed data x; p(z) represents the prior distribution, D KL Represents KL divergence. ELBO calculation includes reconstruction loss and KL divergence loss. Reconstruction loss measures the decoder to reconstruct x t-1 The KL divergence loss measures the difference between the encoder output distribution and the prior distribution; the chain rule is used to back-propagate the ELBO gradient of the network parameters θ and φ to the encoder and decoder parameters, and the neural network parameters are updated by gradient descent. For each parameter θ i The update rules are as follows:
[0065]
[0066] In formula (7), α represents the learning rate, and the hyperparameters are manually set for training the neural network; the neural network training is achieved by repeating forward propagation, calculating loss, back propagation, and updating parameters 100 times.
[0067] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that these specific embodiments are only illustrative, and those skilled in the art may omit, replace, and change the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, merging the above method steps so as to perform substantially the same functions in substantially the same manner to achieve substantially the same results is within the scope of the present invention. Therefore, the scope of the present invention is limited only by the appended claims.
Claims
1. A method for image sharpening in complex scenes, characterized by: The following steps are involved: Step 1: Image acquisition and data preprocessing, converting image information in complex environments into image data sets; In this step, information fusion is performed using image modalities such as RGB images and depth images to improve the ability to understand and process complex environments; denoising and color correction are performed on the data set to reduce noise and improve image quality; large images are divided into regions and processed in parallel to improve the subsequent model computing capabilities; Step 2: Set up the inverse model. Image blurring is considered as a diffusion process. The DPM diffusion probability model algorithm is used as the image sharpening model. The DPM forward process gradually adds Gaussian noise to the clear image. The reverse process uses the Markov chain structure to perform deep learning operations, denoise and restore the clean image to achieve blur image sharpening. Step 3, model parameter estimation and reverse optimization, reverse diffusion process modeling Markov chain, state transfer process setting transition probability, using U-net neural network model to receive the current state as input, output conditional probability distribution function to achieve model probability parameterization; define negative ELBO lower bound function as loss function to reverse optimize and adjust the neural network; the U-net neural network model includes an encoder, an intermediate layer and a decoder, the encoder includes a series of convolutional layers and pooling layers, the intermediate layer includes multiple convolutional layers, the decoder includes multiple deconvolutional layers and convolutional layers, wherein the decoder convolutional layer output is connected to the pooling layer input, the pooling layer output is connected to the convolutional layer input of the intermediate layer, the convolutional layer output of the intermediate layer is connected to the decoder deconvolutional layer input, the decoder deconvolutional layer output is connected to the decoder convolutional layer input, U-net implements splicing operation in the channel dimension to achieve the connection between the decoder feature map and the corresponding layer of the decoder; Step 4: Improve the model and adjust the model parameters. Evaluate the clarity and detail retention of the sharpened image. Select the SSIM structural similarity index to evaluate the model. Use the test set data to evaluate the performance of the model. Evaluate the accuracy of the model by calculating the difference between the predicted probability distribution and the real data. Obtain the reverse process parameters with better performance. Step 5: Display and apply the results to verify the clarity effect in different complex scenes. Use the neural network training to complete the model application to take blurred pictures and compare them with the original images.
2. The method for image sharpening in complex scenes according to claim 1, characterized in that: The denoising process of the data set adopts bilateral filtering, which effectively smoothes the image and retains the edge detail information of the image based on the spatial distance between pixels and the similarity between pixel values. The formula is as follows: In formula (1), O(x, y) represents the pixel value of the output image, W P (x, y) represents the normalized weight, Ω represents the pixel neighborhood, which covers the surrounding pixels of pixel (x, y), i and j represent the row and column of the pixel coordinates in the image, I(i, j) represents the grayscale value of the pixel located in the i-th row and j-th column in the image, ω s (x, y, i, j) represents the spatial domain weight, which measures the spatial distance between pixels (x, y) and (i, j), σ s Represents the standard deviation in the spatial domain, and the control weight changes with the spatial distance and speed; ω r (x, y, i, j) represents the pixel value domain weight, which measures the similarity of pixel values between pixels (x, y) and (i, j), σ r Represents the standard deviation in the pixel domain, and controls how fast the weight changes with the difference in pixel values.
3. The method for image sharpening in complex scenes according to claim 1, characterized in that: The DPM diffusion model is an extension of the variational autoencoder. The autoencoding process encodes the input x into z, z is used as input and x is used as output to implement the decoding process. The encoding obtains z as input and continues to execute the next level encoding process repeatedly and expands multiple times to form a Markov chain structure; the Markov chain is a double chain structure, and the encoding process is executed T times from left to right as the forward process, and the decoding process is executed T times from right to left as the reverse process. Each step T of the forward process and the reverse process is only related to the previous step. The specific formula is as follows: In formula (2), q φ (z t |z t-1 ) represents the encoding process of a step, p θ (z t-1 |z t ) represents a decoding process of one step, p(x,z 1:T ) represents the joint probability distribution of the entire Markov hierarchical autoencoder, z 1:T Indicates from z1 to z T A complete set of hidden variables is formed, p(x) represents the initial probability of the first step, and q(x) represents the probability of the reverse first step; the forward process continuously adds tiny standard Gaussian noise. When T is greater than 150 times, the original image approaches standard Gaussian noise. The linear Gaussian transformation formula is as follows: In formula (3), x t Represents the mean and x t-1 Linear relationship, α t It means selecting a coefficient less than 1 to achieve a mean close to 0 and a variance close to 1. represents a normal distribution with a mean of 0 and a variance of 1; there are no unknown parameters in the cyclic calculation process, and it is not necessary to calculate x at any time through the model learning process t .
4. The method for image sharpening in complex scenes according to claim 3, characterized in that: The p θ (z t-1 |z t ) in p(z t ) Set the standard normal distribution with mean 0 and variance 1, which cannot be calculated. Fit the inverse model through the U-net neural network; the U-net decoder will encode the image x t The mapping representation feature is mapped back to the image space through the transposed convolution, producing x t-1 Predicted value, the prediction process is represented by x t-1 =Decoder(x t ); the transposed convolution formula is as follows: In formula (4), Y(i,j,:) represents the pixel value of the output feature map, i and j represent the height and width of the output feature map, : represents the output channel, (K H ,K W ) represents the convolution kernel size, (S H ,S W ) represents the step size, X represents the input feature map, m and n represent the offset of the convolution kernel in the height direction and width direction, k represents the channel index of the input feature map, W represents the weight of the transposed convolution, B represents the bias term, and the size is C out ; In the transposed convolution, the convolution kernel has a set of weights for each input channel, which is used to perform convolution operations with the corresponding input channels; for each output channel, the convolution kernel will perform convolution operations with each channel of the input feature map. In the calculation of the transposed convolution, k is a loop variable, which is used to traverse each channel of the input feature map and calculate the value of each pixel position in the output feature map.
5. The method for image sharpening in complex scenes according to claim 1, characterized in that: The last layer of the U-net decoder uses the softmax function to convert the network output into a probability distribution to obtain p θ (z t-1 |z t ) probability, the function expression is as follows: In formula (4), z i Represents the i-th element in the input vector, N represents the length of the input vector, the function exponentializes each input element, and the result is divided by the sum of all exponential results to achieve output as a probability distribution.
6. The method for image sharpening in complex scenes according to claim 1, characterized in that: Set ELBO as the loss function and use variational inference to make the model approach the true posterior distribution during training. The ELBO formula is as follows: ELBO(θ,φ)=E qφ(z|x) (πlogp θ (x|z))-D KL (q φ (z|x)||p(z)) (6) In formula (6), where p θ (x|z) represents the probability of the decoder generating observation data x given the latent variable z, q φ (z|x) represents the potential variable distribution of the encoder given the observed data x; p(z) represents the prior distribution, D KL Represents KL divergence. ELBO calculation includes reconstruction loss and KL divergence loss. Reconstruction loss measures the decoder to reconstruct x t-1 The KL divergence loss measures the difference between the encoder output distribution and the prior distribution; the chain rule is used to back-propagate the ELBO gradient of the network parameters θ and φ to the encoder and decoder parameters, and the neural network parameters are updated by gradient descent. For each parameter θ i The update rules are as follows: In formula (7), α represents the learning rate, and the hyperparameters are manually set for training the neural network; the neural network training is achieved by repeating forward propagation, calculating loss, back propagation, and updating parameters 100 times.
7. The method for image sharpening in complex scenes according to claim 1, characterized in that: The evaluation model uses part of the data set as a validation set to evaluate the performance of the model on unseen data, monitor the loss and accuracy on the validation set, and determine whether the model is overfitting or underfitting.
Citation Information
Cited By
Petrochemical park gas leakage diffusion prediction method and system, electronic equipment and storage medium
CN119623358A
Image big data analysis method based on feedback enhancement
CN119648709A
Quality optimization method for aerial image of unmanned aerial vehicle and related equipment
CN121073824A
