Road surface defect image super-resolution reconstruction method based on improved SwinIR

Abstract

The invention discloses a pavement defect image super-resolution reconstruction method based on improved SwinIR, and the method specifically comprises the following steps: 1, obtaining a pavement defect image, and constructing a first data set; 2, generating a second data set based on the first data set; 3, constructing a pavement defect image super-resolution reconstruction model based on the improved SwinIR; step 4, designing an improved L1 loss function combined with a Wasserstein distance; 5, training the model by adopting the training set and the verification set in the second data set; 6, performing performance evaluation on the model by using the test set of the second data set; and 7, jointly inputting the model weight obtained by training in the step 5 and the first data set into an improved SwinIR model, and outputting a high-resolution image. Compared with the prior art, the pavement image super-resolution reconstruction method based on the improved SwinIR provided by the invention can effectively improve the resolution of the pavement defect image and enhance the pavement defect detail recovery capability.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and particularly to a method for super-resolution reconstruction of pavement defect images based on improved SwinIR. Background Art

[0002] With the increase in the service life of highways and the influence of environmental factors (such as temperature changes, traffic loads, rain erosion, etc.), pavement defect problems gradually emerge, such as cracks, potholes, etc. If these defects are not detected and repaired in a timely manner, it will not only significantly shorten the service life of the highway, but may also lead to traffic safety accidents. Therefore, regular pavement inspections are crucial for ensuring the safety of highways and extending their service life.

[0003] However, the current pavement defect detection methods still face many challenges. The traditional pavement defect image acquisition methods are affected by the quality of imaging devices and external environments (such as light changes, climate conditions, etc.), resulting in low-quality acquired images with blurred details. Especially when the defects are relatively subtle, it is difficult for imaging devices to clearly capture these features. In addition, pavement defect images are often interfered by noise, further reducing the image clarity and affecting the accuracy of subsequent defect recognition and pavement performance analysis. If defect detection is directly performed on low-quality images, the results often have large errors. Therefore, super-resolution reconstruction on low-resolution and blurred pavement image data has become the key to improving detection accuracy.

[0004] Traditional image processing methods mainly use interpolation algorithms, such as bilinear interpolation, cubic interpolation, etc. These methods are simple to calculate, have a fast running speed, and can meet the requirements of real-time processing. However, since the interpolation methods only rely on the existing pixel information in the low-resolution image and do not fully explore potential image features, when dealing with complex textures and fine structures, problems such as missing details and blurred edges will occur in the reconstructed image, thus affecting the accuracy of crack recognition. In contrast, the super-resolution reconstruction method based on deep learning can learn the mapping relationship between low-resolution and high-resolution images in an end-to-end manner, significantly improving the image detail restoration effect, providing a clearer and more accurate input image for pavement defect detection, and having important application value in highway maintenance and safety management. However, with the increase in the number of layers of deep learning models, the model scale continues to expand, and there are still great challenges in deploying on resource-constrained in-vehicle platforms or edge devices. Summary of the Invention

[0005] Aiming at the problems existing in the prior art, the present invention proposes a pavement defect image super-resolution reconstruction method based on improved SwinIR. Through the improved SwinIR model, the quality of pavement defect images affected by device performance and environmental interference is effectively enhanced, making subtle defects easier to identify, thereby improving the accuracy of subsequent pavement defect detection. At the same time, this method reduces the computational overhead while maintaining the reconstruction effect and can be applied to practical scenarios with limited resources.

[0006] The technical solution provided by the present invention includes the following steps:

[0007] Step 1: Obtain pavement defect images and construct a first data set; the pavement images in the first data set can be taken by a digital camera or obtained from a driving recorder or surveillance video.

[0008] Step 2: Generate a second data set based on the first data set; the second data set generates low-resolution image pairs corresponding to high-resolution images by performing two-fold downsampling on the first data set and divides them into a training set, a validation set, and a test set.

[0009] Step 3: Construct a pavement defect image super-resolution reconstruction model based on improved SwinIR; further, step 3 includes steps 3.1 to 3.3:

[0010] Step 3.1: Input the low-quality image training set and validation set of the second data set into the shallow feature extraction module, and the shallow feature extraction module uses a 3×3 convolutional layer to extract initial features and generate a preliminary feature map.

[0011] Step 3.2: Input the preliminary feature map into the deep feature extraction module, and the deep feature extraction module consists of 5 RSTB modules, 5 SimAM attention mechanisms, and 1 3×3 convolution, where each RSTB module contains 6 STL layers and 1 depthwise separable convolution module.

[0012] Specifically, a SimAM attention mechanism is added after each RSTB module in the SwinIR model to optimize the feature extraction process; and a depthwise separable convolution is introduced inside the RSTB module to reduce the number of model parameters.

[0013] Step 3.3: Send the output of the deep feature extraction module to the high-quality restoration module for image reconstruction, and the high-quality restoration module consists of an upsampling module and a 3×3 convolution.

[0014] Specifically, the upsampling module uses the PixelShuffle method to upsample the feature map to improve the image resolution; the upsampled feature map is further refined through a 3×3 convolutional layer to effectively repair the image details and finally output a high-quality reconstructed image.

[0015] Step 4: Design an L1 loss function improved by combining the Wasserstein distance;

[0016] Specifically, for the improved L1 loss function, its calculation formula is:

[0017] L Total = loss_weight × L L1 + wass_weight × L Wass (1)

[0018] Among them, L L1 represents L1Loss, which is used to measure the absolute difference between the predicted value and the true value; L Wass represents the Wasserstein distance loss, which is used to measure the distribution difference between the predicted value and the true value; loss_weight and wass_weight are the weights of L1Loss and the Wasserstein distance respectively, and the values are 1.0 and 0.1 here;

[0019] The calculation formula of L1 Loss is:

[0020]

[0021] The calculation formula of the Wasserstein distance is:

[0022]

[0023] Among them, f(x i ) and y i represent the predicted value and the corresponding true value of the i-th sample respectively, n is the number of samples, and Hist(·) represents the histogram operation.

[0024] Step 5: Use the training set and validation set of the second dataset to train the pavement defect image super-resolution reconstruction model described in Step 3, and save the trained model and weights;

[0025] Furthermore, Step 5 specifically includes Steps 5.1 to 5.4:

[0026] Step 5.1: Set the training parameters of the pavement defect image super-resolution reconstruction model based on the improved SwinIR. The model training parameters include: reconstruction ratio, learning rate, optimizer, number of iteration rounds, batch size;

[0027] Step 5.2: Input the training set and validation set image pairs of the second dataset into the improved SwinIR pavement defect image super-resolution reconstruction model, use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, and update the model parameters according to the gradient to gradually reduce the loss function;

[0028] Step 5.3: Monitor the loss function value and performance metrics during the training process. When the loss function of the training set no longer decreases and the evaluation metrics PSNR and SSIM no longer improve, stop the training to avoid overfitting of the model;

[0029] Step 5.4: After the training is completed, select the optimal model from them and save the training weights.

[0030] Step 6: Use the test set of the second dataset to test the optimal model selected in Step 5.4 and obtain the final pavement defect super-resolution reconstruction model;

[0031] Further, Step 6 specifically includes Steps 6.1 to 6.3:

[0032] Step 6.1: Input the test set image pairs into the optimal model selected in Step 5 for testing;

[0033] Step 6.2: Calculate the model performance metric PSNR. The specific calculation formula of PSNR is as follows:

[0034]

[0035] where x(i,j) and y(i,j) represent the values of image x and image y at the i,j pixel positions respectively; m and n represent the width and height of the image, and MAX x is the maximum pixel value in image x;

[0036] Calculate the model performance metric SSIM. The specific calculation formula of SSIM is as follows:

[0037]

[0038] where μ x and μ y represent the local means of two images x and y respectively, used to estimate the brightness; c1 and c2 are constant terms to prevent the divisor from being zero; and represent the local variances of images x and y respectively, used to measure the contrast of the images; σ xy represents the covariance of images x and y, used to measure the structural similarity between the images;

[0039] Step 6.3: Evaluate the performance metrics of the test set. If the PSNR and SSIM metrics of the test set are similar to those of the training set, it indicates that the model meets the generalization requirements, and the final super-resolution reconstruction model for road surface defect images based on the improved SwinIR is obtained.

[0040] Step 7: Input the first data set and the training weights saved in Step 5 into the improved SwinIR super-resolution reconstruction model for road surface defect images for reconstruction, and the final super-resolution reconstructed image is obtained.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] (1) The present invention discloses a super-resolution reconstruction method for road surface defect images based on the improved SwinIR. The method introduces the SimAM attention mechanism into the deep feature extraction module of SwinIR to enhance the model's feature perception and extraction ability for road surface defect areas, effectively improving the recognition rate of road surface defect areas.

[0043] (2) The method introduces depthwise separable convolutions inside the RSTB module to reduce the computational complexity while maintaining the ability to reconstruct the details of road surface defect images, achieving model lightweight while ensuring that the reconstruction effect is not significantly affected.

[0044] (3) The method uses an L1 loss function improved based on the Wasserstein distance to enhance the optimization effect of the model in the super-resolution reconstruction task, making the reconstructed road surface defect images clearer, reducing the interference of artifacts, and thus improving the accuracy of subsequent road surface defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of the super-resolution reconstruction method for road surface defect images based on the improved SwinIR of the present invention;

[0046] Figure 2 is an architecture diagram of the super-resolution reconstruction model for road surface defect images based on the improved SwinIR of the present invention;

[0047] Figure 3 is a schematic diagram of the SimAM attention mechanism;

[0048] Figure 4 is a schematic diagram inside the RSTB module;

[0049] Figure 5 is the processing flow of depthwise separable convolutions; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] In order to make the technical solutions, structural features, achieved objectives and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be noted that the specific embodiments described herein are only used to explain the present invention more clearly and are not used to limit the present invention.

[0051] Figure 1 FIG. is a flowchart of a method for super-resolution reconstruction of pavement defect images based on improved SwinIR disclosed by the present invention, and its implementation process is as follows:

[0052] Step 1: Obtain pavement defect images and construct a first data set. In the first data set, pavement defect images can be captured by a digital camera, obtained from a driving recorder or a surveillance video.

[0053] In this embodiment, in order to better evaluate the reconstruction effect of a method for super-resolution reconstruction of pavement defect images based on improved SwinIR disclosed by the present invention, the Japan data in the publicly available RDD2022 data set is used as the first data set.

[0054] Step 2: Generate a second data set based on the first data set. The second data set generates a low-resolution image pair corresponding to the high-resolution image by performing two-fold downsampling on the first data set, and divides it into a training set, a validation set and a test set.

[0055] Preferably, the divided data set includes: 6320 high-resolution - low-resolution image pairs as the training set, 790 as the test set, and 10 as the validation set. In this embodiment, the size of the low-resolution image is 300×300, and the corresponding size of the high-resolution image is 600×600.

[0056] Step 3: Construct a super-resolution reconstruction model for pavement defect images based on improved SwinIR. The structure of the improved SwinIR model is as Figure 2 shown, and the model construction process specifically includes steps 3.1 to 3.3:

[0057] Step 3.1: Input the low-quality image training set and validation set of the second data set into the shallow feature extraction module. The shallow feature extraction module uses a 3×3 convolutional layer to extract initial features and generate a preliminary feature map.

[0058] Step 3.2: Input the preliminary feature map into the deep feature extraction module. The deep feature extraction module consists of 5 RSTB modules, 5 SimAM attention mechanisms and 1 3×3 convolution. Among them, each RSTB module contains 6 STL layers and 1 depthwise separable convolution module.

[0059] Specifically, the SimAM attention mechanism is added after each RSTB module of the SwinIR model to optimize the feature extraction process; and depthwise separable convolutions are introduced inside the RSTB module to reduce the number of model parameters.

[0060] Furthermore, the structural diagram of the SimAM attention mechanism is as Figure 3 shown: It is a 3-D parameter-free attention module ( Figure 3 c), compared with 1-D attention ( Figure 3 a) and 2-D attention ( Figure 3 b), it simultaneously focuses on the importance of both channel and spatial features, and can derive three-dimensional weights without adding extra parameters to the network. SimAM does not require adding extra training layers and parameters to the original network, can evaluate the importance of each neuron, and has high computational efficiency. The energy function formula for defining each neuron is shown in Equation (1):

[0061]

[0062] In the formula: e t represents the obtained attention weight; M is the number of samples; w t and b t represent the bias and weight respectively; y represents the sample label; x i represents the feature vector of the i-th sample; λ is the coefficient of the regularization term in the formula; t represents the mean of the current feature channel, where the parameters w t and b t have a fast closed-form solution and can be solved without iteration or other optimization algorithms, as shown in Formulas (2) and (3):

[0063]

[0064] In the formula: μ t and σ t 2 represent the mean and variance of all neurons except t, and λ is the coefficient of the regularization term;

[0065] Furthermore, a lightweight RSTB module is designed in the deep feature extraction stage, as Figure 4 shown. By introducing depthwise separable convolutions, the number of model parameters is significantly reduced.

[0066] Among them, depthwise separable convolution mainly includes two steps: depth convolution and pointwise convolution, and its processing flow is as Figure 5As shown below. First, the depth convolution performs a K×K convolution on each channel of the input feature separately, calculating only in the spatial dimension without changing the number of channels, thereby extracting local features. Second, the pointwise convolution uses a 1×1 convolution to fuse the information of all channels and adjust the number of channels to enhance the feature expression ability.

[0067] Step 3.3: Send the output of the deep feature extraction module to the high-quality restoration module for image reconstruction. The high-quality restoration module consists of an upsampling module and a 3×3 convolution;

[0068] Specifically, the upsampling module uses the PixelShuffle method to upsample the feature map to improve the image resolution; the upsampled feature map is further refined through a 3×3 convolution layer to effectively repair the image details, and finally outputs a high-quality reconstructed image.

[0069] Step 4: Design an L1 loss function improved by combining the Wasserstein distance, and its calculation formula is:

[0070] L Total = loss_weight × L L1 + wass_weight × L Wass (4)

[0071] Among them, L L1 represents L1Loss, which is used to measure the absolute difference between the predicted value and the true value; L Wass represents the Wasserstein distance loss, which is used to measure the distribution difference between the predicted value and the true value; loss_weight and wass_weigiht are the weights of L1Loss and the Wasserstein distance respectively, and the values are 1.0 and 0.1 here;

[0072] The calculation formula of L1 Loss is:

[0073]

[0074] The calculation formula of the Wasserstein distance is:

[0075]

[0076] Among them, f(x i ) and y i represent the predicted value and the corresponding true value of the i-th sample respectively, n is the number of samples, and Hist(·) represents the histogram operation;

[0077] Step 5: Use the training set and validation set of the second dataset to train the pavement defect image super-resolution reconstruction model described in Step 3, and save the trained model and its weights. The said Step 5 further includes Steps 5.1 to 5.4:

[0078] Step 5.1: Set the training parameters of the pavement defect image super-resolution reconstruction model based on improved SwinIR. The model training parameters include: reconstruction ratio, learning rate, optimizer, number of iterations, batch size.

[0079] In this embodiment, the optimizer reconstruction ratio is 2, the initial learning rate lr is 2e-4, the optimizer is Adam, the number of iteration rounds total_iter is 500000, and the batch size is 1.

[0080] Step 5.2: Input the training set and validation set image pairs of the second dataset into the improved SwinIR pavement defect image super-resolution reconstruction model. Use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, and update the model parameters according to the gradient to gradually reduce the loss function.

[0081] Specifically, in each training iteration, first, the model calculates the feature representation of the low-resolution image through forward propagation, and gradually restores the high-resolution details through the hierarchical window multi-head self-attention (W-MSA) and RSTB modules, and finally generates a super-resolution reconstructed image;

[0082] Secondly, calculate the error between the reconstructed image and the real high-resolution image, and calculate the loss function based on the error (such as based on the improved L1 LOSS);

[0083] Thirdly, use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, and update the model parameters through the optimization algorithm;

[0084] Repeat the above process, and the model parameters are gradually optimized, making the generated high-resolution image closer to the real image, thereby improving the super-resolution reconstruction effect.

[0085] Step 5.3: Monitor the loss function value and performance metrics during the training process. When the loss function of the training set no longer decreases and the evaluation metrics PSNR and SSIM no longer improve, stop the training to avoid overfitting of the model.

[0086] Step 5.4: After the training is completed, select the optimal model from them and save the training weights.

[0087] Step 6: Use the test set of the second dataset to test the optimal model selected in Step 5.4, and obtain the final pavement defect super-resolution reconstruction model. The said Step 6 further includes Steps 6.1 to 6.3:

[0088] Step 6.1: Input the test set image pairs into the optimal model selected in Step 5 for testing.

[0089] Step 6.2: Calculate the model performance metric PSNR. The PSNR (Peak Signal-to-Noise Ratio) is mainly used to evaluate the quality of an image after compression or reconstruction processing. The higher its value, the smaller the image distortion and the better the reconstruction effect. The specific calculation formula of PSNR is as follows:

[0090]

[0091] where x(i,j) and y(i,j) represent the values of the image x and the image y at the i,j pixel positions respectively; m and n represent the width and height of the image, and MAX x is the maximum pixel value in the image x.

[0092] Calculate the model performance metric SSIM. The SSIM (Structural Similarity Index) combines the distortions of the contrast, structure, and brightness of an image for modeling, so that the output result is between [0,1]. The closer the SSIM value is to 1, the smaller the image distortion, that is, the better the image reconstruction effect. The specific calculation formula of SSIM is as follows:

[0093]

[0094] where μ x and μ y represent the local means of the two images x and y respectively, used to estimate the brightness; c1 and c2 are constant terms to prevent the divisor from being zero; and represent the local variances of the images x and y, used to measure the contrast of the images; σ xy represents the covariance of the images x and y, used to measure the structural similarity between the images.

[0095] The number of model parameters is used to measure the complexity of the neural network model, that is, the total number of weights that the model needs to learn and store during training and inference. The smaller the number of parameters, usually the lighter the model, the lower the computational resource consumption, and it is suitable for actual deployment. Therefore, on the premise of ensuring the image reconstruction effect, the smaller the number of parameters, the better.

[0096] Step 6.3: Evaluate the performance metrics of the test set. If the PSNR and SSIM metrics of the test set are similar to those of the training set, it indicates that the model meets the requirements of generalization, and the final super-resolution reconstruction model of pavement defect images based on the improved SwinIR is obtained.

[0097] Step 7: Input the first data set together with the training weights saved in Step 5 into the improved SwinIR pavement defect image super-resolution reconstruction model for reconstruction, that is, the final super-resolution reconstructed image is obtained.

[0098] In this embodiment, in order to verify the effect of the pavement defect image super-resolution reconstruction model based on the improved SwinIR disclosed in the present invention, the EDSR model, the HAN model, the SwinIR model, the HAT model and the model proposed in the present invention are used to test on the second data set, and the evaluation results are shown in Table 1. Among them, the model proposed in the present invention is superior to other comparison models in the evaluation indexes PSNR, SSIM of the pavement defect image super-resolution reconstruction data, and the model parameters.

[0099] Table 1 Comparison experiment results

[0100]

[0101] In this embodiment, in order to further verify the effectiveness of the improved SwinIR model, the reconstructed images before and after improvement of the present invention are respectively input into the YOLOv7 model for pavement defect detection, and the detection results are compared and analyzed in detail. As shown in Table 2, the improved model has improved in indexes such as map@0.5, Precision, Recall, etc.

[0102] Table 2 Pavement defect detection results before and after processing by the improved SwinIR algorithm

[0103]

[0104] In this embodiment, in order to verify the effectiveness of the pavement defect image super-resolution reconstruction model based on the improved SwinIR disclosed in the present invention, an ablation experiment is carried out based on the self-made data set, and the evaluation results are shown in Table 3. Among them, the model proposed in the present invention is superior to other models in the evaluation indexes PSNR, SSIM of the pavement defect image super-resolution reconstruction data, and the model parameters.

[0105] Table 3 Ablation experiment results

[0106]

[0107] The above is only one embodiment of the present invention, and thus does not limit the patent scope of the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An image super-resolution reconstruction method for pavement defects based on improved SwinIR, characterized in that, Specifically, it includes the following steps: Step 1: Obtain pavement defect images and construct the first dataset; in the first dataset, pavement defect images can be captured by a digital camera, obtained from a driving recorder or a surveillance video; Step 2: Generate the second dataset based on the first dataset. The second dataset is generated by downsampling the first dataset by a factor of two to generate low-resolution image pairs corresponding to high-resolution images, and dividing them into a training set, a validation set, and a test set; Step 3: Construct a pavement defect image super-resolution reconstruction model based on improved SwinIR. The model includes a shallow feature extraction module, a deep feature extraction module, and a high-quality restoration module. The construction of the model further includes steps 3.1 to 3.3: Step 3.1: Input the low-quality image training set and validation set of the second dataset into the shallow feature extraction module. The shallow feature extraction module uses a 3×3 convolutional layer to extract initial features and generate a preliminary feature map; Step 3.2: Input the preliminary feature map into the deep feature extraction module. The deep feature extraction module consists of 5 RSTB modules, 5 SimAM attention mechanisms, and 1 3×3 convolution. Among them, each RSTB module contains 6 STL layers and 1 depthwise separable convolution module; Step 3.3: Send the output of the deep feature extraction module to the high-quality restoration module for image reconstruction. The high-quality restoration module consists of an upsampling module and a 3×3 convolution; the upsampling module uses the PixelShuffle method to upsample the feature map to improve the image resolution; the upsampled feature map is further refined through a 3×3 convolutional layer to effectively repair image details, and finally output a high-quality reconstructed image; Step 4: Design an L1 loss function improved by combining the Wasserstein distance, and its calculation formula is: L Total = loss_weight × L L1 + wass_weight × L Wass (1) Among them, L L1 represents the L1Loss, which is used to measure the absolute difference between the predicted value and the true value; L Wass represents the Wasserstein distance loss, which is used to measure the distribution difference between the predicted value and the true value; loss_weight and wass_weight are the weights of the L1Loss and the Wasserstein distance respectively; The calculation formula of L1 Loss is: The calculation formula of the Wasserstein distance is: where f(x i ) and y i represent the predicted value and the corresponding true value of the i-th sample respectively, n is the number of samples, and Hist(·) represents the histogram operation; Step 5: Use the training set and validation set of the second dataset to train the pavement defect image super-resolution reconstruction model based on improved SwinIR in step 3, and save the trained model and its weights. Step 5 further includes steps 5.1 to 5.4: Step 5.1: Set the training parameters of the pavement defect image super-resolution reconstruction model based on improved SwinIR; Step 5.2: Input the training set and validation set image pairs of the second dataset into the improved SwinIR pavement defect image super-resolution reconstruction model, use the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters, and update the model parameters according to the gradient to gradually reduce the loss function; Step 5.3: Monitor the loss function value and performance metrics during the training process. When the loss function of the training set no longer decreases and the evaluation metrics PSNR and SSIM no longer improve, stop the training to avoid overfitting of the model; Step 5.4: After the training is completed, select the optimal model and save the training weights; Step 6: Use the test set of the second data set to test the optimal model selected in Step 5.4, and evaluate the test results. If the PSNR and SSIM of the model meet the generalization requirements, the final pavement defect image super-resolution reconstruction model based on the improved SwinIR is obtained. Step 6 further includes Steps 6.1 to 6.3: Step 6.1: Input the test set image pairs into the optimal model selected in Step 5 for testing; Step 6.2: Calculate the model performance metric PSNR. The specific calculation formula for PSNR is as follows: Where x(i,j) and y(i,j) respectively represent the values of the i,j pixel positions in image x and image y; m and n represent the width and height of the image, and MAX x is the maximum pixel value in image x; Calculate the model performance metric SSIM. The specific calculation formula for SSIM is as follows: Among them, μ x and μ y respectively represent the local means of two images x and y, which are used to estimate the brightness; c1 and c2 are constant terms to prevent the divisor from being zero; and represent the local variances of images x and y, which are used to measure the contrast of the images; σ xy represents the covariance of images x and y, which is used to measure the structural similarity between the images; Step 6.3: Evaluate the performance metrics of the test set. If the PSNR and SSIM metrics of the test set are similar to those of the training set, it indicates that the model meets the generalization requirements, and the final pavement defect image super-resolution reconstruction model based on the improved SwinIR is obtained; Step 7: Input the first data set and the training weights saved in Step 5 into the improved SwinIR pavement defect image super-resolution reconstruction model for reconstruction, and the final super-resolution reconstruction image is obtained.

Images