A method for constructing a road surface pit semantic segmentation model based on an improved PSPNet
Patent Information
- Application Number
- CN202610804833.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-05
- Publication Date
- 2026-10-16
AI Technical Summary
[0006]本申请的主要目的在于提供一种基于改进PSPNet的路面坑槽语义分割模型的构建方法，旨在解决如何提升路面坑槽的语义分割精度的问题
[0021] An improved version of PS-PSPNet was used to train and validate on a public road pothole dataset. A shallow feature branch was constructed in the decoder and fused with the original decoder output containing PPM, optimizing the model's segmentation performance. Simultaneously, multi-segmentation head training supervision was applied to the model's final output mask and the shallow feature branch mask, and a new composite loss function was proposed to train the semantic segmentation model, effectively improving its segmentation accuracy. Experiments show that PS-PSPNet has a significant advantage in semantic pothole segmentation accuracy compared to other mainstream models, providing more refined and accurate pothole segmentation results, and offering richer and more effective data support for future road maintenance and repair.
Smart Images
Figure CN122929191A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of deep learning technology, and in particular to a method for constructing a road surface pothole semantic segmentation model based on an improved PSPNet and the road surface pothole semantic segmentation model. Background Technology
[0002] Asphalt pavement is a crucial component of public transportation, and the quality of the pavement itself is closely linked to traffic safety. Therefore, the repair and maintenance of pavement defects are essential. Potholes are a common type of pavement defect, typically presenting as large-area damage with varying shapes and significantly lower elevations than the surrounding pavement. They represent a major hidden danger to traffic surfaces, making automated and accurate pothole detection crucial. To ensure high-quality pavement maintenance and traffic safety, research on pothole detection has emerged continuously in recent years. However, mainstream detection methods have varying degrees of shortcomings and cannot meet the demands for automated and accurate pothole detection.
[0003] Among the relevant technical solutions, deep learning-based semantic segmentation is a mainstream method for detecting potholes on highway surfaces. Compared with traditional image processing methods, deep learning-based methods not only have stronger generalization capabilities but also higher detection accuracy.
[0004] However, mainstream semantic segmentation models, such as PSPNet and FCN, focus excessively on extracting deep semantic features while neglecting the importance of shallow features and lacking multi-scale feature fusion. As a result, they frequently miss segments and their segmentation accuracy for road surface defects and potholes is not ideal. In addition, traditional semantic segmentation models use a single segmentation detection head, and supervision is usually performed at the final output position of the model. However, deep learning-based semantic segmentation network structures, such as ResNet50 and VGG16, are often quite deep. Supervising only the final output mask of the model is insufficient, and there is a lack of targeted supervision and training for feature maps at other locations in the model.
[0005] In view of this, this application proposes a method for constructing a road surface pothole semantic segmentation model based on an improved PSPNet, aiming to improve the semantic segmentation accuracy of road surface potholes. Summary of the Invention
[0006] The main purpose of this application is to provide a method for constructing a road surface pothole semantic segmentation model based on an improved PSPNet, aiming to solve the problem of how to improve the semantic segmentation accuracy of road surface potholes.
[0007] To achieve the above objectives, this application provides a method for constructing a road surface pothole semantic segmentation model based on an improved PSPNet, applicable to a semantic segmentation model built on the PSPNet architecture. The method for constructing the road surface pothole semantic segmentation model based on the improved PSPNet includes the following steps:
[0008] S10: Construct shallow feature branches in the decoder of PSPNet to enhance the model's multi-scale feature representation capability.
[0009] S20, set an auxiliary segmentation head at the mask output of the shallow feature branch and a main segmentation head at the output of the semantic segmentation model, so as to supervise the shallow feature extraction effect of the model and supervise the final output mask segmentation result of the model respectively.
[0010] S30: Train the semantic segmentation model with the segmentation head set on the road pothole dataset to complete the construction of the improved road pothole semantic segmentation model.
[0011] Optionally, in S10, the constructed shallow feature branch satisfies the following steps:
[0012] S11, extract the outputs of Layer 1, Layer 2 and Layer 3 of the backbone network to obtain three shallow feature maps SF1, SF2 and SF3, which are used as three inputs to the shallow feature branches respectively;
[0013] S12 performs pixel-wise feature fusion of SF3 and SF2, extracts features using a 3×3 depthwise convolution, upsamples using bilinear interpolation, and then performs pixel-wise feature fusion with SF1. After extracting features and upsampling again, a 1×1 convolution is used to reduce the number of output channels, resulting in a shallow semantic segmentation mask with dimensions equal to the input image.
[0014] Optionally, after S10, the method further includes:
[0015] S14, the shallow semantic segmentation mask result is added pixel by pixel to the semantic segmentation mask result output by the deep feature branch to enhance the multi-scale feature expression capability of the model.
[0016] Optionally, the auxiliary segmentation head is supervised using a cross-entropy loss function.
[0017] Optionally, the main segmentation head is supervised using the cross-entropy loss function and the Dice loss function.
[0018] Optionally, in S30, the hyperparameters of the training process include: 40 training iterations, 512×512 input image size, 4 multithreads, 2 or 4 batch size, 11 random seed value, 8 model downsampling coefficients, FP32 training floating-point precision, SGD optimizer type, 1e-2 initial learning rate, and PyTorch model training framework.
[0019] In addition, to achieve the above objectives, this application also provides a road surface pothole semantic segmentation model obtained by the construction method of the road surface pothole semantic segmentation model based on the improved PSPNet as described in any of the preceding claims.
[0020] This application has at least the following beneficial effects:
[0021] An improved version of PS-PSPNet was used to train and validate on a public road pothole dataset. A shallow feature branch was constructed in the decoder and fused with the original decoder output containing PPM, optimizing the model's segmentation performance. Simultaneously, multi-segmentation head training supervision was applied to the model's final output mask and the shallow feature branch mask, and a new composite loss function was proposed to train the semantic segmentation model, effectively improving its segmentation accuracy. Experiments show that PS-PSPNet has a significant advantage in semantic pothole segmentation accuracy compared to other mainstream models, providing more refined and accurate pothole segmentation results, and offering richer and more effective data support for future road maintenance and repair. Attached Figure Description
[0022] Figure 1 This is a flowchart illustrating the construction method of the road surface pothole semantic segmentation model based on the improved PSPNet involved in the embodiments of this application;
[0023] Figure 2 This is a comparison diagram of the traditional PSPNet and PS-PSPNet network architectures involved in the embodiments of this application;
[0024] Figure 3 This is a comparison chart of the segmentation accuracy of PS-PSPNet and other semantic segmentation models involved in the embodiments of this application on the validation set of the public dataset UDTIRI.
[0025] Figure 4 This is a comparison of the pit and slot semantic segmentation results of PS-PSPNet and PSPNet involved in the embodiments of this application.
[0026] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0027] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.
[0028] First Embodiment
[0029] Reference Figure 1 This embodiment provides a method for constructing a road surface pothole semantic segmentation model based on an improved PSPNet, which is applied to a semantic segmentation model built on the PSPNet architecture. The method for constructing the road surface pothole semantic segmentation model based on the improved PSPNet includes the following steps:
[0030] S10: Construct shallow feature branches in the decoder of PSPNet to enhance the model's multi-scale feature representation capability.
[0031] In this embodiment, the original PSPNet decoder containing PPM is completely preserved and named the deep feature branch, and a new shallow feature branch is proposed and constructed in the decoder.
[0032] Further and optionally, the constructed shallow feature branch satisfies the following steps:
[0033] S11, extract the outputs of Layer 1, Layer 2 and Layer 3 of the backbone network to obtain three shallow feature maps SF1, SF2 and SF3, which are used as three inputs to the shallow feature branches respectively;
[0034] In some alternative implementations, SF1 has a size of 128×128×64, SF2 has a size of 64×64×128, and SF3 has a size of 32×32×256. The three are located in the network from shallow to deep.
[0035] The shallow feature branch employs a bottom-up feature fusion strategy, progressively upsampling the deeper feature maps and fusing them with the shallower feature maps, i.e.:
[0036] S12 performs pixel-wise feature fusion of SF3 and SF2, extracts features using a 3×3 depthwise convolution, upsamples using bilinear interpolation, and then performs pixel-wise feature fusion with SF1. After extracting features and upsampling again, a 1×1 convolution is used to reduce the number of output channels, resulting in a shallow semantic segmentation mask with dimensions equal to the input image.
[0037] Furthermore, in some optional implementations, after obtaining the shallow semantic segmentation mask result image, the semantic segmentation mask result output by the shallow feature branch and the semantic segmentation mask result output by the deep feature branch are added pixel by pixel to enhance the multi-scale feature expression capability of the model, which helps to reduce the omission and missegmentation in the model segmentation result and improve the segmentation effect.
[0038] S20, set an auxiliary segmentation head at the mask output of the shallow feature branch and a main segmentation head at the output of the semantic segmentation model, so as to supervise the shallow feature extraction effect of the model and supervise the final output mask segmentation result of the model respectively.
[0039] In this embodiment, segmentation heads are added to the shallow feature branches and the model output, respectively, to more comprehensively supervise and improve model performance during the training phase.
[0040] Furthermore, and optionally, the Aux Head is supervised using the cross-entropy loss function CEloss. This is used to supervise the shallow feature extraction performance of the model during training. Notably, the supervision computation of the Aux Head occurs before merging with the deeper strip branches, effectively supervising the shallow feature branches independently.
[0041] The model's final output uses a primary head with a composite loss function: cross-entropy loss (CEloss) and Dice loss (Diceloss) to better supervise the final output mask segmentation result. CEloss focuses more on classification guidance, while Diceloss focuses more on region overlap guidance. Since the feature map supervised by the primary head is the result of fusing shallow and deep feature branches, it is similar to simultaneously supervising the output mask of both shallow and deep branches.
[0042] In this embodiment, the model's loss function is the sum of the loss values of the Aux Head and the Primary Head, which together constitute a complex training supervision method that combines dual segmentation heads and a composite loss function. Thus, the improvement of PSPNet is completed, resulting in the PS-PSPNet architecture proposed in this embodiment.
[0043] It should be noted that the traditional PSPNet and PS-PSPNet network architectures are similar to those used in other networks. Figure 2As shown, the feature extraction backbone networks in the encoders of both models are essentially the same. The biggest difference lies in the decoders: the PSPNet decoder only has three parts: the PPM module, upsampling, and classification convolution; while the PS-PSPNet decoder introduces the concepts and structures of shallow and deep feature branches, and uses main and auxiliary segmentation heads and a composite loss function for comprehensive training supervision. Experimental predictions show that the PS-PSPNet network structure is more suitable for the semantic segmentation task of road potholes, producing more complete segmentation results with significantly reduced missed and incorrect segmentation. Compared to the baseline PSPNet model, PS-PSPNet demonstrates significantly better accuracy and visualization.
[0044] S30: Train the semantic segmentation model with the segmentation head set on the road pothole dataset to complete the construction of the improved road pothole semantic segmentation model.
[0045] In this implementation, the semantic segmentation model with the segmentation head set is finally trained on the road pothole dataset to complete the construction of the improved road pothole semantic segmentation model.
[0046] Further and optionally, the hyperparameters of the training process include: 40 training iterations, 512×512 input image size, 4 multithreads, 2 or 4 batch size, 11 random seed value, 8 model downsampling coefficients, FP32 training floating-point precision, SGD optimizer type, 1e-2 initial learning rate, and PyTorch model training framework.
[0047] In the technical solution provided in this embodiment, a shallow feature branch is constructed in the decoder and its features are fused with the original decoder output containing PPM, thereby optimizing the segmentation effect of the model. At the same time, the final output mask of the model and the shallow feature branch mask are trained and supervised by multiple segmentation heads, and a new composite loss function is proposed to train the semantic segmentation model, which effectively improves the segmentation accuracy of the model.
[0048] Second Embodiment
[0049] Furthermore, as an implementation scheme, this application embodiment also relates to a road surface pothole semantic segmentation model obtained by using the construction method of the road surface pothole semantic segmentation model based on the improved PSPNet as described above.
[0050] The following section verifies the performance of the road surface pothole semantic segmentation model PS-PSPNet involved in this embodiment.
[0051] In this embodiment, the selected dataset is the UDTIRI road surface pothole semantic segmentation dataset, which is divided into training and validation sets in a 6:1 ratio. Specifically, there are 600 training images and 100 validation images, with two categories of labels: potholes in the foreground and other objects in the background.
[0052] In this embodiment, the evaluation metrics used are mainly mIoU and mPA / mRecall, and their formulas are expressed as follows:
[0053] (1)
[0054] In the formula, MIoU is the average interaction ratio, k is the number of foreground categories, Pii is the correctly predicted pixel, and Pij and Pji are both incorrectly predicted pixels.
[0055] (2)
[0056] In the formula, MPA is the average pixel accuracy, k is the number of foreground categories, Pii is the correctly predicted pixel, and Pij is the incorrectly predicted pixel.
[0057] The ablation experiments of the PS-PSPNet architecture proposed in this application are shown in Table 1:
[0058] Table 1. Comparison of ablation experimental results of PS-PSPNet
[0059]
[0060] As shown in Table 1, when PSPNet is not improved in any way, its segmentation accuracy is 88.0%; when shallow feature branches are constructed, the accuracy of the model is improved to 89.02%; on this basis, when an auxiliary segmentation head is added, the accuracy of the model reaches 89.42%; finally, when the composite loss function is adopted, the PS-PSPNet of this application can be obtained by training, and the accuracy of the model reaches the highest of 90.01%.
[0061] Furthermore, the PS-PSPNet model was tested on the UDTIRI pit validation set, and its segmentation accuracy was compared in detail with other semantic segmentation models. This invention selected some mainstream CNN architecture models such as U-net, Deeplabv3+, PSPNet, and Bisenet v1, as well as other semantic segmentation models based on the Transformer architecture, such as A2FPN and UnetFormer, to more comprehensively compare the segmentation performance of PS-PSPNet. The comparison of the segmentation accuracy of PS-PSPNet and other semantic segmentation models on the UDTIRI public dataset validation set is shown in Table 2. Figure 3 As shown:
[0062] Table 2. Comparison of segmentation accuracy of different semantic segmentation models
[0063]
[0064] As can be seen, PS-PSPNet has the highest mIoU and mRecall / mPA on the validation set of the UDTIRI public dataset, which are significantly higher than other mainstream CNN or Transformer models by 2.01%~22.94% and 2.59%~21.64%, respectively. In summary, PS-PSPNet has a clear advantage in road surface distress pothole segmentation accuracy compared with several excellent models.
[0065] Furthermore, to more intuitively compare the pothole semantic segmentation results of PSPNet and PS-PSPNet, this invention also compares the road surface pothole prediction results of PSPNet and PS-PSPNet. The pothole semantic segmentation result images of PS-PSPNet and PSPNet are compared below. Figure 4 As shown.
[0066] from Figure 4 As can be seen from the results, PS-PSPNet significantly outperforms PSPNet on the four images shown. Specifically: In the first image, PSPNet missed segmenting small potholes in the upper half, while PS-PSPNet segmented these small targets quite well; in the second image, PS-PSPNet's pothole segmentation is significantly more complete than PSPNet's; in the third image, PSPNet exhibits severe omissions, failing to detect the pothole entirely, while PS-PSPNet, despite some missegmentation issues, still achieves significantly better results; in the fourth image, PSPNet misses segmenting the right half of the pothole, while PS-PSPNet almost completely segments the pothole, although the boundary details are still not ideal. In conclusion, PS-PSPNet's segmentation accuracy and performance are significantly superior to the benchmark PSPNet, making it more suitable for automatic pothole detection and further maintenance work.
[0067] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0068] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for constructing a road surface pothole semantic segmentation model based on an improved PSPNet, characterized in that, The method for constructing a road surface pothole semantic segmentation model based on the improved PSPNet architecture, applicable to semantic segmentation models built on the PSPNet architecture, includes the following steps: S10: Construct shallow feature branches in the decoder of PSPNet to enhance the model's multi-scale feature representation capability. S20, set an auxiliary segmentation head at the mask output of the shallow feature branch and a main segmentation head at the output of the semantic segmentation model, so as to supervise the shallow feature extraction effect of the model and supervise the final output mask segmentation result of the model respectively. S30: Train the semantic segmentation model with the segmentation head set on the road pothole dataset to complete the construction of the improved road pothole semantic segmentation model.
2. The method for constructing a road surface pothole semantic segmentation model based on the improved PSPNet as described in claim 1, characterized in that, In step S10, the constructed shallow feature branch satisfies the following steps: S11, extract the outputs of Layer 1, Layer 2 and Layer 3 of the backbone network to obtain three shallow feature maps SF1, SF2 and SF3, which are used as three inputs to the shallow feature branches respectively; S12 performs pixel-wise feature fusion of SF3 and SF2, extracts features using a 3×3 depthwise convolution, upsamples using bilinear interpolation, and then performs pixel-wise feature fusion with SF1. After extracting features and upsampling again, a 1×1 convolution is used to reduce the number of output channels, resulting in a shallow semantic segmentation mask with dimensions equal to the input image.
3. The method for constructing a road surface pothole semantic segmentation model based on the improved PSPNet as described in claim 2, characterized in that, Following S10, the following is also included: S14, the shallow semantic segmentation mask result is added pixel by pixel to the semantic segmentation mask result output by the deep feature branch to enhance the multi-scale feature expression capability of the model.
4. The method for constructing a road surface pothole semantic segmentation model based on the improved PSPNet as described in claim 1, characterized in that, The auxiliary segmentation head is supervised using the cross-entropy loss function.
5. The method for constructing a road surface pothole semantic segmentation model based on the improved PSPNet as described in claim 1, characterized in that, The main segmentation head is supervised using the cross-entropy loss function and the Dice loss function.
6. The method for constructing a road surface pothole semantic segmentation model based on the improved PSPNet as described in claim 1, characterized in that, In S30, the hyperparameters of the training process include: 40 training iterations, 512×512 input image size, 4 multithreads, 2 or 4 batch size, 11 random seed value, 8 model downsampling coefficients, FP32 training floating-point precision, sgd optimizer type, 1e-2 initial learning rate, and PyTorch model training framework.
7. A road surface pothole semantic segmentation model obtained by constructing a road surface pothole semantic segmentation model based on an improved PSPNet as described in any one of claims 1 to 6.