A method for identifying urban road flooding based on Swing Transformer and coordinate attention enhancement

Abstract

This invention relates to the field of urban road waterlogging recognition technology, specifically a method for urban road waterlogging recognition based on Swin Transformer and coordinate attention enhancement. The method includes: preprocessing road waterlogging image data; improving the YOLOv8-seg model; training and optimizing the improved YOLOv8-seg model; and using the trained improved YOLOv8-seg model to perform waterlogging recognition on unknown road images, outputting waterlogged areas and quantified information. This invention achieves significantly improved segmentation accuracy, completely solving the problems of insufficient fine-grained segmentation and inaccurate edge localization; significantly enhanced scene adaptability, greatly improving anti-interference capabilities for complex scenes such as waterlogging reflection, different road surface materials, day and night lighting changes, and blurred edges; a better balance between real-time performance and efficiency, meeting the application needs of real-time urban road waterlogging detection; strong scalability, supporting training with thousands of samples; and higher quantified output accuracy, providing more accurate data support for road waterlogging risk assessment.

Description

Technical Field

[0001] This invention relates to the field of urban road waterlogging recognition technology, specifically to an urban road waterlogging recognition method based on Swing Transformer and coordinate attention enhancement. Background Technology

[0002] In existing urban road flooding identification technologies, deep learning-based segmentation schemes have become mainstream, with the YOLOv8-seg model being widely used due to its balance between speed and accuracy. This model's backbone network employs a C2f module combined with the CBAM attention mechanism, extracting image features through local convolution operations to achieve segmentation and localization of flooded areas.

[0003] The core structure and principle of the existing technology are as follows: The YOLOv8-seg model's backbone obtains feature maps at different scales through multi-stage convolutional downsampling; the C2f module enhances feature propagation using residual connections; the CBAM attention mechanism optimizes feature weights through channel attention and spatial attention; and finally, the segmentation head outputs the mask and bounding box of the water accumulation region. The aforementioned related technologies are disclosed in patent number CN202410235678.9, entitled "Automatic Management Method, Device, Computer Equipment, and Storage Medium for Skeleton Screens," and their core relies on local convolutional operations to capture feature associations.

[0004] However, current technologies have the following shortcomings in use: 1. Insufficient long-distance feature dependency capture: The CBAM attention mechanism only optimizes the weights of local features, failing to effectively capture long-distance edges and texture relationships in waterlogged areas (such as the edge continuity of large-area waterlogging and the global correlation of scattered waterlogged areas), resulting in insufficient fine-grained segmentation accuracy and broken or mis-segmented waterlogged edges. 2. Limited segmentation robustness: For complex scenarios (such as waterlogged reflections, different road surface materials, and changes in day and night lighting), the local receptive field of traditional convolution limits the global integration capability of features, leading to poor adaptability of the model to scene changes and high segmentation errors (the relative error of current technologies is generally above 10%). 3. Imbalanced feature extraction efficiency: To improve the global feature capture capability, simply increasing the number of convolutional layers leads to an exponential increase in computation, failing to meet real-time requirements and making it difficult to meet the practical application scenarios of rapid detection of waterlogged areas on urban roads. 4. Lack of spatial location dependency capture: Although the CBAM attention mechanism can optimize channel and local spatial feature weights, it lacks the encoding of coordinate information, making it difficult to accurately locate segmentation edges and resulting in insufficient segmentation accuracy in scenarios such as blurred waterlogged edges. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a method for identifying urban road waterlogging based on Swing Transformer and coordinate attention enhancement.

[0006] The technical problem to be solved by this invention is achieved by the following technical solution: A method for identifying urban road flooding based on Swing Transformer and coordinate attention enhancement includes the following steps: Step 1: Preprocess the road flooding image data; Step 2: Improve the design of the YOLOv8-seg model: Some C2f modules in the backbone network layer of the YOLOv8-seg model are replaced with Swing Transformer window attention modules, and the CBAM modules in the Neck layer of the YOLOv8-seg model are replaced with coordinate attention modules. The Swing Transformer window attention module and the coordinate attention module work together to enhance performance. Step 3: Optimize the improved YOLOv8-seg model through training; Step 4: Use the trained and improved YOLOv8-seg model to perform water accumulation inference recognition and quantization output on the unknown road image.

[0007] As a further improvement of the present invention, the preprocessing in step 1 includes: Data cleaning: Removing missing annotations and duplicate image data; Format conversion: Convert the JSON annotation format of LabelMe to the TXT format of YOLO-seg; Dataset partitioning: Randomly partition the dataset into training, validation, and test sets according to a set ratio.

[0008] As a further improvement of the present invention, in step 2, the second and third C2f modules in the C3 stage and the first and second C2f modules in the C4 stage of the YOLOv8-seg model backbone network layer are replaced with Swing Transformer window attention modules, and feature channel alignment is achieved through a 1×1 convolutional layer.

[0009] As a further improvement of the present invention, the Swin Transformer window attention module includes window partitioning, in-window self-attention calculation, and window shifting interaction.

[0010] As a further improvement of the present invention, in step 2, the CBAM module in the Neck layer of the YOLOv8-seg model is removed, and a coordinate attention module is inserted into the 128-channel and 256-channel feature layers after Concat feature fusion.

[0011] As a further improvement of the present invention, the coordinate attention module includes channel pooling, coordinate encoding, and attention calculation.

[0012] As a further improvement of the present invention, attention calculation includes channel attention calculation and spatial attention calculation.

[0013] As a further improvement to the present invention, step 3, model training optimization, includes: Parameter configuration: Set the input image size to 512×512~800×800, batch size to 2~8, number of training epochs to 200~500, optimizer selection, and initial learning rate to 0.001~0.005; Regularization optimization: Insert Dropout layers after the MLP layer in the window attention module and the convolutional layer in the coordinate attention module of the Swing Transformer, and set weight decay coefficients to suppress redundant parameter growth. Mixed precision training: Enable AMP automatic mixed precision training and half-precision inference; Model preservation: During each round of training, the model weights with the best segmentation accuracy are retained through evaluation on the validation set.

[0014] As a further improvement to the present invention, step 4 is specifically as follows: Input preparation: Obtain the road image to be identified, and perform resize and normalization preprocessing; Model inference: The processed road image is input into the improved YOLOv8-seg model. The YOLOv8-seg model extracts global long-range features, optimizes edge details, and outputs the water accumulation area mask and bounding box. Quantization calculation: Calculate the water accumulation area, minimum / maximum width, and location information based on the mask; Output results: water accumulation area segmentation map, confidence level, and quantification data.

[0015] The beneficial effects of this invention are: Significantly improved segmentation accuracy: The Swin Transformer window attention module effectively captures long-distance edges and texture dependencies in the water accumulation area, while the coordinate attention module accurately locates the details of the segmentation edges. The synergistic effect of the two improves the integrity of the water accumulation edge segmentation, completely solving the problems of insufficient fine-grained segmentation and inaccurate edge positioning.

[0016] Significantly enhanced scene adaptability: Through global feature integration and edge detail optimization, the improved YOLOv8-seg model has greatly enhanced its anti-interference ability in complex scenes such as water reflection, different road surface materials, day and night lighting changes, and edge blurring.

[0017] A better balance between real-time performance and efficiency: The Swin Transformer window attention module reduces complexity through partitioned computation, and the coordinate attention module has a lightweight structure, meeting the application requirements for real-time detection of urban road water accumulation.

[0018] Highly scalable: The Swin Transformer window attention module fusion method is compatible with other YOLOv8 series models (s / m / l / x). It can be adapted to different computing power devices by adjusting the window size (e.g., 16×16), the number of attention heads, and the coordinate attention module pooling method. It also supports expanded training with thousands of samples.

[0019] Higher accuracy in quantitative output: It can directly output quantitative information such as water accumulation area and width, providing more accurate data support for road water accumulation risk assessment and making it more practical. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the present invention; Figure 2 A schematic diagram illustrating the fusion of attention modules to improve the YOLOv8-seg model. Detailed Implementation

[0021] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0022] like Figure 1 As shown, a method for identifying urban road flooding based on Swin Transformer and coordinate attention enhancement includes the following steps: Step 1: Preprocess the road flooding image data to provide qualified data. The specific steps are as follows: a. Data cleaning: Remove missing or duplicate image data from the road flooding image dataset.

[0023] b. Format conversion: Call the convert_json_to_yoloseg function to convert the LabelMe JSON annotation format to the YOLO-seg TXT format, completing coordinate normalization and rectangle annotation to polygon processing.

[0024] c. Dataset partitioning: The training set, validation set, and test set are randomly partitioned proportionally. In this embodiment, 4123 training images, 1178 validation images, and 589 test images are randomly partitioned according to a 7:2:1 ratio to ensure consistent distribution across different road types, water accumulation levels, and environmental scenarios.

[0025] Step 2: Improve the design of the YOLOv8-seg model, as shown in the schematic diagram below. Figure 2 As shown. Based on the Ultralytics YOLOv8-seg model framework, model improvement is achieved by inserting a dual attention module. The specific operation is as follows: 1. Replace some C2f modules in the backbone network layers of the YOLOv8-seg model with Swing Transformer window attention modules. Specifically, replace the 2nd and 3rd C2f modules in stage C3 and the 1st and 2nd C2f modules in stage C4 of the YOLOv8-seg model backbone network layers with Swing Transformer window attention modules, and use 1×1 convolutional layers to align feature channels to ensure compatibility with subsequent network layers.

[0026] Furthermore, the Swin Transformer window attention module includes: Window partitioning: In this embodiment, the feature map is divided into non-overlapping windows of size 8×8 pixels, using the following formula: x_windows=view(B,H / / Ws,W / / Ws,Ws,C); In this embodiment, Ws=8, B is the batch size, H and W are the feature map sizes, and C is the number of channels.

[0027] Intra-window self-attention calculation: Introduce a relative position bias table (dimension (2Ws-1)×(2Ws-1)×num_heads), and calculate intra-window self-attention by attn=softmax(QK^T / √dk+relative_bias).

[0028] Window shifting interaction: In this embodiment, a cyclic shifting operation with a shift step size of 4 pixels is used to achieve cross-window feature association.

[0029] 2. Replace the CBAM module in the Neck layer of the YOLOv8-seg model with a coordinate attention module, and use the SwingTransformer window attention module in conjunction with the coordinate attention module for synergistic enhancement. Specifically, remove the CBAM module from the Neck layer of the YOLOv8-seg model, and insert coordinate attention modules into the 128-channel and 256-channel feature layers after Concat feature fusion.

[0030] Furthermore, the coordinate attention module includes: Channel pooling: Perform global average pooling in the horizontal and vertical directions on the feature maps to obtain H×1×C and 1×W×C feature maps.

[0031] Coordinate encoding: Horizontal / vertical feature maps are concatenated, and after 1×1 convolution to compress the number of channels and GELU activation, horizontal / vertical attention weights are generated.

[0032] Attention computation: including channel attention computation and spatial attention computation.

[0033] Channel attention calculation: Attention weighting is applied to the original input feature map along the channel dimension to enhance key channel features.

[0034] Spatial attention calculation: Combining horizontal and vertical attention weights, spatial attention weighting is applied to the channel-weighted feature map to accurately locate edge positions.

[0035] The above improvements enable dual attention collaboration: capturing global long-distance features through the Swin Transformer window attention module and locating local edges through the coordinate attention module.

[0036] Step 3: Optimize the improved YOLOv8-seg model through training.

[0037] Based on the preprocessed dataset, the improved model is trained as follows: a. Parameter configuration: Set the input image size to 512×512~800×800, batch size to 2~8, number of training rounds to 200~500, optimizer selection, and initial learning rate to 0.001~0.005.

[0038] In this embodiment, the input image size is 640×640; the batch size is 4; the number of training rounds is 300; the optimizer is AdamW, the initial learning rate is 0.002; and cosine annealing learning rate scheduling is enabled.

[0039] b. Regularization optimization: Insert a Dropout layer (probability 0.15) after the MLP layer in the window attention module and the convolutional layer in the coordinate attention module of the Swing Transformer, and set a weight decay coefficient, which is 0.0005 in this embodiment; suppress redundant parameter growth.

[0040] c. Mixed Precision Training: Enable AMP automatic mixed precision training and half-precision inference to reduce video memory usage.

[0041] d. Model preservation: During each round of training, the model weights with the best segmentation accuracy are retained through evaluation on the validation set.

[0042] Step 4: Use the trained, improved YOLOv8-seg model to perform water accumulation inference, identification, and quantization output on the unknown road image. The specific steps are as follows: a. Input preparation: Acquire the road image to be identified, and perform resize and normalization preprocessing. Road images support JPG / PNG format, video streams, and real-time camera images.

[0043] b. Model inference: Input the processed road image into the improved YOLOv8-seg model. The YOLOv8-seg model extracts global long-range features, optimizes edge details, and outputs the water accumulation area mask and bounding box.

[0044] c. Quantization calculation: Calculate the water accumulation area, minimum / maximum width, and location information based on the mask.

[0045] Specifically, the formula for calculating the pixel area and proportion of the water accumulation area is as follows: The function of calculating the water accumulation area (pixel level) based on the mask is implemented in the cal_seg_percent function, which is the core of the entire quantization calculation.

[0046] Single-region pixel area calculation: The area calculation of a closed polygon contour is based on Green's theorem and is implemented using OpenCV's native cv2.contourArea method. The calculation formula is as follows: ; In the formula, This represents the pixel area of ​​a single water accumulation region, and its value is a non-negative integer. This is represented as a native OpenCV function that calculates the pixel area of ​​a closed polygon contour based on Green's theorem. C represents the set of coordinate points of a single closed contour of a water accumulation region extracted from the mask, expressed as follows: ; In the formula, Represented as the pixel area of ​​a single water accumulation area; This is represented by the fixed coefficients used in Green's formula to calculate the area of ​​a closed polygon; This is represented as a summation operator, which calculates the summation over all vertices of the contour sequentially. Represented as the pixel coordinates of the i-th vertex of the contour (where i is the horizontal direction and y is the vertical direction); The boundary condition is represented as a closed polygon, where the next vertex after the last vertex is the first vertex.

[0047] Multi-region accumulation: If multiple water accumulation contours exist, the total pixel area is the sum of the areas of each region: ; In the formula, This is represented as the total pixel area of ​​all water-filled areas in the image; This is represented as a summation operator, which accumulates the pixel area of ​​all water accumulation regions. This is represented by the pixel area of ​​the i-th water accumulation region; It is represented as the set of coordinate points of the closed contour of the i-th water accumulation area.

[0048] The formula for calculating the percentage of water accumulation area is as follows: ; In the formula, This is expressed as the percentage of the water accumulation area relative to the total area of ​​the input image. Represented as the pixel area of ​​all water accumulation areas; This represents the width of the original input image; This is represented as the height of the original input image; Represented as the total number of pixels in the input image; Convert to a 100% coefficient.

[0049] The formula for converting pixel area to actual physical area is as follows: Core conversion formula: ; In the formula, Represented as the actual physical area of ​​the waterlogged region; The pixel area of ​​the water accumulation region is calculated from the mask. It is represented as a pixel scale, that is, the actual physical length corresponding to 1 pixel; Represented as the square operator, it performs the conversion from length dimension to area dimension. The actual area of ​​a single pixel is: .

[0050] Calculation of minimum / maximum width and location of water accumulation area: Core calculation principle (based on minimum bounding rectangle) 1. Calculation of principal axes and minimum bounding rectangle: Fit the minimum bounding rectangle (`cv2.minAreaRect`) to the contour of the waterlogged area to obtain the center coordinates, width, height, and rotation angle of the rectangle. ; In the formula, The set of characteristic parameters is represented as the minimum bounding rectangle; This is represented as a native OpenCV function used to fit the minimum bounding rectangle to a closed contour. It is represented as a set of coordinate points of a closed contour of a single water accumulation region extracted from the mask; Represented as the center pixel coordinates of the smallest bounding rectangle. The first element in; This is represented by the width and height of the minimum bounding rectangle. The second element in; The rotation angle, represented by the smallest bounding rectangle, characterizes the angle between the principal axis of the profile and the horizontal axis. The third element in.

[0051] 2. Width extremum extraction: Minimum width: the length of the shorter side of the circumscribed rectangle; Maximum width: The length of the longer side of the bounding rectangle; The formula is as follows: ; In the formula, This represents the minimum width of the water accumulation area; This represents the maximum width of the waterlogged area. This is represented as the minimum value operator, which takes the smaller value between the width and height of the enclosing rectangle; This is represented as the maximum value operator, which takes the larger value between the width and height of the enclosing rectangle; It represents the width of the minimum bounding rectangle; It represents the height of the smallest bounding rectangle.

[0052] 3. Width-to-position coordinates: The minimum / maximum width is determined by the coordinates of the center point of the corresponding side of the bounding rectangle, as shown in the following formula: ; In the formula, This is represented by the position corresponding to the minimum width; This represents the position corresponding to the maximum width. This function calculates the midpoint coordinates. It takes the coordinates of the two endpoints of a rectangle's edge as input and outputs the coordinates of the edge's center. ShortEdge represents the short side of the minimum bounding rectangle.

[0053] d. Output Results: Output water accumulation area segmentation map, confidence threshold, and quantified data. In this embodiment, the confidence threshold is 0.2 to 0.3, specifically 0.25, the single-frame inference time is ≤0.15s, and the segmentation relative error is ≤5%.

[0054] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely prisms of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying urban road flooding based on Swing Transformer and coordinate attention enhancement, characterized in that: Includes the following steps: Step 1: Preprocess the road flooding image data; Step 2: Improve the design of the YOLOv8-seg model: Some C2f modules in the backbone network layer of the YOLOv8-seg model are replaced with Swing Transformer window attention modules, and the CBAM modules in the Neck layer of the YOLOv8-seg model are replaced with coordinate attention modules. The Swing Transformer window attention module and the coordinate attention module work together to enhance performance. Step 3: Optimize the improved YOLOv8-seg model through training; Step 4: Use the trained and improved YOLOv8-seg model to perform water accumulation inference recognition and quantization output on the unknown road image.

2. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 1, characterized in that: The preprocessing in step 1 includes: Data cleaning: Removing missing annotations and duplicate image data; Format conversion: Convert the JSON annotation format of LabelMe to the TXT format of YOLO-seg; Dataset partitioning: Randomly partition the dataset into training, validation, and test sets according to a set ratio.

3. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 1, characterized in that: In step 2, the second and third C2f modules in the C3 stage and the first and second C2f modules in the C4 stage of the YOLOv8-seg model backbone network are replaced with Swing Transformer window attention modules, and feature channel alignment is achieved through 1×1 convolutional layers.

4. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 3, characterized in that: The Swin Transformer window attention module includes window partitioning, in-window self-attention calculation, and window shifting interaction.

5. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 1, characterized in that: In step 2, the CBAM module in the Neck layer of the YOLOv8-seg model is removed, and a coordinate attention module is inserted into the 128-channel and 256-channel feature layers after Concat feature fusion.

6. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 5, characterized in that: The coordinate attention module includes channel pooling, coordinate encoding, and attention calculation.

7. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 6, characterized in that: Attention computation includes channel attention computation and spatial attention computation.

8. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 1, characterized in that: Step 3, model training optimization, includes: Parameter configuration: Set the input image size to 512×512~800×800, batch size to 2~8, number of training epochs to 200~500, optimizer selection, and initial learning rate to 0.001~0.005; Regularization optimization: Insert Dropout layers after the MLP layer in the window attention module and the convolutional layer in the coordinate attention module of the Swing Transformer, and set weight decay coefficients to suppress redundant parameter growth. Mixed precision training: Enable AMP automatic mixed precision training and half-precision inference; Model preservation: During each round of training, the model weights with the best segmentation accuracy are retained through evaluation on the validation set.

9. The urban road waterlogging identification method based on Swing Transformer and coordinate attention enhancement according to claim 1, characterized in that: Step 4 is as follows: Input preparation: Obtain the road image to be identified, and perform resize and normalization preprocessing; Model inference: The processed road image is input into the improved YOLOv8-seg model. The YOLOv8-seg model extracts global long-range features, optimizes edge details, and outputs the water accumulation area mask and bounding box. Quantization calculation: Calculate the water accumulation area, minimum / maximum width, and location information based on the mask; Output results: water accumulation area segmentation map, confidence level, and quantification data.

Images