Road disease detection method, electronic equipment and storage medium

Abstract

The invention discloses a road disease detection method, an electronic device and a storage medium, a target road picture and a road size are acquired, the road picture is mapped into a road image including a real physical size, the road image is gridded according to road disease positioning precision, and each grid is used for outputting a disease category label; a segmentation model is constructed and trained, the segmentation model is used for identifying the disease of each grid according to the road image, the segmentation model comprises a fusion module, SE convolution and a residual structure are embedded in the fusion module, the SE convolution is used for weighting channel information in the road image, and the residual structure is used for extracting the channel information in the road image; the residual structure is used for reserving the spatial details of the high-resolution features through jump connection. According to the invention, a road disease detection scheme based on grid segmentation is provided, and the image is divided into grids to output a disease result, so that the marking cost can be reduced and efficient statistics can be realized; and the optimized segmentation model is combined with an SE module residual mechanism and a category weighted cost function, so that the small sample disease detection rate is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of pavement damage image processing, and in particular relates to a road damage detection method, electronic equipment and a storage medium.

[0002] Background technology.

[0003] Road defect detection technology, as part of transportation infrastructure maintenance, is evolving from traditional models to intelligent and automated approaches with the development of technologies such as artificial intelligence and computer vision. Road condition assessment encompasses processes such as road image acquisition, defect detection, and calculation of relevant good road rates. Traditional defect detection relies on manual processing of defect areas to obtain relevant data. Driven by technological advancements, several road defect detection solutions based on specific technologies have been implemented, each with its own unique characteristics for acquiring defect information. However, some detection solutions lack accuracy in the labeling of defect areas, impacting subsequent related calculations. Other detection solutions, in actual operation, face challenges such as high resource consumption in the labeling phase and complex subsequent processing, which limits the efficiency of obtaining defect length or area information. Furthermore, some detection models struggle to achieve ideal detection results for certain defects, particularly those with small sample sizes. Summary of the Invention

[0004] The purpose of the present invention is to provide a road disease detection method, electronic equipment and storage medium, which can solve the technical problems existing in the prior art, such as the output results are not correlated with the actual road disease area, the labeling workload is large, and the detection rate of individual road diseases is low.

[0005] To achieve the above objectives, a specific embodiment of the present invention provides a road damage detection method, the method comprising: Obtaining a target road image and road dimensions, mapping the road image into a road image including real physical dimensions, gridding the road image according to the road defect positioning accuracy, and outputting a defect category label for each grid; A segmentation model is constructed and trained to identify defects in each grid based on a road image. The segmentation model includes a fusion module in which a SE convolution and a residual structure are embedded. The SE convolution is used to weight channel information in the road image, and the residual structure is used to preserve spatial details of high-resolution features through skip connections.

[0006] In one or more embodiments of the present invention, the segmentation model includes a feature extraction module, a downsampling module, an upsampling module, a fusion module and a segmentation head module, wherein: The feature pre-extraction module is used to extract features from road images and generate feature maps. The downsampling module is used to reduce the size of the feature map to extract disease features in various dimensions. The upsampling module is used to expand the feature map size to the grid size of the specified output to achieve feature information fusion. The fusion module includes SE convolution and residual structure. The fusion module is used to fuse the disease information extracted from the downsampling modules in various dimensions so that the grid output can learn the multi-dimensional features of the disease. The final segmentation head module is used to learn the comprehensive features of each disease from the dimension of the grid size, further extract the features from the fusion module, and output the disease results in a grid shape.

[0007] In one or more embodiments of the present invention, the downsampling modules are provided with several groups to perform multi-dimensional sampling of the disease features in the feature map according to the preset size of each downsampling module, and the number of the fusion modules corresponds one-to-one to the number of downsampling modules to perform multi-dimensional fusion according to the disease features in the feature map.

[0008] In one or more embodiments of the present invention, the step of weighting the channel information in the feature map by the SE module includes: Perform global average pooling on each channel in the feature map, compressing the feature map into several channel vectors of preset sizes to statistically analyze the distribution intensity of the disease in the global space; Two fully connected layers are used to generate the weights of each channel to enhance the response suppression background of disease-related channels; The channel weights are multiplied by each vector of the original feature map channel by channel to obtain a calibrated feature map, so that the segmentation model focuses on the disease-prominent area.

[0009] In one or more embodiments of the present invention, the step of training the segmentation model includes: Collect a number of road images and divide them into a dataset, a validation set, and a test set according to a preset ratio; The proportion of each disease category in the dataset is consistent with the proportion of real road diseases. During training, the categories are balanced by oversampling diseases with a small proportion. The number of disease categories in the validation set is balanced to guide the segmentation model to evenly train the performance of each disease category; The proportion of road surface defects in the test set is the proportion of defects that may occur on real roads.

[0010] In one or more embodiments of the present invention, the training step of the segmentation model further includes adjusting the segmentation head module, and the adjustment step includes: Adjust the number of road images of the disease categories with low original proportion in the dataset so that the proportion of all disease categories in the dataset is equal; Read the model parameter file during segmentation model training and set it to use only the segmentation head module for secondary segmentation model training.

[0011] In one or more embodiments of the present invention, the cost function used when training the segmentation model is: ; in, L is the original Focal Loss loss function, For the model to predict the category c The probability range is 0~1. is the probability that the model predicts belongs to the foreground, γ The adjustment factor for difficult sample mining is in the range of 0 to 5; The segmentation model training is performed using all parameters, and the segmentation model training is set to be performed for at least 200 rounds.

[0012] In one or more embodiments of the present invention, the cost function used in the secondary training of the segmentation head module is: ; in, is the Focal loss variant function, freq ( c ) is the disease category c The frequency of occurrence of the disease category c The frequency of occurrence is consistent with the proportion of each disease category in the segmentation model training. For category c Weights in the Focal Loss variant cost function.

[0013] In another aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory, wherein the memory stores instructions, which, when executed by the at least one processor, enable the at least one processor to execute a road deterioration detection method.

[0014] In another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the road damage detection method are implemented.

[0015] Beneficial Effects: Compared with the prior art, the road damage detection method of the embodiment of the present invention uses a grid segmentation road damage detection scheme to divide the image into a number of grids according to preset areas based on the actual road surface. The presence of damage is determined based on the grids. The number of grids can be directly correlated to the area value of the damage, facilitating the calculation of the formula for the ratio of good road technical conditions. The present invention divides the image into a grid and labels the diseased area according to the grid, which greatly reduces the labeling cost and realizes efficient labeling and direct area statistics. Furthermore, the full-pixel segmentation model is optimized into a grid-based segmentation model to ensure high accuracy of the diseased area. To address the problem of poor detection results due to a small number of samples of individual road defects, the present invention improves the expressive power of shallow features when fusing features through the SE module and the residual mechanism when constructing a segmentation model, thereby enhancing the model's ability to learn various defects. Furthermore, a category-weighted cost function is used to guide the model to focus on learning defect types with a small number of data samples, thereby improving the detection rate for defects with a small number of samples by enhancing channel attention and residual connections. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments described in the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0017] Figure 1 Flowchart of a road damage detection method according to one embodiment of the present invention; Figure 2 This is a flow chart of a grid-based road defect marking method according to one embodiment of the present invention; Figure 3 A flowchart of using the interactive grid annotation tool in one embodiment of the present invention; Figure 4 A diagram showing a segmentation model structure in one embodiment of the present invention; Figure 5 This is a structural diagram of each module in the segmentation model in one embodiment of the present invention; Figure 6 A schematic diagram of the marking results in one embodiment of the present invention; Figure 7 A schematic diagram of a label format according to an embodiment of the present invention; Figure 8 A statistical graph showing the accuracy of road damage detection in one embodiment of the present invention; Figure 9 This is a schematic diagram of the disease results outputted in one embodiment of the present invention; Figure 10 FIG. 1 is a hardware structure diagram of a computing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0019] As described in the background technology, the existing road disease detection methods have the following problems: the model output results cannot be directly correlated with the actual disease area, the model cannot take into account the edge accuracy of the disease area, the pre-training labeling cost is high, and the detection effect is poor due to the small number of individual road disease samples.

[0020] In response to the above technical issues, such as Figure 1 As shown, the present invention introduces a road disease detection method, which includes the following steps: S1. Obtain a target road image and road dimensions, map the road image into a road image with real physical dimensions, grid the road image according to the road defect positioning accuracy, and output a defect category label for each grid. S2. Build and train a segmentation model, where the segmentation model is used to identify defects in each grid based on the road image. The segmentation model includes a fusion module, in which SE convolution and a residual structure are embedded. The SE convolution is used to weight channel information in the road image, and the residual structure is used to retain spatial details of high-resolution features through skip connections.

[0021] like Figures 2 and 3 As shown in further embodiments, existing road defect data annotation methods have the following shortcomings: First, the annotation granularity does not meet actual needs: traditional pixel-level annotation (such as polygon outlining) overly pursues details, while maintenance units in actual projects are more concerned with 10cm-level defect positioning accuracy; second, annotation efficiency is low: annotating a 2000×3000 pixel image can take more than 10 minutes; third, physical size information is lost: conventional image annotations are not associated with actual physical dimensions, making detection results unable to be directly used for engineering quantification (such as calculating pothole area). Therefore, the present invention uses a grid-based data annotation solution associated with physical dimensions, which can focus on 10cm-level defects and directly correspond the annotation results to the area information of the defects.

[0022] In this example, the input road image is converted to its actual physical dimensions and then divided into H×W grids of 10cm×10cm (e.g., a 2000mm×4000mm road surface is mapped to a 20×40 grid). Each grid outputs a single defect category label, replacing traditional pixel-by-pixel classification. This significantly improves labeling efficiency, reducing the segmentation and labeling time for a single image from over 10 minutes to under 1 minute. In this example, data labeling utilizes an interactive grid labeling tool. First, the defect category is selected, supporting 15 types of road defects and other markers (including cracks, potholes, and patches). Then, using the stroke labeling mode, the mouse automatically activates the category of the grid as it is drawn over the defect area. The software provides visual category feedback, displaying the grid coloring effect in real time (different colors represent different categories). The labeling results are saved to generate a 20×40 matrix label file (TXT format, with values ranging from 0 to 15).

[0023] In a further embodiment, Figures 4 and 5 As shown, the segmentation model includes a feature extraction module, a downsampling module, an upsampling module, a fusion module and a segmentation head module, wherein, The feature pre-extraction module is used to extract features from road images and generate feature maps. The downsampling module is used to reduce the size of the feature map to extract disease features in various dimensions. The upsampling module is used to expand the feature map size to the specified output grid size to achieve feature information fusion. The fusion module includes SE convolution and residual structure. The fusion module is used to fuse the disease information extracted from the downsampling modules in various dimensions so that the grid output can learn the multi-dimensional features of the disease. Finally, the segmentation head module is used to learn the comprehensive features of each disease from the dimension of the grid size, further extract the features from the fusion module, and output the disease results in a grid shape.

[0024] In a further embodiment, the feature pre-extraction module comprises at least two layers: a convolutional layer, a BatchNorm layer, and a ReLU activation function. The convolutional layer extracts features by scanning the image using a convolution kernel; the BatchNorm layer normalizes the data to stabilize the training process; and the ReLU activation function sets values less than 0 to 0, introducing nonlinearity and enhancing the model's expressiveness. This module extracts basic features from the entire image, focusing on shallow and detailed information within the image. It can extract features such as subtle cracks, providing preliminary feature data for subsequent modules.

[0025] The downsampling module consists of a max pooling layer and at least two combinations of a convolutional layer, a BatchNorm layer, and a ReLU activation function. The max pooling layer reduces the image size by a factor of 2 by selecting the maximum value within the pooling window. Subsequent operations such as convolution further extract features. The downsampling module progressively reduces the image size by a factor of 2 each time, extracting disease features from various dimensions, from fine to coarse. Low-level features such as texture and detail are first extracted, followed by a gradual focus on higher-level features such as location, reducing the amount of data while extracting more abstract semantic information.

[0026] The upsampling module consists of a set of 2D deconvolution layers and two sets of "convolution layer + BatchNorm layer + ReLU activation function." The 2D deconvolution layers are used to increase the size of the feature map; subsequent operations such as convolution refine the expanded features. The upsampling module is used to increase the size of the feature map to the specified output grid size, preparing for the fusion of feature information at different levels and ensuring that the feature map size is suitable for subsequent fusion operations.

[0027] The fusion module includes SE convolution and residual structure. SE convolution dynamically evaluates the feature channel weights through Squeeze-Excitation operation; the residual structure directly adds shallow and deep features through skip connection.

[0028] The fusion module is used to integrate the disease information extracted by the downsampling modules in each dimension. SE convolution strengthens the weights of disease-related feature channels and selects key semantic features. The residual structure preserves the spatial details of shallow high-resolution features, avoiding feature dilution during upsampling. The combination of these two forms a dual "semantic screening-detail preservation" pathway, enabling the model to learn the multi-dimensional characteristics of diseases.

[0029] The segmentation head module consists of two convolutional layers (normalized by a BatchNorm layer in between). It learns the comprehensive features of each disease from the dimension of grid size, further extracts and integrates the features from each fusion module, and finally outputs the disease segmentation results in a grid shape, completing the pixel-level classification of the diseases in the image.

[0030] In a further embodiment, the Squeeze-and-Excitation Module (SE) is a channel-wise attention mechanism that dynamically adjusts the weights of feature channels to enhance the model's focus on key features. Its core concept is to enable the network to automatically learn the importance of each channel, thereby improving feature representation. This solution introduces the SE module to weight deep features by channel, suppressing irrelevant channels (such as background texture), and enhancing the response to disease-related features.

[0031] In a further embodiment, the step of weighting the channel information in the feature map by the SE module includes: Perform global average pooling (GAP) on each channel of the feature map, compress the H×W×C feature map into a 1×1×C channel description vector, and count the distribution intensity of defects such as cracks and potholes in the global space; the channel description vector is: ; Generate channel weights through two fully connected layers to enhance the response of disease-related channels (such as crack edge texture) and suppress background (such as asphalt texture);

[0032] in: ( r is the compression ratio, usually 16), is the RELU activation function, is the Sigmoid function.

[0033] Multiply the weights by the original feature map channel by channel to obtain calibrated features, so that the model focuses on areas with significant defects (such as cracks). ; in is the channel weight, is the feature map of the input SE module, It is the feature map after SE module channel weighting.

[0034] The fusion module of SE convolution and residual architecture offers a synergistic advantage. The SE module dynamically evaluates and strengthens the weights of defect-related feature channels (e.g., the weight of crack edge texture channels is increased to 0.8+, while the weight of background texture channels is reduced to 0.2-) through a channel attention mechanism (Squeeze-Excitation operation), addressing the problem of feature channel redundancy in traditional convolution. The residual architecture, on the other hand, preserves the spatial details of shallow, high-resolution features (e.g., the precise location of cracks) through skip connections, avoiding feature dilution during upsampling. The combination of these two forms a dual-pathway mechanism known as "semantic filtering and detail preservation": the SE module selects key features from the channel dimension, while the residual connection supplements the original information from the spatial dimension. This allows the model to improve the detection accuracy of small defects (e.g., fine cracks) without compromising the positioning accuracy of larger objects (e.g., manhole covers).

[0035] In a further embodiment, the step of training the cut model includes: Collect a number of road images and divide them into a dataset, a validation set, and a test set according to a preset ratio; The proportion of each disease category in the dataset is consistent with the proportion of real road diseases. During training, the categories are balanced by oversampling diseases with a small proportion. The number of disease categories in the validation set is balanced to guide the segmentation model to evenly train the performance of each disease category; The proportion of road surface defects in the test set is the proportion of defects that may occur on real roads.

[0036] In a further embodiment, a sample frequency-aware Focal Loss variant cost function is designed to address model bias caused by data imbalance. The original Focal Loss loss function is based on the binary cross-entropy loss function and is used to address the extreme imbalance in the number of positive and negative samples in one-stage object detection. In the task of road defect detection, due to the extreme imbalance in the number of defects, this solution improves the calculation method of class weights in the original Focal Loss loss function to further improve the poor model performance caused by class imbalance.

[0037] The original Focal Loss loss function is as follows: ; in, L is the original Focal Loss loss function, is the probability that the model predicts that the sample belongs to category c, and its value range is 0–1. It is the probability that the model predicts that the sample belongs to the foreground. γ is the adjustment factor for difficult sample mining, which makes the model training focus on difficult classes, and its range is [0, 5].

[0038] The probability of occurrence of each disease category is calculated by the inverse of the square root of the category frequency, and the loss weight is further adjusted. For diseases with low occurrence categories, the inverse of the square root of the frequency is used to increase their weight, so that the model pays more attention to the performance of disease categories with a small number of diseases.

[0039] Improved Focal loss variant function, where freq ( c ) is the category c The frequency of occurrence, For category c The weight coefficient in the Focal Loss variant cost function is inversely proportional to the square root of the class frequency. This ensures that classes with lower frequency receive better loss function weights, guiding the model to focus on the training performance of these classes. This cost function can effectively balance the training effects of various classes.

[0040] ; in is the Focal loss variant function, freq ( c ) is the frequency of occurrence of category c, is the weight of category c in the Focal Loss variant cost function.

[0041] In a further embodiment, the segmentation model is trained using a progressive two-stage training strategy, specifically a training-fine-tuning two-stage framework. The first stage pretrains the backbone network by oversampling to balance data distribution; the second stage freezes shallow layer parameters and fine-tunes the segmentation head using a dynamic weighted loss. This strategy, while ensuring model generalization, specifically optimizes the recognition of low-frequency defects, ultimately achieving an improvement in pothole recall from 37.5% to 80% while maintaining the same precision.

[0042] In a further embodiment, in order to verify the beneficial effects of the technical solution of the present invention, the solution is further described with specific quantitative values: like Figures 5 and 6 As shown, physical-size-aligned grid annotation involves collecting road images and the actual road surface dimensions corresponding to them. To ensure that the grid annotation results contain information about the actual road surface dimensions, the images must be mapped to their actual dimensions before the grid annotation process is performed. The original road image size is 2042 × 4096 pixels. Given that the actual road surface dimensions corresponding to the image are 2m × 4m (2000mm * 4000mm), the image size must be mapped to 2000 × 4000 pixels, ensuring that 100 pixels in the image equals 100mm in reality. After the mapping operation, the image has a resolution of 2000 × 4000 pixels, corresponding to a 2m × 4m actual road surface area.

[0043] like Figure 7 As shown, after loading an image into the grid annotation tool, the software automatically generates a 100×100 pixel grid overlay. The user first selects the defect category and then clicks on the grid area where the defect is located to begin annotation. Once annotation is complete, the software saves the label file. For example, if the grid size is 20×40, the output label file is a 20-row×40-column TXT file. Numerical values represent different defects. For example, 0 represents background, 1 represents horizontal cracks, and 2 represents vertical cracks. Other numerical values correspond to defect categories in a similar way.

[0044] In a further embodiment the training of the segmentation model includes a first stage - full parameter pre-training: Data Preparation: A dataset of 3,000 images was prepared for model training, a validation set of 500 images was used to assist in model optimization, and a test set of 10,000 images was used to comprehensively evaluate model performance. The proportion of each disease category in the 3,000 training images was consistent with the proportion of road defects found in real-world pavement. During training, the categories were balanced by oversampling defects with a smaller proportion (i.e., training with each image multiple times). The 500-image validation set had a balanced number of disease categories to guide balanced training performance for each category. The 10,000-image test set had a proportion of road defects that was consistent with the proportion of defects likely found on real roads.

[0045] Model training: Traditional focal loss is used as the cost function, where the weights of each category are specified as follows: background: cracks: potholes: markings: repairs: joints: broken plates: manhole covers = 0.1:2:1:1:1:1.5:2:0.5. The model is trained using all parameters and is set to train for 200 rounds. After training, the model is saved as a ".pth" file.

[0046] Second stage - class-weighted fine-tuning of segmentation head: Data Preparation: Prepare a dataset of 1,000 images for model training, keeping the validation and test sets unchanged. Increase the number of images of damage categories, such as potholes, that originally had a low proportion, so that the proportion of all damage categories is close to 1:1.

[0047] Model training: Read the model parameter file of the first stage full parameter pre-training and freeze the model except the segmentation head module (such as Figure 4 ) are trained for all layers except the original. The cost function uses the improved Focal Loss, and the proportion of each disease category freq ( c ) is calculated from the original proportions of each category.

[0048] After the segmentation model is trained, it is converted into a .trt file and deployed on an NVIDIA 3090 server, achieving an inference speed of 10 images per second. The model output is a .txt file or table, storing the total area of each defect type and the coordinates of the defect. The area of each defect can be directly used as input to calculate the road's technical condition rating, assisting maintenance units in assessing road condition.

[0049] like Figure 8 As shown in the figure, after fine-tuning the model with category weighting, the performance of the pothole category, which accounts for a relatively low proportion, is significantly improved. For example, the accuracy of the pothole category is increased from 37.5% to 80%, while the performance of other categories remains almost unchanged.

[0050] Figure 9The damage results obtained after segmentation model inference are shown in Table 1. Red indicates linear cracks, blue indicates cement cracks, grass green indicates manhole covers, bright green indicates road markings, dark green indicates exposed cement damage, orange indicates pothole damage, pink indicates broken plates, and purple indicates road surface patching. The results output by the present invention are shown in Table 1. The file stores the actual road surface size of the image, the number of divided grids, and the size of each grid. The area and specific coordinate information of each type of damage are then output. For example, the "number of linear crack grids" indicates that the total number of linear crack grids is 14, representing an area of 14×100, i.e., 14dm 2 In this table, the "grid number" field indicates the grid number of the disease, that is, the area of the disease.

[0051] Test results table. Table 1

[0052] like Figure 10 , a hardware structure diagram of a computing device 30 for a road defect detection method according to an embodiment of this specification is shown. The computing device 30 may include at least one processor 301, a memory 302 (e.g., a non-volatile memory), a storage 303, and a communication interface 304. The at least one processor 301, the storage 302, the storage 303, and the communication interface 304 are connected together via a bus 305. The at least one processor 301 executes at least one computer-readable instruction stored or encoded in the storage 302.

[0053] It should be understood that the computer executable instructions stored in the memory 302, when executed, cause the at least one processor 301 to perform the above combined operations in the various embodiments of this specification. Figure 1-9 Describes the various operations and functions.

[0054] In the embodiments of the present specification, the computing device 30 may include, but is not limited to, a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile computing device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld system, a messaging device, a wearable computing device, a consumer electronic device, and the like.

[0055] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0057] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that is implemented in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0059] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0060] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A road disease detection method, characterized in that: The method comprises: Obtaining a target road image and road dimensions, mapping the road image into a road image including real physical dimensions, gridding the road image according to the road defect positioning accuracy, and outputting a defect category label for each grid; A segmentation model is constructed and trained to identify defects in each grid based on a road image. The segmentation model includes a fusion module in which a SE convolution and a residual structure are embedded. The SE convolution is used to weight channel information in the road image, and the residual structure is used to preserve spatial details of high-resolution features through skip connections.

2. A road damage detection method according to claim 1, characterized in that: The segmentation model includes a feature extraction module, a downsampling module, an upsampling module, a fusion module and a segmentation head module. in, The feature pre-extraction module is used to extract features from road images and generate feature maps. The downsampling module is used to reduce the size of the feature map to extract disease features in various dimensions. The upsampling module is used to expand the feature map size to the grid size of the specified output to achieve feature information fusion. The fusion module includes SE convolution and residual structure. The fusion module is used to fuse the disease information extracted from the downsampling modules in various dimensions so that the grid output can learn the multi-dimensional features of the disease. The final segmentation head module is used to learn the comprehensive features of each disease from the dimension of the grid size, further extract the features from the fusion module, and output the disease results in a grid shape.

3. A road damage detection method according to claim 2, characterized in that: The downsampling modules are provided with several groups to sample the disease features in the feature map in multiple dimensions according to the preset size of each downsampling module. The number of the fusion modules corresponds one-to-one to the number of downsampling modules to perform multi-dimensional fusion according to the disease features in the feature map.

4. A road damage detection method according to claim 2, characterized in that: The step of weighting the channel information in the feature map by the SE module includes: Perform global average pooling on each channel in the feature map, compressing the feature map into several channel vectors of preset sizes to statistically analyze the distribution intensity of the disease in the global space; Two fully connected layers are used to generate the weights of each channel to enhance the response suppression background of disease-related channels; The channel weights are multiplied by each vector of the original feature map channel by channel to obtain a calibrated feature map, so that the segmentation model focuses on the disease-prominent area.

5. A road disease detection method according to claim 2, characterized in that: The training steps of the segmentation model include: Collect a number of road images and divide them into a dataset, a validation set, and a test set according to a preset ratio; The proportion of each disease category in the dataset is consistent with the proportion of real road diseases. During training, the categories are balanced by oversampling diseases with a small proportion. The number of disease categories in the validation set is balanced to guide the segmentation model to evenly train the performance of each disease category; The proportion of road surface defects in the test set is the proportion of defects that may occur on real roads.

6. A road damage detection method according to claim 5, characterized in that: The training step of the segmentation model also includes adjusting the segmentation head module, and the adjustment step includes: Adjust the number of road images of the disease categories with low original proportion in the dataset so that the proportion of all disease categories in the dataset is equal; Read the model parameter file during segmentation model training and set it to use only the segmentation head module for secondary segmentation model training.

7. A road disease detection method according to claim 5, characterized in that: The cost function used when training the segmentation model is: ； in, L is the original Focal Loss loss function, The probability of the model predicting that it belongs to category c ranges from 0 to 1. is the probability that the model predicts belongs to the foreground, γ The adjustment factor for difficult sample mining is in the range of 0 to 5; The segmentation model training is performed using all parameters, and the segmentation model training is set to be performed for at least 200 rounds.

8. A road damage detection method according to claim 7, characterized in that: The cost function used in the secondary training of the segmentation head module is: ； in, is the Focal loss variant function, freq ( c ) is the disease category c The frequency of occurrence of the disease category c The frequency of occurrence is consistent with the proportion of each disease category in the segmentation model training. For category c Weights in the Focal Loss variant cost function.

9. An electronic device, characterized in that: The invention comprises a processor and a memory coupled to the processor, wherein the memory stores program instructions executable by the processor; when the processor executes the program instructions stored in the memory, the road damage detection method according to any one of claims 1 to 8 is implemented.

10. A storage medium, characterized in that: The storage medium stores program instructions, and when the program instructions are executed by the processor, the road damage detection method according to any one of claims 1 to 8 can be implemented.

Images