Road disease identification method and device, electronic equipment and storage medium

Abstract

The invention provides a road disease identification method and device, electronic equipment and a storage medium, and belongs to the technical field of road detection, and the method comprises the steps: obtaining a pavement edge mask pattern based on an initial road image; obtaining a pavement effective area image based on the pavement edge mask pattern and the initial road image; extracting multi-dimensional features from the pavement effective area image, and determining a target pavement paving type based on the multi-dimensional features; if the target pavement paving type is a cement pavement type, calculating a local segmentation threshold value of each pixel in the pavement effective area image, and taking the pixel meeting a first condition as a candidate disease pixel to obtain a disease candidate mask; obtaining an effective disease area based on the disease candidate mask, and obtaining a disease identification result based on the effective disease area and the pavement effective area image; and if the target pavement paving type is a gravel pavement type, executing gravel disease identification operation to obtain a disease identification result. According to the invention, the accuracy of rural road disease identification can be improved.

Description

Technical Field

[0001] This application belongs to the field of road inspection technology, and more specifically, relates to road defect identification methods and devices, electronic devices, and storage media. Background Technology

[0002] Existing road defect identification technologies are mostly developed for wide urban highways, without considering the narrow rural roads and the interference from surrounding farmland, weeds, and dirt roads. This makes it easy to misidentify non-road areas as defects, and serious omissions of minor defects such as small cracks and broken edges. Existing general road identification models are difficult to adapt to the complex road conditions in rural areas, resulting in large fluctuations in recognition accuracy. Summary of the Invention

[0003] The purpose of this application is to provide a method and device for identifying road defects, electronic equipment, and storage medium to improve the accuracy of identifying road defects in rural areas.

[0004] A first aspect of this application provides a method for identifying road defects, including: Acquire an initial road image, perform preprocessing and edge detection on the initial road image to obtain a road surface edge mask image; Based on the road surface edge mask map and the initial road image, the effective area image of the road surface is obtained through a semantic segmentation model; Multi-dimensional features are extracted from the effective area image of the road surface, and the target road surface pavement type is determined from multiple pavement types based on the multi-dimensional features. The multi-dimensional features include gray-level mean, texture entropy value and roughness, and the multiple pavement types include cement pavement type and gravel pavement type. If the target pavement type is cement pavement, calculate the local segmentation threshold corresponding to each pixel in the effective area image of the pavement, and use the pixels in the effective area image of the pavement that meet the first condition as candidate defect pixels to obtain the defect candidate mask; perform morphological closing operation and connected component filtering on the defect candidate mask to obtain the effective defect area, and obtain the defect identification result based on the effective defect area and the effective area image of the pavement; the first condition is that the pixel gray value is not greater than the local segmentation threshold corresponding to the pixel; If the target pavement type is gravel pavement, then perform the gravel defect identification operation to obtain the defect identification results.

[0005] A second aspect of this application provides a road defect identification device, comprising: The image mask acquisition module is used to acquire the initial road image, perform preprocessing and edge detection on the initial road image, and obtain the road surface edge mask image; The effective region segmentation module is used to obtain the effective region image of the road surface based on the road surface edge mask map and the initial road image through a semantic segmentation model; The road surface type segmentation module is used to extract multi-dimensional features from the effective area image of the road surface, and determine the target road surface pavement type from multiple pavement types based on the multi-dimensional features. The multi-dimensional features include gray-level mean, texture entropy value and roughness, and the multiple pavement types include cement pavement type and gravel pavement type. The cement pavement defect identification module is used to calculate the local segmentation threshold corresponding to each pixel in the effective area image of the pavement if the target pavement type is cement pavement. Pixels in the effective area image that meet the first condition are selected as candidate defect pixels to obtain a defect candidate mask. Morphological closing operation and connected component filtering are performed on the defect candidate mask to obtain the effective defect area. The defect identification result is obtained based on the effective defect area and the effective area image of the pavement. The first condition is that the pixel gray value is not greater than the local segmentation threshold corresponding to the pixel. The gravel pavement defect identification module is used to perform gravel defect identification operations and obtain defect identification results if the target pavement type is gravel pavement.

[0006] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the road defect identification method described above.

[0007] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the road defect identification method described above.

[0008] The beneficial effects of the road defect identification method and device, electronic device, and storage medium provided in this application embodiment are as follows: This application's embodiments can solve the problem of misjudgment due to non-road surface interference and improve recognition reliability. This application's embodiments generate a road surface edge mask map through preprocessing and edge detection, and then combine it with a semantic segmentation model to extract pure road surface areas from the initial road image to form an effective region image. This thoroughly filters out interference from weeds, farmland, etc., avoiding misjudgment of non-road surface areas from the source, making the recognition results more consistent with the actual rural road scenarios.

[0009] The embodiments of this application can improve the recognition accuracy. First, the embodiments of this application accurately distinguish between pavement types such as cement and gravel using multi-dimensional features such as grayscale mean. Then, for cement pavements, local segmentation thresholds are designed to extract candidate defects, and morphological closing operations and connected component filtering are combined to enhance the recognition of subtle defects. Simultaneously, dedicated recognition logic is configured for gravel pavements, achieving dual optimization of pavement type adaptation and subtle defect recognition, significantly reducing the false negative rate and improving the recognition accuracy. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic flowchart illustrating a road defect identification method provided in an embodiment of this application; Figure 2 This is a structural block diagram of a road defect identification device provided in an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] It is understood that in the embodiments of this application, data such as user information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0014] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0015] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a road defect identification method according to an embodiment of this application. The method can be executed by an electronic device, and specifically, the method may include S101 to S105.

[0016] S101: Obtain the initial road image, perform preprocessing and edge detection on the initial road image, and obtain the road surface edge mask image.

[0017] In this embodiment, the initial road image is preprocessed and edge detected to obtain a road surface edge mask image, specifically including: Gaussian filtering is applied to the initial road image to obtain a denoised initial road image; Generate a gradient image based on the denoised initial road image; Edge detection is performed on the gradient image based on the first gradient threshold and the second gradient threshold to obtain a binarized edge image; Connectivity analysis is performed on the binarized edge image to obtain multiple continuous edge regions. All continuous edge regions in the binarized edge image whose continuous edge length is greater than the target pixel threshold are retained, and all continuous edge regions whose continuous edge length is not greater than the target pixel threshold are removed to obtain the road surface edge mask image.

[0018] In this embodiment, the initial road image refers to the original visual data of rural roads collected by vehicle-mounted cameras, portable devices, etc., including background information such as the road surface and surrounding farmland weeds. Edge detection refers to the processing method of identifying gray-level abrupt change regions in the image to extract the road surface boundary. The road surface edge mask image refers to a binarized image that retains only the road surface edge region and masks the rest. Gaussian filtering refers to the operation of smoothing the image using a Gaussian kernel function to reduce noise. Gradient image refers to an image that characterizes the magnitude and direction of gray-level changes in image pixels. The first gradient threshold and the second gradient threshold refer to two gray-level gradient thresholds used to distinguish edge pixels. The binarized edge image refers to an image in which pixels that meet the threshold conditions in the gradient image are set as foreground and the rest as background. Connected component analysis refers to the operation of identifying continuous regions formed by adjacent foreground pixels in the binarized image. Continuous edge region refers to an uninterrupted region formed by adjacent edge pixels in the binarized image. Target pixel threshold refers to the pixel length threshold used to filter effective continuous edges.

[0019] The underlying consideration of this embodiment is to address the interference of noise and non-road edges in the initial road image on subsequent road surface extraction, specifically for scenarios involving narrow rural roads with significant surrounding interference. This embodiment first removes image noise using Gaussian filtering to prevent noise from being misidentified as edges. Then, it highlights the grayscale difference between the road surface and the background using gradient images, laying the foundation for edge extraction. This embodiment employs dual gradient thresholding to accurately distinguish between strong and weak edges, reducing missed or false detections. Finally, this embodiment uses connected component analysis and length filtering to remove short interfering edges formed by weeds, pebbles, etc., retaining continuous road surface edges to ensure the accuracy of the road surface edge mask image, providing a reliable boundary basis for subsequent semantic segmentation to extract pure road surface regions.

[0020] For example, this embodiment can acquire initial road images of rural roads using a portable high-definition camera. Gaussian filtering is applied to the acquired initial road images using a 5×5 Gaussian kernel to smooth random noise caused by camera shake and environmental particles, resulting in a denoised initial road image. Based on the denoised initial road image, the gray-level gradients in the x and y directions are calculated using the Sobel operator and fused to obtain a gradient image, highlighting areas of abrupt gray-level changes between the road surface and the surrounding background. This embodiment can set a first gradient threshold of 50 and a second gradient threshold of 150, performing edge detection on the gradient image. Pixels with gradient values ​​higher than the second gradient threshold are identified as strong edges, while pixels with gradient values ​​between the two thresholds and connected to strong edges are identified as weak edges, generating a binarized edge image. This embodiment can perform connected component analysis on the binarized edge image, marking continuous edge regions formed by all adjacent foreground pixels. This embodiment can calculate the length of continuous edges in each region, setting a target pixel threshold of 100, retaining continuous edge regions longer than 100 pixels, and removing short interfering edge regions, ultimately obtaining a road surface edge mask image.

[0021] This embodiment effectively removes noise from the initial image through Gaussian filtering, improving the accuracy of edge detection. A dual-gradient threshold design balances strong edge extraction with weak edge preservation, reducing missed road edge detection. Connected component analysis and length filtering precisely eliminate non-road interfering edges, ensuring the purity of the road edge mask image. The resulting road edge mask image provides reliable boundary constraints for subsequent semantic segmentation, avoiding interference from non-road areas at the source. This lays the foundation for extracting clean, effective road surface areas, significantly improving the initial data quality for rural road defect identification.

[0022] S102: Based on the road surface edge mask map and the initial road image, the effective area image of the road surface is obtained through a semantic segmentation model.

[0023] In this embodiment, the semantic segmentation model includes an encoder and a decoder; based on the road surface edge mask map and the initial road image, the effective road surface area image is obtained through the semantic segmentation model, specifically including: Based on the initial road image, a multi-scale feature map is obtained through an encoder; the encoder is used to perform multi-level downsampling on the initial road image through depthwise separable convolution to obtain the multi-scale feature map. Based on multi-scale feature maps and road surface edge mask maps, a road surface area mask image is generated by a decoder. The decoder performs layer-by-layer upsampling of multi-scale feature maps through deconvolution to obtain upsampled feature maps at each level. Skip connections are used to concatenate and weight-fuse the upsampled feature maps at each level with the corresponding feature maps output by the encoder, resulting in a fused feature map. The road edge mask image is converted into a binary constraint tensor matching the size of the fused feature map. The binary constraint tensor is then element-wise multiplied with the fused feature map to obtain a feature map with edge contour constraints. A classification convolution is used to predict the road probability for each pixel in the edge contour-constrained feature map, resulting in a pixel-level classification probability map. The pixel-level classification probability map is then binarized to obtain a road area mask image. The effective area image of the road surface is obtained based on the road surface area mask image and the initial road image.

[0024] In this embodiment, the semantic segmentation model refers to a deep learning model capable of pixel-level classification of images, distinguishing between road and non-road areas. The encoder refers to the feature extraction module in the semantic segmentation model, which compresses image size and extracts features through multi-layer processing. The decoder refers to the feature restoration module in the semantic segmentation model, which restores image resolution and generates classification results through processing. Multi-scale feature maps refer to images output by the encoder after downsampling at different levels, containing road features at different scales. Depthwise separable convolution refers to splitting standard convolution into depthwise convolution and pointwise convolution, reducing computational cost while retaining feature extraction capabilities. Downsampling refers to the processing method of reducing image size to extract high-level features. Upsampling feature maps refer to feature maps whose resolution is gradually restored after deconvolution processing by the decoder. Skip connections refer to the structure that directly connects the corresponding level feature maps of the encoder to the upsampling feature maps of the decoder. Feature concatenation refers to the operation of merging feature maps from different sources along the channel dimension. Weight fusion refers to the processing of assigning weights to each channel of the concatenated feature map to highlight effective features.

[0025] A fused feature map refers to a feature map containing multi-scale details obtained after feature concatenation and weight fusion. A binary constraint tensor refers to converting a road surface edge mask image into a tensor containing only 0s and 1s, where 1 corresponds to the area enclosed by the road surface edge and 0 corresponds to the outer area. Element-wise multiplication refers to the operation of multiplying the binary constraint tensor with the corresponding pixels of the fused feature map one by one. An edge-contour-constrained feature map refers to an image that retains only the features of the area enclosed by the road surface edge after element-wise multiplication. Classification convolution refers to the convolution operation used to predict the road / non-road surface probability of feature map pixels. Road surface probability prediction refers to the process of calculating the probability value of each pixel belonging to the road surface region. A pixel-level classification probability map refers to an image where each pixel corresponds to a road surface probability value. Binarization refers to the operation of comparing the probability values ​​in the pixel-level classification probability map with a threshold to classify road surface and non-road surface areas. A road surface region mask image refers to a binary image after binarization, where white represents the road surface region and black represents the non-road surface region. A valid road surface region image refers to an image where the road surface region mask image is superimposed on the initial road image, retaining only the pixels corresponding to the white areas in the mask.

[0026] The underlying consideration of this embodiment is to address the problem of non-road surface background interference in the initial images of rural roads, ensuring accurate extraction of the road surface area. This embodiment first uses depthwise separable convolutional downsampling by the encoder to extract multi-scale road surface features while reducing computational load. This embodiment then uses deconvolutional upsampling by the decoder to restore resolution, and combines skip connections to supplement road surface details lost during downsampling, avoiding the omission of subtle road surface areas. This embodiment introduces a binary constraint tensor for road surface edge mask image transformation, limiting the feature calculation range through element-wise multiplication, retaining only the area surrounded by the road surface edges, thus eliminating non-road surface interference at the source. This embodiment predicts pixel probabilities through classification convolution, generates a mask through binarization, and finally overlays it with the initial image to obtain the pure road surface area, providing a clean data foundation for subsequent defect identification.

[0027] For example, this embodiment can use a lightweight semantic segmentation model. The encoder is configured with three depthwise separable convolutional layers. The first layer takes the initial road image as input (assuming a resolution of 1920×1080), and maps the number of channels from 3 to 32, downsampling to obtain a 960×540 feature map. The second layer maps the number of channels from 32 to 64, downsampling to obtain a 480×270 feature map. The third layer maps the number of channels from 64 to 128, downsampling to obtain a 240×135 feature map. These three layers together constitute a multi-scale feature map. The decoder first applies a 4×4 deconvolution with a stride of 2 to the 240×135 feature map to obtain a 480×270 upsampled feature map. This upsampled feature map is then concatenated with the 480×270 feature map output by the encoder through skip connections. A weight of 0.6 is assigned to the encoder feature map and 0.4 to the upsampled feature map for fusion, resulting in a 480×270 fused feature map. This process is repeated to obtain 960×540 and 1920×1080 fused feature maps in sequence.

[0028] This embodiment converts the road surface edge mask image (1920×1080) into a 1920×1080 binary constraint tensor, and performs element-wise multiplication with the 1920×1080 fused feature map to obtain a feature map with edge contour constraints. A 1×1 classification convolution is used to process this feature map, predicting the road surface probability for each pixel and generating a pixel-level classification probability map. This embodiment can set a probability threshold of 0.5; pixels with a probability ≥ 0.5 are set to white, otherwise to black, resulting in a road surface area mask image. This embodiment can overlay the road surface area mask image with the initial road image, retaining the initial image pixels corresponding to the white areas in the mask and removing the black area pixels, ultimately obtaining the effective road surface area image.

[0029] This embodiment reduces computational load through depthwise separable convolution of the encoder, making it suitable for on-site inspection equipment on rural roads. The decoder uses skip connections to supplement road surface details, avoiding the omission of minor road surface features. Binary constraint tensors precisely limit the computational range, significantly reducing interference from non-road surface backgrounds. The resulting image of the effective road surface area has high purity, providing reliable data support for subsequent multi-dimensional feature extraction and pavement type determination, effectively improving the accuracy and efficiency of the entire rural road defect identification process and reducing the risk of misjudgment in subsequent steps.

[0030] S103: Extract multi-dimensional features from the effective area image of the road surface, and determine the target road surface paving type from multiple road surface paving types based on the multi-dimensional features; the multi-dimensional features include gray-scale mean, texture entropy value and roughness, and the multiple road surface paving types include cement road surface type and gravel road surface type.

[0031] In this embodiment, multi-dimensional features are extracted from the effective area image of the road surface, specifically including: Convert the effective area image of the road surface into a grayscale image, and calculate the average pixel value of the grayscale image as the grayscale mean. A gray-level co-occurrence matrix is ​​constructed based on a gray-level image. The gray-level co-occurrence matrix includes multiple gray-level value pairs. The texture entropy value is calculated based on the probability of different gray-level value pairs appearing in the gray-level image. Edge detection is performed on the grayscale image to obtain the road surface edge image. The fractal dimension of the contour in the road surface edge image is calculated using the box dimension method, and the roughness is calculated based on the fractal dimension of the contour.

[0032] In this embodiment, the target pavement type is determined from multiple pavement types based on multi-dimensional features, specifically including: A classification feature vector is constructed based on gray-level mean, texture entropy value, and roughness. For each pavement type, the probability that the classification feature vector is classified into that pavement type is calculated using the sample dataset corresponding to that pavement type; The pavement type with the highest probability is taken as the target pavement type.

[0033] In this embodiment, a grayscale image refers to an image of the effective area of ​​the road surface converted from RGB three-channel to a single-channel image, retaining only pixel brightness information to simplify feature calculation. A grayscale co-occurrence matrix (GCMM) is a matrix that statistically analyzes the occurrence of grayscale value pairs between adjacent pixels at fixed distances and angles in a grayscale image, used to quantify road surface texture features. A grayscale value pair refers to the pairwise combination of pixel grayscale values ​​in the GCMM, reflecting the grayscale correlation between adjacent pixels. The occurrence probability refers to the proportion of a particular grayscale value pair appearing in the grayscale image out of the total number of grayscale value pairs, and is the basis for calculating texture entropy. The box-counting method refers to a method of calculating fractal dimension by covering the image edge contour with grids of different sizes and counting the number of grids, used to quantify roughness. Fractal dimension is a numerical value describing the complexity of the edge contour; its magnitude is positively correlated with the road surface roughness and is the core basis for calculating roughness. A classification feature vector refers to a three-dimensional data vector formed by combining the grayscale mean, texture entropy value, and roughness in a fixed order, used as input for pavement type classification. The sample dataset refers to a dataset containing labels for cement and gravel pavement types, with corresponding classification feature vectors extracted, used to calculate classification probabilities. Probability refers to the likelihood that a classification feature vector belongs to a certain pavement type, and is the basis for determining the target pavement type.

[0034] The underlying consideration of this embodiment is to address the poor adaptability of rural road pavement types, laying the foundation for subsequent targeted defect identification. Considering the significant differences between cement and gravel pavements in grayscale (cement is more mid-grayscale, gravel is more light-grayscale), texture (cement has a fine texture, gravel has a coarse texture), and roughness (cement has a smooth surface, gravel has a strong granular feel), this embodiment selects these three features to effectively distinguish between the two types of pavements. Calculating the classification probability using a sample dataset is more robust than simple threshold determination, avoiding misjudgments caused by single feature anomalies. This embodiment determines the target pavement type with the highest probability, ensuring accurate classification results and providing a reliable basis for subsequent dedicated defect identification processes for cement and gravel pavements.

[0035] For example, this embodiment can use a color space conversion method to convert the image from RGB format to grayscale, traversing all pixels of the grayscale image and calculating the arithmetic mean of the pixel grayscale values. This embodiment can set the window size to 15×15, the step size to 1, and the angle to 0°, counting the grayscale value pairs of all adjacent pixels within this window in the grayscale image, and recording the occurrence frequency of each grayscale value pair. Assuming the total occurrence frequency is 12000, and a certain grayscale value pair occurs 1800 times, then its occurrence probability is 1800 / 12000 = 0.15. This embodiment can obtain the texture entropy value based on the occurrence probability of all grayscale value pairs according to the entropy value calculation rules.

[0036] This embodiment performs Canny edge detection (dual thresholds 50 and 150) on a grayscale image to obtain a road edge image containing only the road surface edges. This embodiment can use the box-counting method, covering the contours of the road edge image with grids of sizes 1 pixel, 2 pixels, 4 pixels, and 8 pixels, respectively, resulting in corresponding grid counts of 800, 220, 65, and 20. This embodiment can calculate the fractal dimension based on the grid size and count.

[0037] This embodiment combines the grayscale mean, texture entropy, and roughness into a classification feature vector. This embodiment can access a sample dataset (containing 1000 sets of cement pavement samples and 1000 sets of gravel pavement samples, each containing a classification feature vector and type label), and calculate the probability that the feature vector belongs to cement pavement and the probability that it belongs to gravel pavement using the Naive Bayes equal probability calculation method. This embodiment compares the two probabilities and selects the pavement type corresponding to the highest probability as the target pavement type.

[0038] This embodiment accurately captures the essential differences between cement and gravel pavements using three features: grayscale mean, texture entropy, and roughness, avoiding misjudgment based on a single feature. Based on a probabilistic classification method using a sample dataset, this embodiment improves classification robustness in complex scenarios, addresses the poor adaptability of existing general models to rural road pavement types, and provides an accurate basis for subsequent calls to dedicated cement and gravel pavement defect identification processes. This reduces defect identification errors caused by misjudgment of pavement type from the source, improving overall defect identification accuracy.

[0039] S104: If the target pavement type is cement pavement, calculate the local segmentation threshold corresponding to each pixel in the effective area image of the pavement, and take the pixels in the effective area image of the pavement that meet the first condition as candidate defect pixels to obtain defect candidate masks; perform morphological closing operation and connected component filtering on the defect candidate masks to obtain effective defect areas, and obtain defect identification results based on the effective defect areas and the effective area image of the pavement; the first condition is that the pixel gray value is not greater than the local segmentation threshold corresponding to the pixel.

[0040] In this embodiment, morphological closing operations and connected component filtering are performed on the candidate disease mask to obtain the effective disease region. Based on the effective disease region and the effective road surface region image, the disease identification result is obtained, specifically including: Perform a morphological closing operation on the candidate disease mask to obtain the closed mask; Perform connected component analysis on the mask after the closing operation to obtain multiple connected components, and calculate the area of ​​each connected component; All connected domains with an area greater than the connected domain area threshold are considered as valid disease areas; For each connected region in the effective disease area, the contour is extracted, and polygon fitting is performed on the extracted contour to obtain the fitted contour of the connected region. Calculate the circularity and rectangularity of the fitted contour of each connected component. If the circularity of the fitted contour of the connected component is not greater than the first circularity threshold and the rectangularity of the fitted contour of the connected component is greater than the first rectangularity threshold, then the road surface area corresponding to the connected component is determined to be a broken plate area. Calculate the broken plate area and the broken plate area location. The area and location of the broken plate area are used as the results of broken plate disease identification. If the circularity of the fitted contour of the connected component is greater than the first circularity threshold and / or the rectangularity of the fitted contour of the connected component is not greater than the first rectangularity threshold, then edge enhancement is performed on the road surface area corresponding to the connected component. Linear crack segments are identified from the enhanced road surface area corresponding to the connected component. The number of all linear crack segments is calculated, the length of each linear crack segment is extracted, and the number of intersection points and the included angle between each linear crack segment are calculated. Based on the number of linear crack segments, the length of each linear crack segment, the number of intersection points and the included angle between each linear crack segment, the identification result of the network crack disease is determined. The results of identifying broken plate defects and network crack defects are used as the defect identification results.

[0041] In this embodiment, morphological closing operation refers to a morphological processing method that first performs a dilation operation on the candidate disease mask to fill the internal small holes, and then performs an erosion operation to restore the original contour of the disease area, used to optimize the integrity of the disease area. The mask after closing operation refers to a binary mask where the holes in the disease area are filled and the contour is more continuous after morphological closing operation processing. The connected component area threshold refers to the critical value of the connected component area used to screen effective diseases, set based on the minimum maintenance disease size of cement pavement combined with the image calibration ratio. Contour extraction refers to the operation of extracting boundary pixels from effective connected components to form continuous lines, used to obtain the boundary shape of the disease area. Polygon fitting refers to a processing method that approximates the extracted contour with polygons, simplifying the shape while retaining key boundary features. The fitted contour refers to the polygonal lines representing the boundaries of connected components obtained after polygon fitting. Circularity refers to a parameter describing how close the fitted contour is to a circle; the smaller the value, the closer it is to a non-circular block, such as a broken slab. Rectangularity refers to a parameter describing how close the fitted contour is to a rectangle; the larger the value, the closer it is to a rectangular block. The first roundness threshold and the first rectangularity threshold refer to the critical values ​​that distinguish broken slabs from other defects. They are determined based on a large number of cement pavement defect samples, such as roundness ≤ 0.3 and rectangularity > 0.6.

[0042] The term "broken slab region" refers to the pavement area corresponding to the connected domain identified as broken slab defects. The broken slab region area refers to the actual area of ​​the broken slab region, calculated from the number of pixels in the connected domain combined with the image calibration ratio. The broken slab region location refers to the coordinate information of the broken slab region in the effective pavement area image (such as center coordinates and boundary coordinates). The broken slab defect identification result refers to structured data containing the area and location of the broken slab region. Edge enhancement refers to operations that strengthen the contrast of linear crack edges using methods such as the Laplacian operator, used to improve crack recognition clarity. A linear crack segment refers to a single straight crack that constitutes a network of cracks. The number of intersection points refers to the number of points formed by the intersection of linear crack segments. The included angle refers to the angle between two linear crack segments. The network crack defect identification result refers to structured data containing the number, length, number of intersection points, and included angle of linear crack segments.

[0043] The consideration behind this embodiment is to address the significant morphological differences in core defects of cement pavements, such as blocky broken slabs and intersecting network cracks, avoiding type confusion and parameter omissions caused by general identification. This embodiment first uses morphological closing operations to fill defect mask holes and connected component filtering to eliminate small noise, ensuring the purity of the effective defect area. Then, it accurately distinguishes between the two types of defects based on dual thresholds of circularity (broken slabs are non-circular blocks) and rectangularity (broken slabs are nearly rectangular). This embodiment directly calculates the area and location of broken slabs to meet the quantitative requirements of blocky defects. For network cracks, this embodiment obtains key parameters of linear defects through edge enhancement, linear crack identification, and intersection point statistics, achieving morphological differentiation, classification, and quantitative output, providing accurate data support for maintenance.

[0044] For example, in this embodiment, a 5×5 rectangular structural element can be selected to perform morphological closing operation. First, the small holes with a diameter ≤ 3 pixels in the mask are expanded and filled (such as holes formed by crack gaps). Then, the original outline of the diseased area is restored by erosion to obtain the mask after closing operation. At this time, there are no voids inside the broken plate area and the crack outline is more continuous.

[0045] This embodiment can perform connected component analysis on the mask after the closing operation, mark the independent connected components, calculate the area of ​​each connected component according to the image calibration ratio, and if the area of ​​two connected components is less than a preset threshold, it can be determined as noise. After removing the areas of these two connected components, the remaining connected components that are not less than the preset threshold are retained as the effective disease area.

[0046] In this embodiment, contour extraction is performed on each valid diseased area to obtain continuous boundary lines, and then polygon fitting is used to obtain the fitted contour. This embodiment can set the first roundness threshold to 0.3 and the first rectangularity threshold to 0.6, and calculate the parameters of 5 fitted contours. Assuming that the roundness of 3 fitted contours are 0.22, 0.19, and 0.26 (all ≤ 0.3), and the rectangularity are 0.67, 0.71, and 0.69 (all > 0.6), they are identified as broken slab areas. This embodiment calculates the area according to the calibration ratio and records the center coordinates and boundary coordinates of each area as the broken slab disease identification result.

[0047] The remaining two fitted contours have circularity values ​​of 0.48 and 0.53 (both > 0.3) and rectangularity values ​​of 0.56 and 0.59 (both ≤ 0.6), respectively. Laplacian edge enhancement is applied to the corresponding areas in the effective road surface image to strengthen the grayscale difference between the crack edge and the surrounding road surface. This embodiment can use Hough line transform to identify linear crack segments. Assuming 7 and 9 linear crack segments are identified in each region, the length of each segment is measured, the number of intersections is counted, and the included angle is calculated.

[0048] This embodiment identifies network cracks based on the number of linear crack segments (≥3), the number of intersections (≥3), and the included angle range. This embodiment integrates the number of segments, the length of each segment, the number of intersections, and the included angle as the network crack defect identification result. This embodiment merges the broken slab defect identification result with the network crack defect identification result to obtain a complete cement pavement defect identification result including defect type, location, area, and line parameters.

[0049] This embodiment designs a dedicated identification logic based on the morphological differences of two core types of defects in cement pavements. It accurately distinguishes between broken slabs and network cracks by using roundness and rectangularity thresholds, avoiding type confusion and improving identification accuracy. Morphological closing operations and connected component filtering effectively eliminate noise, ensuring the purity of valid defect areas and reducing the false positive rate. This embodiment outputs the area and location of broken slabs and the number, length, intersection points, and angles of line segments for network cracks, achieving quantitative defect identification rather than simple type determination. The results can directly guide maintenance operations, providing reliable data support for precise maintenance of cement pavements in rural roads.

[0050] S105: If the target pavement type is gravel pavement, then perform the gravel defect identification operation to obtain the defect identification results.

[0051] In this embodiment, the sand and gravel disease identification operation includes: Pixels that meet the second condition in the effective area image of the road surface are used as candidate pixels for potholes to obtain the candidate pothole region. The pothole region fitting area, pothole location and pothole shape of the candidate pothole region are then determined. Pixels that meet the third condition in the effective area image of the road surface are taken as loose candidate pixels to obtain loose candidate regions. If the difference between the texture entropy value of the loose candidate region and the texture entropy value of the adjacent road surface region is greater than the texture entropy value difference threshold, the effective loose region area and the location of the loose region are determined. The fitted area of ​​the pit, the location of the pit, the shape of the pit, the area of ​​the effective loose area, the location of the loose area, and the texture entropy value are used as the results of the disease identification. The second condition is that the pixel grayscale value is not greater than the first grayscale threshold, or the pixel grayscale value is greater than the first grayscale threshold, not greater than the second grayscale threshold, and the local texture entropy value corresponding to the pixel is not greater than the texture entropy value threshold; the third condition is that the pixel grayscale value is greater than the first grayscale threshold, not greater than the second grayscale threshold, and the local texture entropy value corresponding to the pixel is greater than the texture entropy value threshold.

[0052] In this embodiment, the first grayscale threshold refers to the grayscale critical value for initially screening candidate pixels for potholes. The second grayscale threshold refers to the grayscale critical value for assisting in distinguishing potholes from loose candidate pixels. The local texture entropy value refers to the quantified value of the texture complexity of the local area surrounding a single pixel. Pothole candidate pixels refer to pixels that meet the second condition, and pothole candidate regions refer to the regions formed by pothole candidate pixels. The fitted area of ​​the pothole region refers to the actual area calculated after fitting the contour of the pothole candidate region. The pothole location refers to the coordinates of the pothole candidate region in the effective area image of the road surface. The pothole shape refers to the morphological characteristics of the contour of the pothole candidate region. Loose candidate pixels refer to pixels that meet the third condition, and loose candidate regions refer to the regions formed by loose candidate pixels. The adjacent road surface region refers to the normal gravel road surface region surrounding the loose candidate region. The texture entropy value difference refers to the difference in texture entropy values ​​between the loose candidate region and the adjacent road surface region. The texture entropy value difference threshold refers to the critical value of the texture entropy value difference for determining the validity of the loose region. The effective loose region area refers to the actual area determined as a valid loose region. The location of the loose area refers to the coordinates of the effective loose area in the image of the effective area of ​​the road surface.

[0053] The underlying consideration in this embodiment is to address the issue of potholes and loose surfaces on gravel pavements being similar in appearance (both may be irregular areas) and easily confused. Accurate differentiation is achieved through a combination of multiple conditions. Potholes, due to differences in depth or compaction, may exhibit low grayscale (≤ the first grayscale threshold) or medium grayscale but with regular texture (low local texture entropy). Loose surfaces, due to their disordered particle composition, exhibit medium grayscale and complex texture (high local texture entropy). Therefore, the second and third conditions are used to filter the two types of candidate regions respectively. Further verification through texture entropy differences eliminates pseudo-loose areas with small differences from normal pavement texture, ensuring the purity of both types of defect areas and providing a reliable foundation for subsequent quantitative parameter extraction.

[0054] For example, this embodiment can set a first grayscale threshold of 75, a second grayscale threshold of 85, a texture entropy threshold of 3.8, and a texture entropy difference threshold of 0.4. In this embodiment, for an effective area image of a gravel road surface with a resolution of 1280×720, pixels meeting the second condition are selected, and pixels with a grayscale value ≤ 75 are directly used as candidate pixels for potholes. Pixels with a grayscale value of 76-85 and a local texture entropy value ≤ 3.8 (such as latent potholes) are also used as candidate pixels for potholes, and these are integrated to obtain a pothole candidate region. In this embodiment, the contour of the pothole candidate region is fitted (fitting error ≤ 2 pixels), the fitted area of ​​the pothole region is calculated according to a calibrated ratio, and the pothole position and shape are recorded.

[0055] Pixels with grayscale values ​​of 76-85 and local texture entropy values ​​> 3.8 constitute loose candidate regions. In this embodiment, the texture entropy value of this loose candidate region is calculated to be 4.2, while the texture entropy value of its adjacent normal road surface area is 3.7. The difference is 0.5 > 0.4, thus it is determined to be a valid loose region. The area of ​​the valid loose region is calculated according to the calibration ratio, and the location of the loose region is recorded. This embodiment can integrate the quantization parameters and texture entropy values ​​of potholes and loose surfaces as the result of defect identification.

[0056] This embodiment, by combining dual grayscale thresholds with local texture entropy values, can accurately distinguish between potholes and loose areas on gravel roads, avoiding misjudgments caused by similar shapes and improving the accuracy of identifying both types of defects. Texture entropy value difference verification eliminates pseudo-loose areas, reducing the misjudgment rate. The quantitative parameters output by this embodiment can directly guide the calculation of maintenance material usage and the delineation of the work area, providing data support for precise maintenance of rural road gravel pavements.

[0057] Corresponding to the road defect identification method in the above embodiments, Figure 2 This is a structural block diagram of a road defect identification device according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The road defect identification device 20 includes: an image mask acquisition module 21, an effective area division module 22, a road surface type division module 23, a cement road surface defect identification module 24, and a gravel road surface defect identification module 25.

[0058] Among them, the image mask acquisition module 21 is used to acquire an initial road image, perform preprocessing and edge detection on the initial road image, and obtain a road surface edge mask image; The effective region segmentation module 22 is used to obtain the effective region image of the road surface based on the road surface edge mask map and the initial road image through a semantic segmentation model; The road surface type classification module 23 is used to extract multi-dimensional features from the effective area image of the road surface, and determine the target road surface pavement type from multiple road surface pavement types based on the multi-dimensional features; the multi-dimensional features include gray-level mean, texture entropy value and roughness, and the multiple road surface pavement types include cement pavement type and gravel pavement type; The cement pavement defect identification module 24 is used to calculate the local segmentation threshold corresponding to each pixel in the effective area image of the pavement if the target pavement type is cement pavement, select the pixels in the effective area image of the pavement that meet the first condition as candidate defect pixels, and obtain the defect candidate mask; perform morphological closing operation and connected component filtering on the defect candidate mask to obtain the effective defect area, and obtain the defect identification result based on the effective defect area and the effective area image of the pavement; the first condition is that the pixel gray value is not greater than the local segmentation threshold corresponding to the pixel; The gravel pavement defect identification module 25 is used to perform gravel defect identification operation and obtain defect identification results if the target pavement type is gravel pavement.

[0059] In one embodiment of this application, the sand and gravel disease identification operation includes: Pixels that meet the second condition in the effective area image of the road surface are used as candidate pixels for potholes to obtain the candidate pothole region. The pothole region fitting area, pothole location and pothole shape of the candidate pothole region are then determined. Pixels that meet the third condition in the effective area image of the road surface are taken as loose candidate pixels to obtain loose candidate regions. If the difference between the texture entropy value of the loose candidate region and the texture entropy value of the adjacent road surface region is greater than the texture entropy value difference threshold, the effective loose region area and the location of the loose region are determined. The fitted area of ​​the pit, the location of the pit, the shape of the pit, the area of ​​the effective loose area, the location of the loose area, and the texture entropy value are used as the results of the disease identification. The second condition is that the pixel grayscale value is not greater than the first grayscale threshold, or the pixel grayscale value is greater than the first grayscale threshold, not greater than the second grayscale threshold, and the local texture entropy value corresponding to the pixel is not greater than the texture entropy value threshold; the third condition is that the pixel grayscale value is greater than the first grayscale threshold, not greater than the second grayscale threshold, and the local texture entropy value corresponding to the pixel is greater than the texture entropy value threshold.

[0060] In one embodiment of this application, when the image mask acquisition module 21 performs preprocessing and edge detection on the initial road image to obtain a road surface edge mask image, it is specifically used for: Gaussian filtering is applied to the initial road image to obtain a denoised initial road image; Generate a gradient image based on the denoised initial road image; Edge detection is performed on the gradient image based on the first gradient threshold and the second gradient threshold to obtain a binarized edge image; Connectivity analysis is performed on the binarized edge image to obtain multiple continuous edge regions. All continuous edge regions in the binarized edge image whose continuous edge length is greater than the target pixel threshold are retained, and all continuous edge regions whose continuous edge length is not greater than the target pixel threshold are removed to obtain the road surface edge mask image.

[0061] In one embodiment of this application, the semantic segmentation model includes an encoder and a decoder; the effective region segmentation module 22, when obtaining the effective region image of the road surface based on the road surface edge mask map and the initial road image through the semantic segmentation model, is specifically used for: Based on the initial road image, a multi-scale feature map is obtained through an encoder; the encoder is used to perform multi-level downsampling on the initial road image through depthwise separable convolution to obtain the multi-scale feature map. Based on multi-scale feature maps and road surface edge mask maps, a road surface area mask image is generated by a decoder. The decoder performs layer-by-layer upsampling of multi-scale feature maps through deconvolution to obtain upsampled feature maps at each level. Skip connections are used to concatenate and weight-fuse the upsampled feature maps at each level with the corresponding feature maps output by the encoder, resulting in a fused feature map. The road edge mask image is converted into a binary constraint tensor matching the size of the fused feature map. The binary constraint tensor is then element-wise multiplied with the fused feature map to obtain a feature map with edge contour constraints. A classification convolution is used to predict the road probability for each pixel in the edge contour-constrained feature map, resulting in a pixel-level classification probability map. The pixel-level classification probability map is then binarized to obtain a road area mask image. The effective area image of the road surface is obtained based on the road surface area mask image and the initial road image.

[0062] In one embodiment of this application, the road surface type classification module 23, when extracting multi-dimensional features from the effective area image of the road surface, is specifically used for: Convert the effective area image of the road surface into a grayscale image, and calculate the average pixel value of the grayscale image as the grayscale mean. A gray-level co-occurrence matrix is ​​constructed based on a gray-level image. The gray-level co-occurrence matrix includes multiple gray-level value pairs. The texture entropy value is calculated based on the probability of different gray-level value pairs appearing in the gray-level image. Edge detection is performed on the grayscale image to obtain the road surface edge image. The fractal dimension of the contour in the road surface edge image is calculated using the box dimension method, and the roughness is calculated based on the fractal dimension of the contour.

[0063] In one embodiment of this application, when determining the target pavement type from multiple pavement types based on multi-dimensional features, the pavement type classification module 23 is specifically used for: A classification feature vector is constructed based on gray-level mean, texture entropy value, and roughness. For each pavement type, the probability that the classification feature vector is classified into that pavement type is calculated using the sample dataset corresponding to that pavement type; The pavement type with the highest probability is taken as the target pavement type.

[0064] In one embodiment of this application, when the cement pavement defect identification module 24 performs morphological closing operations and connected component filtering on defect candidate masks to obtain effective defect regions, and obtains defect identification results based on the effective defect regions and the pavement effective region image, it is specifically used for: Perform a morphological closing operation on the candidate disease mask to obtain the closed mask; Perform connected component analysis on the mask after the closing operation to obtain multiple connected components, and calculate the area of ​​each connected component; All connected domains with an area greater than the connected domain area threshold are considered as valid disease areas; For each connected region in the effective disease area, the contour is extracted, and polygon fitting is performed on the extracted contour to obtain the fitted contour of the connected region. Calculate the circularity and rectangularity of the fitted contour of each connected component. If the circularity of the fitted contour of the connected component is not greater than the first circularity threshold and the rectangularity of the fitted contour of the connected component is greater than the first rectangularity threshold, then the road surface area corresponding to the connected component is determined to be a broken plate area. Calculate the broken plate area and the broken plate area location. The area and location of the broken plate area are used as the results of broken plate disease identification. If the circularity of the fitted contour of the connected component is greater than the first circularity threshold and / or the rectangularity of the fitted contour of the connected component is not greater than the first rectangularity threshold, then edge enhancement is performed on the road surface area corresponding to the connected component. Linear crack segments are identified from the enhanced road surface area corresponding to the connected component. The number of all linear crack segments is calculated, the length of each linear crack segment is extracted, and the number of intersection points and the included angle between each linear crack segment are calculated. Based on the number of linear crack segments, the length of each linear crack segment, the number of intersection points and the included angle between each linear crack segment, the identification result of the network crack disease is determined. The results of identifying broken plate defects and network crack defects are used as the defect identification results.

[0065] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 2 The functions of the image mask acquisition module 21, effective area division module 22, road surface type division module 23, cement road surface defect identification module 24, and gravel road surface defect identification module 25 are shown.

[0066] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0067] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0068] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store information about the road surface type.

[0069] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in the embodiments of the road defect identification method provided in this application, or they can execute the implementation methods of the electronic device 300 described in the embodiments of this application, which will not be repeated here.

[0070] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0071] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0072] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0073] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules, units, or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or modules / units, or it may be an electrical, mechanical, or other form of connection.

[0075] The modules / units described as separate components may or may not be physically separate. Similarly, the components shown as modules / units may or may not be physical modules / units; they may be located in one place or distributed across multiple network modules / units. Some or all of the modules / units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0076] Furthermore, the functional modules / units in the various embodiments of this application can be integrated into one processing module / unit, or each module / unit can exist physically separately, or two or more modules / units can be integrated into one module / unit. The integrated modules / units described above can be implemented in hardware or in the form of software functional modules / units.

[0077] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A road disease recognition method characterized by comprising: The method comprises the following steps: obtaining an initial road image, preprocessing and edge detection are performed on the initial road image to obtain a road edge mask image; based on the road edge mask image and the initial road image, a road effective area image is obtained through a semantic segmentation model; extracting multi-dimensional features from the road effective area image, and determining a target road pavement type from a plurality of road pavement types based on the multi-dimensional features; the multi-dimensional features include gray mean value, texture entropy value and roughness, and the plurality of road pavement types include cement pavement type and gravel pavement type; if the target road pavement type is the cement pavement type, then the local segmentation threshold value corresponding to each pixel in the road effective area image is calculated, the pixels in the road effective area image that meet a first condition are taken as candidate disease pixels to obtain a disease candidate mask; morphological closing operation and connected domain screening are performed on the disease candidate mask to obtain an effective disease area, and a disease recognition result is obtained based on the effective disease area and the road effective area image; the first condition is that the pixel gray value is not greater than the local segmentation threshold value corresponding to the pixel; if the target road pavement type is the gravel pavement type, then a gravel disease recognition operation is performed to obtain a disease recognition result.

2. The road disease recognition method according to claim 1, characterized by, The gravel disease recognition operation comprises: taking the pixels in the road effective area image that meet a second condition as pit and groove candidate pixels to obtain a pit and groove candidate area, and determining a pit and groove area fitting area, a pit and groove position and a pit and groove shape of the pit and groove candidate area; taking the pixels in the road effective area image that meet a third condition as loose candidate pixels to obtain a loose candidate area; if the texture entropy value difference between the texture entropy value of the loose candidate area and the texture entropy value of the adjacent road area of the loose candidate area is greater than a texture entropy value difference threshold value, then an effective loose area area and a loose area position of the loose candidate area are determined; taking the pit and groove area fitting area, the pit and groove position, the pit and groove shape, the effective loose area area, the loose area position and the texture entropy value as the disease recognition result; the second condition is that the pixel gray value is not greater than a first gray threshold value, or the pixel gray value is greater than the first gray threshold value, not greater than a second gray threshold value, and the local texture entropy value corresponding to the pixel is not greater than a texture entropy threshold value; the third condition is that the pixel gray value is greater than the first gray threshold value, not greater than the second gray threshold value, and the local texture entropy value corresponding to the pixel is greater than the texture entropy threshold value.

3. The road disease recognition method according to claim 1, characterized by, The preprocessing and edge detection of the initial road image to obtain the road edge mask image comprises: performing Gaussian filtering processing on the initial road image to obtain a denoised initial road image; generating a gradient image based on the denoised initial road image; performing edge detection on the gradient image based on a first gradient threshold value and a second gradient threshold value to obtain a binary edge image; Performing connected domain analysis on the binarized edge image to obtain a plurality of continuous edge regions; retaining all continuous edge regions in the binarized edge image with a continuous edge length greater than a target pixel threshold, and eliminating all continuous edge regions with a continuous edge length not greater than the target pixel threshold, to obtain a road surface edge mask image.

4. The road disease recognition method according to claim 1, characterized by, The semantic segmentation model comprises an encoder and a decoder. The road surface effective region image is obtained based on the road surface edge mask image and the initial road image by using a semantic segmentation model, which comprises: Based on the initial road image, a multi-scale feature map is obtained by using the encoder; the encoder is configured to perform multi-layer down-sampling on the initial road image by using a depth separable convolution to obtain a multi-scale feature map; Based on the multi-scale feature map and the road surface edge mask image, a road surface region mask image is generated by using the decoder; The decoder is configured to perform layer-by-layer up-sampling on the multi-scale feature map by using deconvolution to obtain up-sampled feature maps at different levels, to perform feature splicing and weight fusion on the up-sampled feature maps at different levels and the feature maps at corresponding levels output by the encoder by using a skip connection to obtain a fused feature map; to convert the road surface edge mask image into a binary constraint tensor matching the size of the fused feature map, to multiply the binary constraint tensor and the fused feature map element by element to obtain a feature map after edge contour constraint; to perform road surface probability prediction on each pixel in the feature map after edge contour constraint by using a classification convolution to obtain a pixel-level classification probability map; and to perform binarization processing on the pixel-level classification probability map to obtain a road surface region mask image. The road surface effective region image is obtained based on the road surface region mask image and the initial road image.

5. The road disease recognition method according to claim 1, wherein The multi-dimensional features are extracted from the road surface effective region image, which comprises: The road surface effective region image is converted into a gray-scale image, and a pixel average value of the gray-scale image is calculated as a gray-scale average value; A gray-scale co-occurrence matrix is constructed based on the gray-scale image, the gray-scale co-occurrence matrix comprises a plurality of gray-scale value pairs, and a texture entropy value is calculated based on the occurrence probability of different gray-scale value pairs in the gray-scale image; An edge detection is performed on the gray-scale image to obtain a road surface edge image, a fractal dimension of a contour in the road surface edge image is calculated by using a box dimension method, and a roughness is calculated based on the fractal dimension of the contour.

6. The road disease recognition method according to claim 1, wherein The target road surface paving type is determined from a plurality of road surface paving types based on the multi-dimensional features, which comprises: A classification feature vector is constructed based on the gray-scale average value, the texture entropy value and the roughness; For each road surface paving type, a probability of the classification feature vector being divided into the road surface paving type is calculated by using a sample data set corresponding to the road surface paving type; The road surface paving type corresponding to the highest probability is taken as the target road surface paving type.

7. The road disease recognition method according to claim 1, wherein The effective disease region is obtained by performing a morphological closing operation and connected domain filtering on the disease candidate mask, and a disease recognition result is obtained based on the effective disease region and the road surface effective region image, which comprises: The morphological closing operation is performed on the disease candidate mask to obtain a mask after the closing operation; Performing connected component analysis on the closed operation post-mask to obtain a plurality of connected components, and calculating an area of each connected component; All connected components with an area greater than a connected component area threshold are regarded as effective disease areas; Performing contour extraction on each connected component in the effective disease area, and performing polygon fitting on the extracted contour to obtain a fitted contour of the connected component; Calculating a circularity and a rectangularity of the fitted contour of each connected component, and determining that a road surface area corresponding to the connected component is a broken plate area if the circularity of the fitted contour of the connected component is not greater than a first circularity threshold and the rectangularity of the fitted contour of the connected component is greater than a first rectangularity threshold, and calculating a broken plate area and a broken plate area position of the broken plate area; The broken plate area and the broken plate area position are taken as a broken plate disease recognition result; If the circularity of the fitted contour of the connected component is greater than the first circularity threshold and / or the rectangularity of the fitted contour of the connected component is not greater than the first rectangularity threshold, performing edge enhancement on the road surface area corresponding to the connected component, identifying linear crack line segments from the road surface area corresponding to the connected component after enhancement, calculating a number of all linear crack line segments, extracting a length of each linear crack line segment, and calculating a number of line segment intersection points and an included angle between each linear crack line segment; determining a netted crack disease recognition result based on the number of linear crack line segments, the length of each linear crack line segment, the number of line segment intersection points between each linear crack line segment, and the included angle between each linear crack line segment; The broken plate disease recognition result and the netted crack disease recognition result are taken as the disease recognition result.

8. A road disease recognition apparatus characterized by comprising: The method comprises the following steps: An image mask acquisition module is configured to acquire an initial road image, pre-process and perform edge detection on the initial road image, and obtain a road edge mask image; An effective area division module is configured to obtain a road effective area image by a semantic segmentation model based on the road edge mask image and the initial road image; A road type division module is configured to extract multi-dimensional features from the road effective area image, and determine a target road paving type from a plurality of road paving types based on the multi-dimensional features; the multi-dimensional features comprise a gray mean value, a texture entropy value, and a roughness; and the plurality of road paving types comprise a cement road surface type and a gravel road surface type; A cement road surface disease recognition module is configured to, if the target road paving type is the cement road surface type, calculate a local segmentation threshold corresponding to each pixel in the road effective area image, take pixels in the road effective area image that satisfy a first condition as candidate disease pixels, and obtain a disease candidate mask; Performing morphological closing operation and connected component filtering on the disease candidate mask to obtain an effective disease area, and obtaining a disease recognition result based on the effective disease area and the road effective area image; the first condition is that a pixel gray value is not greater than a local segmentation threshold corresponding to the pixel; A gravel road surface disease recognition module is configured to, if the target road paving type is the gravel road surface type, perform a gravel disease recognition operation to obtain a disease recognition result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

Images