A road surface disease identification method and device, an electronic device, and a storage medium

Abstract

The application provides a road surface disease identification method and device, electronic equipment and storage medium. The method comprises the following steps: collecting initial near-infrared image data and underwater ultrasonic detection data of a target road surface under a water accumulation condition, and obtaining water depth data corresponding to the target road surface; correcting the initial near-infrared image data according to the water depth data to obtain target near-infrared image data; marking each pixel point in the target near-infrared image data according to the water depth data; inputting the marked target near-infrared image data and the underwater ultrasonic detection data into a road surface disease identification model to obtain a disease identification result corresponding to the target road surface. The application realizes road surface disease identification under the water accumulation condition by fusing the near-infrared image and the underwater ultrasonic detection data based on the water depth data, and can improve the detection accuracy of the road surface disease under the water accumulation condition.

Description

Technical Field

[0001] This invention relates to the field of road engineering defect detection technology, and more specifically, to a method, device, electronic device, and storage medium for identifying road defects. Background Technology

[0002] Road surface defect detection is a core foundational task for road maintenance, operation and maintenance assessment, and defect treatment. Accurate and efficient defect identification is key to ensuring road safety, extending road lifespan, and reducing maintenance costs. Traditional road surface defect identification technology mainly relies on machine vision image recognition, which uses visible light cameras to collect road surface images and combines them with deep learning algorithms to identify common defects such as cracks, potholes, and loose surfaces.

[0003] However, in waterlogged conditions, water can obstruct the road surface and cause problems such as specular reflection and light refraction, resulting in a significant decrease in the detection accuracy of traditional road defect identification algorithms under waterlogged conditions. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, device, electronic device and storage medium for identifying road surface defects, which can identify road surface defects under water accumulation conditions by fusing near-infrared images and underwater ultrasonic detection data based on water depth data, thereby improving the accuracy of road surface defect detection under water accumulation conditions.

[0005] In a first aspect, embodiments of this application provide a method for identifying road surface defects, the method comprising: The system collects initial near-infrared image data and underwater ultrasonic detection data of the target road surface under water accumulation conditions, and obtains the water depth data corresponding to the target road surface. The initial near-infrared image data is corrected based on the water depth data to obtain the target near-infrared image data; Based on the water depth data, water accumulation is marked for each pixel in the target near-infrared image data; The labeled target near-infrared image data and the underwater ultrasonic detection data are input into the road surface defect identification model to obtain the defect identification result corresponding to the target road surface.

[0006] In one possible implementation, the step of correcting the initial near-infrared image data based on the water depth data to obtain the target near-infrared image data includes: Based on the water depth data, the initial near-infrared image data is geometrically corrected to obtain intermediate near-infrared image data; Based on the water depth data, segmented grayscale compensation is performed on the grayscale-converted intermediate near-infrared image data to obtain the target near-infrared image data.

[0007] In one possible implementation, the step of performing geometric distortion correction on the initial near-infrared image data based on the water depth data to obtain intermediate near-infrared image data includes: Substitute the water depth data into the following formula to perform geometric distortion correction on the initial near-infrared image data to obtain intermediate near-infrared image data; ; ; ; in, The standard air refractive index; The water refractive index is preset to the near-infrared band; This refers to the air-side incident angle of near-infrared light. The angle of refraction of near-infrared light within the water layer; Let be the x-coordinate of pixel i in the initial near-infrared image data; The water depth is the actual road surface location corresponding to pixel i in the water depth data. Let be the x-coordinate of pixel i in the intermediate near-infrared image data; Let be the ordinate of pixel i in the initial near-infrared image data; Let be the vertical coordinate of pixel i in the intermediate near-infrared image data.

[0008] In one possible implementation, based on the water depth data, segmented grayscale compensation is performed on the grayscale-converted intermediate near-infrared image data to obtain target near-infrared image data, including: For each pixel in the grayscale intermediate near-infrared image data, linear compensation is performed on the grayscale value of the pixel in the grayscale intermediate near-infrared image data according to the preset gradient grayscale gain coefficient corresponding to the preset water depth compensation interval of the actual road surface location corresponding to the pixel, so as to obtain the grayscale value of the pixel in the target near-infrared image data; the water depth corresponding to the actual road surface location of the pixel is included in the water depth data.

[0009] In one possible implementation, water accumulation marking is performed on any pixel of the target near-infrared image data according to the following steps: If the water depth at the actual road surface location corresponding to the pixel in the water depth data is less than a preset depth threshold, then the water depth level corresponding to the pixel is marked as shallow water level. If the water depth at the actual road surface location corresponding to the pixel in the water depth data is greater than or equal to the preset depth threshold, then the water depth level corresponding to the pixel is marked as a deep water level.

[0010] In one possible implementation, the step of inputting the labeled target near-infrared image data and the underwater ultrasonic detection data into the pavement defect identification model to obtain the defect identification result corresponding to the target pavement includes: The labeled target near-infrared image data is input into the first feature extraction layer of the road surface defect identification model to obtain the road surface visual feature vector. The underwater ultrasonic detection data is input into the first feature extraction layer of the road surface defect identification model to obtain the road surface acoustic feature vector. The water accumulation markers corresponding to each pixel, the road surface visual feature vector, and the road surface acoustic feature vector are input into the feature fusion layer of the road surface defect identification model to obtain the target feature vector; The target feature vector is input into the disease detection layer of the road surface disease identification model to obtain the disease identification result corresponding to the target road surface.

[0011] In one possible implementation, the step of inputting the water accumulation markers corresponding to each pixel, the road surface visual feature vector, and the road surface acoustic feature vector into the feature fusion layer of the road surface defect identification model to obtain the target feature vector includes: For the first area where the water depth level is shallow water, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein, the first fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector; For the second region with a water depth level of deep water accumulation, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein, the third fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector.

[0012] Secondly, embodiments of this application also provide a road surface defect identification device, the device comprising: The acquisition module is used to acquire initial near-infrared image data and underwater ultrasonic detection data of the target road surface under water accumulation conditions, and to obtain the water depth data corresponding to the target road surface. The correction module is used to correct the initial near-infrared image data based on the water depth data to obtain the target near-infrared image data; A marking module is used to mark each pixel in the target near-infrared image data with water accumulation based on the water accumulation depth data; The identification module is used to input the marked target near-infrared image data and the underwater ultrasonic detection data into the road surface defect identification model to obtain the defect identification result corresponding to the target road surface.

[0013] In one possible implementation, the correction module is specifically used to perform geometric distortion correction on the initial near-infrared image data based on the water depth data to obtain intermediate near-infrared image data; and to perform segmented grayscale compensation on the grayscaled intermediate near-infrared image data based on the water depth data to obtain target near-infrared image data.

[0014] In one possible implementation, the correction module is specifically used to substitute the water depth data into the following formula to perform geometric distortion correction on the initial near-infrared image data to obtain intermediate near-infrared image data. ; ; ; in, The standard air refractive index; The water refractive index is preset to the near-infrared band; This refers to the air-side incident angle of near-infrared light. The angle of refraction of near-infrared light within the water layer; Let be the x-coordinate of pixel i in the initial near-infrared image data; The water depth is the actual road surface location corresponding to pixel i in the water depth data. Let be the x-coordinate of pixel i in the intermediate near-infrared image data; Let be the ordinate of pixel i in the initial near-infrared image data; Let be the vertical coordinate of pixel i in the intermediate near-infrared image data.

[0015] In one possible implementation, the correction module is specifically used to linearly compensate the grayscale value of each pixel in the grayscale intermediate near-infrared image data according to a preset gradient grayscale gain coefficient corresponding to a preset water depth compensation interval where the actual water depth at the corresponding road surface location of the pixel is located, thereby obtaining the grayscale value of the pixel in the target near-infrared image data; the water depth at the actual road surface location corresponding to the pixel is included in the water depth data.

[0016] In one possible implementation, the marking module is specifically used to mark any pixel in the target near-infrared image data as having accumulated water according to the following steps: if the water depth at the actual road surface location corresponding to the pixel in the water depth data is less than a preset depth threshold, then the water depth level corresponding to the pixel is marked as shallow water level; if the water depth at the actual road surface location corresponding to the pixel in the water depth data is greater than or equal to the preset depth threshold, then the water depth level corresponding to the pixel is marked as deep water level.

[0017] In one possible implementation, the identification module is specifically used to input the marked target near-infrared image data into the first feature extraction layer of the road surface defect identification model to obtain a road surface visual feature vector; input the underwater ultrasonic detection data into the first feature extraction layer of the road surface defect identification model to obtain a road surface acoustic feature vector; input the water accumulation mark corresponding to each pixel point, the road surface visual feature vector, and the road surface acoustic feature vector into the feature fusion layer of the road surface defect identification model to obtain a target feature vector; and input the target feature vector into the defect detection layer of the road surface defect identification model to obtain the defect identification result corresponding to the target road surface.

[0018] In one possible implementation, the identification module is specifically used to perform weighted fusion of the road surface visual feature vector and the road surface acoustic feature vector for a first area with a shallow water depth level to obtain a target feature vector; wherein, the first fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector; for a second area with a deep water depth level, the module performs weighted fusion of the road surface visual feature vector and the road surface acoustic feature vector to obtain the target feature vector; wherein, the third fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector.

[0019] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the road surface defect identification method as described in any of the first aspects.

[0020] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the pavement defect identification method as described in any of the first aspects.

[0021] This application provides a method, apparatus, electronic device, and storage medium for identifying road surface defects. The method includes: acquiring initial near-infrared image data and underwater ultrasonic detection data of a target road surface under waterlogged conditions, and obtaining water depth data corresponding to the target road surface; correcting the initial near-infrared image data based on the water depth data to obtain target near-infrared image data; marking each pixel in the target near-infrared image data with water accumulation based on the water depth data; and inputting the marked target near-infrared image data and underwater ultrasonic detection data into a road surface defect identification model to obtain the defect identification result corresponding to the target road surface. This application, based on water depth data and fusing near-infrared images and underwater ultrasonic detection data, achieves road surface defect identification under waterlogged conditions, which can improve the detection accuracy of road surface defects under waterlogged conditions. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of a pavement distress identification method provided in an embodiment of this application is shown; Figure 2 This paper shows a flowchart of the pavement distress identification model provided in an embodiment of the present application for distress identification. Figure 3 This illustration shows a structural schematic diagram of a road surface defect identification device provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0025] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0026] To enable those skilled in the art to utilize the content of this application, and in conjunction with the specific application scenario of "road engineering defect detection technology," the following embodiments are provided. For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application primarily describes the field of "road engineering defect detection technology," it should be understood that this is merely an exemplary embodiment.

[0027] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0028] Road surface defect detection is a core foundational task for road maintenance, operation and maintenance assessment, and defect treatment. Accurate and efficient defect identification is crucial for ensuring road safety, extending road lifespan, and reducing maintenance costs. Currently, the mainstream road surface defect identification technology in the industry is mainly based on machine vision image recognition. This technology uses visible light cameras to collect road surface images and combines them with deep learning algorithms to identify common defects such as cracks, potholes, and loose surfaces. It has the advantages of automation, high efficiency, and visualization and has been widely used in daily road inspection work.

[0029] However, existing visual recognition technologies have significant limitations in operating conditions. Detection accuracy drops drastically under waterlogged conditions, revealing obvious technical shortcomings. Specific problems are as follows: First, visible light images are highly susceptible to the effects of waterlogging. Water on the road surface can cause specular reflection, light refraction, and water mist obscuring the surface, resulting in blurred road textures and masked defects, leading to widespread missed and false detections. Second, thin layers of water can completely cover minor cracks and loose surface defects, while thick, turbid water can obscure all surface features, making it impossible for pure visual detection to obtain effective defect information. Third, pure visual detection can only identify visible surface defects and cannot detect hidden defects such as subgrade voids, roadbed subsidence, and deep underwater potholes under waterlogging. These hidden defects are the core hidden dangers leading to road collapse and large-scale damage, which existing technologies cannot effectively identify. Fourth, existing single-modal detection algorithms have weak anti-interference capabilities, cannot effectively remove waterlogging noise, have poor model generalization ability, and cannot adapt to complex waterlogging scenarios with different depths and turbidities.

[0030] To address the challenge of detecting flooded pavements, some existing technologies have attempted to optimize detection results using methods such as laser detection and single-point infrared detection. However, laser detection is susceptible to interference from water refraction, and single-point infrared detection cannot achieve full-area detection of large pavements, still suffering from issues such as missing identification of hidden defects, low detection efficiency, and insufficient accuracy. Ultrasonic detection technology has the advantages of penetrating water and detecting the internal structure of the medium, effectively acquiring information on underwater structural defects. Near-infrared imaging technology is resistant to visible light interference and has strong penetration capabilities through thin layers of water, stably extracting surface defect features of flooded pavements. Currently, there is no intelligent pavement defect identification solution specifically designed for flooded conditions that integrates near-infrared vision and ultrasonic detection, making it impossible to achieve integrated and accurate detection of both visible and hidden defects on flooded pavements.

[0031] In view of this, the present application provides a method for identifying pavement defects. The following is a detailed description of the pavement defect identification method provided in the embodiments of the present application.

[0032] Reference Figure 1 The diagram shown is a flowchart illustrating a pavement defect identification method provided in an embodiment of this application. The exemplary steps of this embodiment are described below: S101. Collect initial near-infrared image data and underwater ultrasonic detection data of the target road surface under water accumulation conditions, as well as water depth data corresponding to the initial near-infrared image data.

[0033] In this embodiment, a near-infrared camera in a near-infrared-ultrasound joint detection hardware system acquires image data of the target road surface under water accumulation conditions. Simultaneously, an ultrasonic probe in the system detects the target road surface, obtaining initial underwater ultrasonic detection data. Then, the image data and the initial underwater ultrasonic detection data undergo spatiotemporal synchronization processing to obtain initial near-infrared image data and final underwater ultrasonic detection data.

[0034] The target pavement refers to the pavement for which pavement defect identification is required. Waterlogging condition refers to the presence of water on the target pavement. The near-infrared-ultrasound combined hardware system is a pre-established and calibrated system for pavement defect identification.

[0035] The construction and calibration process of the near-infrared-ultrasound combined hardware system is as follows: A 4K near-infrared camera, a high-frequency waterproof ultrasonic probe, a synchronization controller, and an embedded data processing terminal are integrated into a vehicle-mounted inspection bracket. The installation height and angle of the equipment are adjusted to ensure that the near-infrared camera vertically covers the road surface inspection area, and the ultrasonic probe is vertically downwards aligned with the road surface. Equipment calibration is completed: the correspondence between near-infrared image pixels and actual road surface dimensions is calibrated, and the ultrasonic ranging accuracy is calibrated to ensure a ranging error ≤2mm. The synchronization controller is used to set up simultaneous triggering of both devices, with the timestamp synchronization error controlled within 10ms, completing unified spatial coordinate registration.

[0036] The detection carrier is a vehicle-mounted mobile detection platform or a fixed-point handheld detection device, which can realize rapid inspection of large-scale roads and accurate detection of local defects.

[0037] For example, municipal roads and national and provincial highways with post-rain water accumulation and low-lying areas were selected as test sections. The test vehicle was controlled to travel at a constant speed of 10-30 km / h, and near-infrared image acquisition and ultrasonic detection were started simultaneously. The near-infrared equipment acquired panoramic images of the road surface in real time, penetrating a thin layer of water (0-50 mm) to capture surface cracks, looseness, and damage features. The ultrasonic probe continuously emitted high-frequency ultrasonic waves, penetrating the water layer to collect echo signals from the road base and subgrade, recording acoustic data at different locations, and completing the synchronous storage of dual-modal raw data.

[0038] Here, the near-infrared image data is collected by a near-infrared camera using the 700-900nm near-infrared band. This can avoid interference from visible light reflections from water accumulation and penetrate thin layers of water to capture surface textures, cracks, and damage features. The underwater ultrasonic detection data is obtained by an ultrasonic probe using a high-frequency ultrasonic probe that emits ultrasonic waves vertically downwards, penetrating the water layer to collect acoustic echo data on the road surface's base flatness, structural voids, pothole depth, and loose base layer.

[0039] In addition to the initial near-infrared image data and underwater ultrasonic detection data, water depth data is also output simultaneously; the water depth data includes the water depth at the actual road surface location corresponding to each pixel in the near-infrared image data.

[0040] The water depth data can be detected using traditional methods such as liquid level sensors, pole radar, visual AI algorithms, and ultrasonic ranging.

[0041] S102. Correct the initial near-infrared image data based on the water depth data to obtain the target near-infrared image data.

[0042] In this embodiment, the inherent radial and tangential distortions of the lens in the initial near-infrared image data can be eliminated first using checkerboard calibration parameters, and then the initial near-infrared image data can be corrected based on the water depth data. Alternatively, the initial near-infrared image data can be corrected based on the water depth data.

[0043] Water refraction distortion combined with lens distortion causes a double offset, requiring layer-by-layer separation, with priority given to correcting the lens's inherent errors. For example, the process of eliminating the inherent distortion of the initial near-infrared image data can be as follows: read the calibrated and stored near-infrared camera intrinsic parameters and distortion coefficients; use the OpenCV standard distortion correction mapping algorithm to perform pixel-by-pixel reverse correction on the entire initial near-infrared image data (Imid); output an image (Imid) that retains only the geometric distortion caused by water refraction, and output pixel coordinates (Xmid, Ymid) as the input coordinates for inverse refraction correction.

[0044] Specifically, the process of correcting the initial near-infrared image data based on the water depth data is as follows: Step 1: Based on the water depth data, perform geometric distortion correction on the initial near-infrared image data to obtain the intermediate near-infrared image data.

[0045] In this embodiment, the true coordinates of the road surface are solved pixel-by-pixel using a pixel-level geometric inverse correction formula to eliminate image shift, stretching, and geometric distortion caused by water refraction. Specifically, the water depth data is substituted into the following pixel-level geometric inverse correction formula to correct the geometric distortion of the initial near-infrared image data, resulting in intermediate near-infrared image data: ; ; ; in, The standard air refractive index is 1.0. The water refractive index is preset for the near-infrared band (780-900, fixed value 1.328, different from the visible light refractive index, adapted to the working band of this equipment); This refers to the air-side incident angle of near-infrared light. The angle of refraction of near-infrared light within the water layer; Let be the x-coordinate of pixel i in the initial near-infrared image data; The water depth is the actual road surface location corresponding to pixel i in the water depth data. Let be the x-coordinate of pixel i in the intermediate near-infrared image data; Let be the ordinate of pixel i in the initial near-infrared image data; Let be the vertical coordinate of pixel i in the intermediate near-infrared image data.

[0046] Step 2: Based on the water depth data, perform segmented grayscale compensation on the grayscale-converted intermediate near-infrared image data to obtain the target near-infrared image data.

[0047] In this embodiment, the intermediate near-infrared image data is converted to grayscale to obtain the final intermediate near-infrared image data; then, an adaptive grayscale compensation algorithm based on water depth segmentation is simultaneously used to repair the grayscale distortion caused by water light attenuation in the grayscale intermediate near-infrared image data.

[0048] Specifically, based on the water depth data, segmented grayscale compensation is performed on the grayscale-converted intermediate near-infrared image data to obtain the target near-infrared image data. This includes: for each pixel in the grayscale-converted intermediate near-infrared image data, linear compensation is performed on the grayscale value of the pixel in the grayscale-converted intermediate near-infrared image data according to the preset gradient grayscale gain coefficient corresponding to the preset water depth compensation interval of the actual road surface location corresponding to the pixel, to obtain the grayscale value of the pixel in the target near-infrared image data; the water depth corresponding to the actual road surface location of the pixel is included in the water depth data.

[0049] In this embodiment, the preset water depth compensation range includes 0~5mm, 5~20mm, and 20~50mm. Each preset water depth compensation range corresponds to a preset gradient grayscale gain coefficient. The water depth at the actual road surface location corresponding to each pixel in the water depth data is read; then, linear compensation is performed based on the grayscale value of the pixel corresponding to the preset gradient grayscale gain coefficient within the preset water depth compensation range where the actual road surface location of the pixel falls. Gray_{new}=Gray_{origin}+Gain(h); Where Gray_{new} is the gray value of a pixel in the target near-infrared image data; Gray_{origin} is the gray value of a pixel in the intermediate near-infrared image data; and Gain(h) is the preset gradient gray gain coefficient corresponding to the preset water depth compensation interval where the actual water depth at the road surface location of the pixel is located.

[0050] Optionally, the grayscale values ​​in the target near-infrared image are uniformly constrained to the range of 0~255 to obtain the final target near-infrared image, so as to eliminate the problem of dark images and blurred textures in the water accumulation area.

[0051] Optionally, the near-infrared image of the target can be interpolated by bicubic subpixel interpolation reconstruction to fill the pixel holes generated by the correction mapping, outputting a high-definition, distortion-free road surface image, thus obtaining the final near-infrared image of the target.

[0052] For example, bicubic subpixel interpolation image reconstruction fills and corrects holes: (1) Coordinate inverse mapping will result in blank areas where the target coordinates do not correspond to the original pixels, and integer pixel mapping has sub-millimeter-level size errors: Image pixel coordinate remapping is completed based on the global mapping lookup table; For the blank pixel area after mapping, the surrounding 4×4 neighboring pixels are taken, and the filling gray value is calculated by weighted bicubic interpolation; Compared with bilinear interpolation, the subpixel-level fine edges of cracks and pits are preserved, and the jagged edges and contour stretching distortion of the corrected image are suppressed; The corrected near-infrared image with complete, geometric distortion-free, and uniform gray level is output.

[0053] Alternatively, a 5×5 Gaussian filtering algorithm can be used to denoise the target near-infrared image, and histogram equalization can be used to enhance contrast, among other preprocessing operations.

[0054] In summary, the embodiments of this application fully consider seven major influencing factors: fixed refractive index of air and water in the 780–900nm near-infrared band, independent light incident angle of each pixel, real-time dynamic water depth at each position, water depth disturbance of liquid surface ripples, light attenuation of water body, inherent lens distortion, and pixel mapping discrete error, thereby improving the accuracy of road surface defect identification.

[0055] S103. Based on the water depth data, mark each pixel in the target near-infrared image data with water accumulation.

[0056] In this embodiment of the application, the water accumulation marker can be a water accumulation level marker or a water accumulation coverage area marker.

[0057] Specifically, if the water accumulation marker is a water accumulation level marker, then the following steps are used to mark any pixel in the target near-infrared image data as a water accumulation point, including: if the water accumulation depth at the actual road surface location corresponding to the pixel in the water accumulation depth data is less than a preset depth threshold (e.g., 5mm), then the water accumulation depth level corresponding to the pixel is marked as shallow water accumulation level; if the water accumulation depth at the actual road surface location corresponding to the pixel in the water accumulation depth data is greater than or equal to the preset depth threshold, then the water accumulation depth level corresponding to the pixel is marked as deep water accumulation level.

[0058] Specifically, if the water accumulation is marked as a water-covered area, then an adaptive OTSU threshold segmentation combined with the U-Net semantic segmentation algorithm is used to accurately distinguish between water-covered areas and dry road surfaces in the target near-infrared image data; then, water-occluded areas in the target near-infrared image data are marked as water-covered areas, providing a basis for regional localization for subsequent ultrasonic feature completion and recognition.

[0059] S104. Input the marked target near-infrared image data and the underwater ultrasonic detection data into the road surface defect identification model to obtain the defect identification result corresponding to the target road surface.

[0060] In the embodiments of this application, reference is made to Figure 2 The diagram shown is a flowchart of the pavement distress identification model provided in this application embodiment for distress identification: S201. Input the labeled target near-infrared image data into the first feature extraction layer of the road surface defect identification model to obtain the road surface visual feature vector.

[0061] In this embodiment of the application, visual features such as edge gradient of the target road surface, texture distortion, crack direction, damaged area, and disease outline are extracted by a convolutional neural network to obtain a road surface visual feature vector.

[0062] S202. Input the underwater ultrasonic detection data into the first feature extraction layer of the road surface defect identification model to obtain the road surface acoustic feature vector.

[0063] In this embodiment, the original ultrasonic echo signal is filtered and denoised, and its spectrum is analyzed to extract acoustic features such as echo propagation time, amplitude attenuation coefficient, phase shift, and spectral peak value to obtain the road acoustic feature vector.

[0064] S203. Input the water accumulation mark corresponding to each pixel, the road surface visual feature vector, and the road surface acoustic feature vector into the feature fusion layer in the road surface defect identification model to obtain the target feature vector.

[0065] In this embodiment, the road surface visual feature vector and the road surface acoustic feature vector are normalized and then combined with the water accumulation markers corresponding to each pixel into the feature fusion layer to obtain the target feature vector. Specifically: If the water accumulation marker is a water accumulation level marker, then the water accumulation marker corresponding to each pixel, the road surface visual feature vector, and the road surface acoustic feature vector are input into the feature fusion layer of the road surface defect identification model to obtain the target feature vector. This includes: for a first region with a water accumulation depth level of shallow water accumulation, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein, the first fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector; for a second region with a water accumulation depth level of deep water accumulation, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein, the third fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector.

[0066] Specifically, if the water accumulation marker is a water accumulation coverage area marker, then the water accumulation marker corresponding to each pixel, the road surface visual feature vector, and the road surface acoustic feature vector are input into the feature fusion layer of the road surface defect identification model to obtain the target feature vector. This includes: for the third region where there is no water accumulation coverage area marker, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein the fifth fusion weight corresponding to the road surface visual feature vector is greater than the sixth fusion weight corresponding to the road surface acoustic feature vector; for the fourth region where there is a water accumulation coverage area marker, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein the seventh fusion weight corresponding to the road surface visual feature vector is greater than the eighth fusion weight corresponding to the road surface acoustic feature vector.

[0067] Here, through the method provided in the embodiments of this application, a detection mode that prioritizes near-infrared imaging and supplements ultrasound can be used in shallow water conditions, while automatically switching to a detection mode that prioritizes ultrasonic detection and supplements near-infrared image registration and correction in deep water conditions, thereby completing the intelligent identification of road surface defects across the entire water depth range.

[0068] S204. Input the target feature vector into the disease detection layer in the pavement disease identification model to obtain the disease identification result corresponding to the target pavement.

[0069] In this application's implementation, the disease identification results include disease type and disease area; disease types include surface visible diseases and underwater hidden diseases. Surface visible diseases include cracks, network cracks, loosening, and surface potholes, while underwater hidden diseases include hidden potholes under water accumulation, road base layer voids, roadbed settlement, and underwater structural damage. Furthermore, the length, width, area, depth, and settlement parameters of the disease can be obtained through model-based quantitative calculations, and the disease hazard level classification can be completed in conjunction with road maintenance standards.

[0070] Furthermore, the aforementioned road surface defect identification model is constructed based on an improved Transformer dual-modal fusion identification model, including a first feature extraction layer, a second feature extraction layer, a feature fusion layer, and a defect detection layer. The first and second feature extraction layers respectively complete the in-depth mining of near-infrared visual features and ultrasonic acoustic features. The feature fusion layer uses an attention mechanism to weighted fuse effective dual-modal features and suppress ineffective features such as water accumulation noise.

[0071] Road surface defect identification model training: The feature dataset was divided into training and testing sets in an 8:2 ratio. The training set contained road surface samples with different water depths (0-50mm), different turbidities, and different defect types. The model was iteratively trained using the training set, and an attention mechanism was used to adaptively fuse visual and acoustic features. For water-covered areas, the weight of ultrasonic acoustic features was strengthened, and for exposed road surfaces, the weight of near-infrared visual features was strengthened to suppress water noise interference. After training, the model parameters were optimized using the testing set to obtain the optimal identification model. Real-time collected bimodal features were input into the model to automatically classify and locate defects, quantify parameters such as defect length, width, area, pothole depth, and void extent, and classify the defect hazard level according to the "Highway Technical Condition Assessment Standard".

[0072] Furthermore, after the model recognition is completed, the road mileage and location coordinates are automatically matched to generate a visual disease detection report. The report includes information such as road section location, water accumulation status, disease type, disease size, disease depth, hazard level, and maintenance recommendations. At the same time, a disease ledger is automatically established to realize intelligent detection and digital management of waterlogged pavement diseases.

[0073] Through actual engineering testing, the pavement defect identification method provided in this application has achieved an overall accuracy rate of over 96.5% under various water accumulation conditions. Compared with traditional single visible light visual detection methods, the false negative rate is reduced by over 85%. It can effectively identify underwater hidden defects that traditional technologies cannot detect, and the detection stability and accuracy are greatly improved, fully meeting the detection needs of refined road maintenance.

[0074] In summary, the pavement distress identification method provided in this application has the following beneficial effects: (1) Dual-modal fusion detection with comprehensive identification range. The embodiments of this application integrate near-infrared vision and ultrasonic detection technology, which not only retains the advantages of visual detection in accurate positioning and visualization of surface defects, but also makes up for the shortcomings of pure vision in detecting underwater hidden defects. It can simultaneously identify visible defects such as surface cracks, network cracks, and surface potholes on waterlogged pavements, as well as hidden defects such as underwater potholes, base layer voids, and roadbed subsidence, realizing all-round detection of pavement defects and solving the problem of blind spots in existing technologies.

[0075] (2) Strong anti-interference ability and adaptable to complex water accumulation conditions. The image is acquired using the 700-900nm near-infrared band, which effectively avoids the interference of visible light water reflection and water mist. It can penetrate thin layers of water to stably extract the surface damage characteristics of the road surface. Combined with the detection characteristics of ultrasonic water penetration, it is not affected by the turbidity or depth of the water, which greatly improves the detection stability of complex conditions such as rainy days, post-rain water accumulation, and long-term water accumulation in low-lying areas.

[0076] (3) High recognition accuracy and high level of intelligence. By semantic segmentation, water accumulation noise is accurately removed. Combined with the attention mechanism dual-modal feature fusion model, effective disease features are adaptively selected and invalid interference information is suppressed. The disease recognition accuracy is ≥96%, which is far higher than the traditional single vision detection algorithm. At the same time, the size, depth and hazard level of the disease can be automatically quantified without the need for manual secondary verification, and the detection efficiency is significantly improved.

[0077] (4) Wide range of applicable scenarios and strong practicality. The solution of this invention can be integrated into a vehicle-mounted mobile inspection platform to realize rapid inspection of large-scale roads. It can also be adapted to handheld fixed-point devices for accurate verification of local defects. It can meet the needs of water accumulation condition defect detection for various types of roads such as highways, national and provincial trunk roads, municipal roads, and park roads. It is suitable for large-scale general surveys and refined inspection scenarios and has extremely high engineering application value.

[0078] Based on the same inventive concept, this application also provides a pavement defect identification device corresponding to the pavement defect identification method. Since the principle of the device in this application is similar to that of the pavement defect identification method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0079] Reference Figure 3 The diagram shown is a schematic representation of a road surface defect identification device provided in an embodiment of this application. The device includes: The acquisition module 301 is used to acquire initial near-infrared image data and underwater ultrasonic detection data of the target road surface under water accumulation conditions, and to obtain the water depth data corresponding to the target road surface. Correction module 302 is used to correct the initial near-infrared image data based on the water depth data to obtain target near-infrared image data; The marking module 303 is used to mark each pixel in the target near-infrared image data with water accumulation based on the water accumulation depth data; The identification module 304 is used to input the marked target near-infrared image data and the underwater ultrasonic detection data into the road surface defect identification model to obtain the defect identification result corresponding to the target road surface.

[0080] In summary, the road surface defect identification device provided in this application embodiment can identify road surface defects under waterlogged conditions by fusing near-infrared images and underwater ultrasonic detection data based on water depth data, thereby improving the accuracy of road surface defect detection under waterlogged conditions.

[0081] like Figure 4 As shown in the embodiment of this application, an electronic device 400 includes a processor 401, a memory 402, and a bus. The memory 402 stores machine-readable instructions that can be executed by the processor 401. When the electronic device is running, the processor 401 communicates with the memory 402 via the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the road surface defect identification method described above.

[0082] Specifically, the aforementioned memory 402 and processor 401 can be general-purpose memory and processor, without any specific limitations. When the processor 401 runs the computer program stored in the memory 402, it can execute the aforementioned road surface defect identification method.

[0083] Corresponding to the above-described pavement defect identification method, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the above-described pavement defect identification method.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0085] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0086] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0087] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the road surface defect identification method described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0088] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for identifying road surface defects, characterized in that, The method includes: The system collects initial near-infrared image data and underwater ultrasonic detection data of the target road surface under water accumulation conditions, and obtains the water depth data corresponding to the target road surface. The initial near-infrared image data is corrected based on the water depth data to obtain the target near-infrared image data; Based on the water depth data, water accumulation is marked for each pixel in the target near-infrared image data; The labeled target near-infrared image data and the underwater ultrasonic detection data are input into the road surface defect identification model to obtain the defect identification result corresponding to the target road surface.

2. The method for identifying road surface defects according to claim 1, characterized in that, The step of correcting the initial near-infrared image data based on the water depth data to obtain the target near-infrared image data includes: Based on the water depth data, the initial near-infrared image data is geometrically corrected to obtain intermediate near-infrared image data; Based on the water depth data, segmented grayscale compensation is performed on the grayscale-converted intermediate near-infrared image data to obtain the target near-infrared image data.

3. The method for identifying road surface defects according to claim 2, characterized in that, The step of performing geometric distortion correction on the initial near-infrared image data based on the water depth data to obtain intermediate near-infrared image data includes: Substitute the water depth data into the following formula to perform geometric distortion correction on the initial near-infrared image data to obtain intermediate near-infrared image data; ； ； ； in, The standard air refractive index; The water refractive index is preset to the near-infrared band; This refers to the air-side incident angle of near-infrared light. The angle of refraction of near-infrared light within the water layer; Let be the x-coordinate of pixel i in the initial near-infrared image data; The water depth is the actual road surface location corresponding to pixel i in the water depth data. Let be the x-coordinate of pixel i in the intermediate near-infrared image data; Let be the ordinate of pixel i in the initial near-infrared image data; Let be the vertical coordinate of pixel i in the intermediate near-infrared image data.

4. The method for identifying road surface defects according to claim 2, characterized in that, Based on the water depth data, segmented grayscale compensation is performed on the grayscale-converted intermediate near-infrared image data to obtain the target near-infrared image data, including: For each pixel in the grayscale intermediate near-infrared image data, linear compensation is performed on the grayscale value of the pixel in the grayscale intermediate near-infrared image data according to the preset gradient grayscale gain coefficient corresponding to the preset water depth compensation interval of the actual road surface location corresponding to the pixel, so as to obtain the grayscale value of the pixel in the target near-infrared image data; the water depth corresponding to the actual road surface location of the pixel is included in the water depth data.

5. The method for identifying road surface defects according to claim 1, characterized in that, Water accumulation marking is performed on any pixel of the target near-infrared image data according to the following steps: If the water depth at the actual road surface location corresponding to the pixel in the water depth data is less than a preset depth threshold, then the water depth level corresponding to the pixel is marked as shallow water level. If the water depth at the actual road surface location corresponding to the pixel in the water depth data is greater than or equal to the preset depth threshold, then the water depth level corresponding to the pixel is marked as a deep water level.

6. The method for identifying road surface defects according to claim 1, characterized in that, The step of inputting the labeled target near-infrared image data and the underwater ultrasonic detection data into the road surface defect identification model to obtain the defect identification result corresponding to the target road surface includes: The labeled target near-infrared image data is input into the first feature extraction layer of the road surface defect identification model to obtain the road surface visual feature vector. The underwater ultrasonic detection data is input into the first feature extraction layer of the road surface defect identification model to obtain the road surface acoustic feature vector. The water accumulation markers corresponding to each pixel, the road surface visual feature vector, and the road surface acoustic feature vector are input into the feature fusion layer of the road surface defect identification model to obtain the target feature vector; The target feature vector is input into the disease detection layer of the road surface disease identification model to obtain the disease identification result corresponding to the target road surface.

7. The method for identifying road surface defects according to claim 6, characterized in that, The step of inputting the water accumulation markers corresponding to each pixel, the road surface visual feature vector, and the road surface acoustic feature vector into the feature fusion layer of the road surface defect identification model to obtain the target feature vector includes: For the first area where the water depth level is shallow water, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein, the first fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector; For the second region with a water depth level of deep water accumulation, the road surface visual feature vector and the road surface acoustic feature vector are weighted and fused to obtain the target feature vector; wherein, the third fusion weight corresponding to the road surface visual feature vector is greater than the second fusion weight corresponding to the road surface acoustic feature vector.

8. A road surface defect identification device, characterized in that, The device includes: The acquisition module is used to acquire initial near-infrared image data and underwater ultrasonic detection data of the target road surface under water accumulation conditions, and to obtain the water depth data corresponding to the target road surface. The correction module is used to correct the initial near-infrared image data based on the water depth data to obtain the target near-infrared image data; A marking module is used to mark each pixel in the target near-infrared image data with water accumulation based on the water accumulation depth data; The identification module is used to input the marked target near-infrared image data and the underwater ultrasonic detection data into the road surface defect identification model to obtain the defect identification result corresponding to the target road surface.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the pavement defect identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the pavement defect identification method as described in any one of claims 1 to 7.

Images