Road pavement disease real-time dynamic detection method based on lightweight deep learning

Abstract

The invention discloses a road pavement disease real-time dynamic detection method based on lightweight deep learning, and relates to the technical field of pavement detection.The method comprises the steps that a multi-modal sensor is arranged on an inspection vehicle, and a suspected disease area is obtained through rapid positioning; carrying out computing resource allocation on the multi-mode sensor; obtaining a video frame time period when each suspected disease area is found, and distributing a unique ID for each suspected disease; constructing a disease recognition model, and screening the suspected disease areas to obtain a target disease area; identifying disease geometric parameters of the target disease area, and performing severity assessment to obtain disease risk parameters; obtaining a pavement material, and generating a maintenance processing decision in combination with the pavement material and the disease risk parameters; and generating a self-learning training package, carrying out self-training on the disease recognition model, and carrying out continuous optimization on the disease recognition model. The method has the effect of improving the accuracy, intelligence and applicability of highway pavement disease recognition.

Description

Technical Field

[0001] This application relates to the technical field of pavement inspection, and in particular to a lightweight deep learning method for real-time dynamic detection of highway pavement defects. Background Technology

[0002] Road surface inspection is a very important task, mainly used to detect defects in highways and then repair and maintain the road surface based on the inspection results.

[0003] In existing technologies, road surface inspection is generally conducted manually on the target highway using either visual inspection or specialized flaw detection instruments. However, these methods have drawbacks. For example, manual inspection is susceptible to environmental influences, leading to misjudgments. Furthermore, when analyzing road defects, it's easy to overlook underlying issues. In dynamic inspections, previously inspected defects may be re-inspected, resulting in wasted resources. Therefore, how to efficiently, accurately, and intelligently perform real-time dynamic detection of road surface defects has become an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight deep learning-based real-time dynamic detection method for highway pavement defects, in order to solve the problems mentioned in the background art.

[0005] This application provides a lightweight deep learning-based real-time dynamic detection method for highway pavement defects, the method comprising: Multimodal sensors are deployed on the inspection vehicle to acquire visible light image data, infrared light image data, and laser point cloud data of the road surface, and lightweight data processing is performed to quickly locate suspected defect areas. Identify the distance data between the suspected disease area and the inspection vehicle, and allocate computational resources to the multimodal sensor based on the distance data; The video frame time period when each suspected disease area is discovered is obtained, and the trajectory of each suspected disease area is identified according to the video frame time period to obtain the actual length and development trajectory of each suspected disease area, and a unique ID is assigned to each suspected disease. A disease identification model is constructed to perform high-precision classification and measurement of the suspected disease areas, and the suspected disease areas are screened to obtain the target disease areas; Identify the geometric parameters of the disease in the target disease area, and assess the severity of the disease based on the geometric parameters to obtain disease risk parameters; Based on the visible light image data and the infrared light image data, the road surface material is obtained. Combining the road surface material and the disease risk parameters, a maintenance treatment decision is generated. Obtain the disease identification parameters for each target disease area to generate a self-learning training package, and perform self-training on the disease identification model based on the self-learning training package to continuously optimize the disease identification model.

[0006] Preferably, the steps of acquiring visible light image data, infrared image data, and laser point cloud data of the road surface, and performing lightweight data processing to quickly locate suspected defect areas are as follows: Multimodal sensors are deployed on the inspection vehicle to acquire visible light image data, infrared light image data, and laser point cloud data of the road surface; Based on the visible light image data, region identification is performed on the visible light image data to obtain valid image regions and invalid image regions; Based on the failed image area, target area data of the same area in the infrared light image data and the laser point cloud data are extracted to supplement the failed image area, thereby obtaining a compensated image area. Based on the effective image area, combined with the infrared image data and the laser point cloud data, image recognition is performed to obtain the suspected disease area. Based on the compensated image area, the suspected disease area is obtained by performing perspective identification according to the infrared light image data and the laser point cloud data.

[0007] Preferably, the step of identifying the distance data between the suspected disease area and the inspection vehicle, and calculating resource allocation for the multimodal sensor based on the distance data, specifically includes: The suspected diseased area is spatially located to obtain its spatial coordinate data; Based on the spatial coordinate data, the distance between the suspected disease area and the inspection vehicle is obtained; Based on the distance data, the suspected disease areas are classified into near-field, mid-field, and far-field areas. Based on the near-field region, the data accuracy of the visible light image data is improved, while the data accuracy of the infrared light image data and the laser point cloud data is reduced. Based on the aforementioned mid-field region, the accuracy of the visible light image data, the infrared light image data, and the laser point cloud data is kept constant according to preset standard values; Based on the far-field region, the data accuracy of the visible light image data is reduced, while the data accuracy of the infrared light image data and the laser point cloud data is improved.

[0008] Preferably, the step of performing trajectory identification on each suspected disease area based on the video frame time period to obtain the actual length and development trajectory of each suspected disease area, and assigning a unique ID to each suspected disease, specifically includes: The video frame time period when each suspected diseased area is discovered is obtained, and the video frame time period is extended forward and backward by a preset number of video frames based on the discovery frame. During the video frame time period, the optical flow data of the suspected disease area and the current vehicle speed are extracted. Based on the current vehicle speed and the optical flow data, the motion trajectory of the suspected disease area is obtained. For each video frame following the discovery frame, identify the appearance features of the disease to obtain appearance feature change data; Based on the vehicle speed, the rate of change of observation angle is obtained. Based on the rate of change of observation angle and the appearance feature change data, the actual length and development trajectory of each suspected disease area are obtained. Determine whether the motion trajectory matches the appearance feature change data; If the motion trajectory is determined to match the appearance feature change data, a unique ID is assigned to the same suspected disease area spanning multiple frames.

[0009] Preferably, the steps of constructing a disease identification model, performing high-precision classification and measurement of the suspected disease areas, and screening the suspected disease areas to obtain the target disease areas are as follows: A disease identification model is constructed, and high-precision visible light images, high-precision infrared images, and high-precision point cloud data of the suspected disease areas are extracted based on the disease identification model. Based on the high-precision visible light image, high-precision infrared light image, and high-precision point cloud data, cross-validation of the suspected disease area is performed to obtain the multi-source fusion parameters of the suspected disease area. Based on the multi-source fusion parameters, the authenticity of the disease is evaluated to obtain the disease authenticity value of the suspected disease area; Determine whether the true severity value of the disease exceeds a preset true severity threshold. If the true severity value of the disease exceeds the true severity threshold, then mark the suspected disease area as the target disease area.

[0010] Preferably, the step of identifying the geometric parameters of the disease in the target disease area and assessing the severity based on the geometric parameters to obtain disease risk parameters specifically includes: Based on the multi-source fusion parameters corresponding to the target disease area, geometric identification is performed on the multi-source fusion parameters to obtain disease geometric parameters, which include gap width, pit depth and area ratio. Based on the infrared image data, the target disease area is identified and deeply mined to obtain the associated hidden diseases and the depth of the diseases. By combining the associated hidden diseases, the disease depth, the disease geometric parameters, the actual length, and the development trajectory, the severity of the target disease area is assessed to obtain a disease severity value. Based on the disease geometric parameters and the infrared image data, the disease type is obtained. A risk assessment is then performed by combining the disease type and the disease severity value to obtain disease risk parameters.

[0011] Preferably, the step of obtaining the road surface material based on the visible light image data and the infrared light image data, and then generating a maintenance treatment decision by combining the road surface material and the defect risk parameters, specifically includes: Based on the visible light image data, a road surface image is obtained. Color and texture recognition are performed on the road surface image to obtain the color features and texture features of the road surface. The first material parameter is generated by combining the color features and the texture features. Based on the infrared light image data, a road surface temperature distribution image is obtained. Based on the road surface temperature distribution image, the road surface thickness is identified to obtain the road surface thickness distribution parameters. Based on the thickness distribution parameters, a second material parameter is generated. By combining the first material parameter and the second material parameter, the material of the road surface is identified to obtain the road surface material; Based on the road surface material, the material information and treatment process for road maintenance are obtained. According to the disease risk parameters, the quantity of materials, maintenance area and maintenance depth for road maintenance are obtained. Based on the material information, the processing technology, the quantity of material, the curing area, and the curing depth, a curing treatment decision is generated.

[0012] Preferably, the step of obtaining disease identification parameters for each target disease area to generate a self-learning training package, and self-training the disease identification model based on the self-learning training package, specifically includes: Obtain the disease identification parameters for each target disease area, and then collect these disease identification parameters after standardized data preprocessing to generate a self-learning training package; The disease identification model learns by recognizing the training data in the self-learning training package and records the learning results after recognition and learning. The learning outcomes are compared with the training data, and the recognition errors generated during the comparison process are extracted. Based on the recognition error, record the erroneous recognition operations that generate errors during the training process, and generate error correction processing methods for the erroneous recognition operations; The error correction process is then incorporated into the disease identification model to optimize it.

[0013] In summary, this application includes at least one of the following beneficial technical effects: By deploying multimodal sensors on an inspection vehicle, visible light image data, infrared image data, and laser point cloud data of the road surface are collected. This data is first processed using lightweight methods to quickly locate suspected defect areas. Then, the distance between the suspected defect area and the inspection vehicle is identified, and computational resources for the multimodal sensors are dynamically allocated based on this distance data. The video frame time period when a suspected defect area is discovered is identified, and trajectory recognition is performed on the suspected defect area within this time period to obtain its actual length and development trajectory, assigning a unique ID to each suspected defect area. A defect identification model is then constructed to perform high-precision classification and measurement of suspected defect areas, followed by screening to identify target defect areas with confirmed defects. The geometric parameters of the defect in the target defect areas are identified, and their severity is assessed to obtain defect risk parameters. Then, based on the visible light and infrared image data, the road surface material is determined. Combining the road surface material and defect risk parameters, maintenance decisions are generated. Finally, the processed defect identification parameters are aggregated to generate a self-learning training package, allowing the defect identification model to train and optimize itself using this package. It improves the accuracy, intelligence, and applicability of identifying road surface defects. Attached Figure Description

[0014] Figure 1 This application provides a step-by-step flowchart of a lightweight deep learning-based real-time dynamic detection method for highway pavement defects. Detailed Implementation

[0015] The following combination Figure 1 This application will be described in further detail, but the embodiments of the present invention are not limited thereto. An embodiment of this application discloses a lightweight deep learning method for real-time dynamic detection of road surface defects.

[0016] In this embodiment, a lightweight deep learning-based real-time dynamic detection method for highway pavement defects is described, the method comprising: S100: Multimodal sensors are deployed on the inspection vehicle to acquire visible light image data, infrared light image data, and laser point cloud data of the road surface, and perform lightweight data processing to quickly locate suspected defect areas. S200: Identifies the distance data between suspected defect areas and the inspection vehicle, and allocates computing resources to multimodal sensors based on the distance data; S300: Obtain the video frame time period when each suspected disease area is discovered, and perform trajectory recognition on each suspected disease area according to the video frame time period to obtain the actual length and development trajectory of each suspected disease area, and assign a unique ID to each suspected disease. S400: Construct a disease identification model, perform high-precision classification and measurement of suspected disease areas, screen suspected disease areas, and obtain target disease areas; S500: Identify the geometric parameters of the disease in the target disease area, and assess the severity based on the geometric parameters to obtain disease risk parameters; S600: Based on visible light image data and infrared light image data, the road surface material is obtained, and maintenance treatment decisions are generated by combining the road surface material and the risk parameters of road defects. S700: Obtain the disease identification parameters for each target disease area to generate a self-learning training package, and perform self-training on the disease identification model based on the self-learning training package to continuously optimize the disease identification model.

[0017] The steps for acquiring visible light image data, infrared image data, and laser point cloud data of the road surface, and performing lightweight data processing to quickly locate suspected defect areas are as follows: Multimodal sensors are deployed on the inspection vehicle to acquire visible light image data, infrared light image data, and laser point cloud data of the road surface; Based on visible light image data, region identification is performed on the visible light image data to obtain valid image regions and invalid image regions; Based on the failed image area, target area data of the same area in the infrared light image data and laser point cloud data are extracted to supplement the failed image area and obtain the compensated image area. Based on the effective image area, combined with infrared light image data and laser point cloud data, image recognition is performed to obtain suspected disease areas. Based on the compensated image area, perspective identification is performed using infrared image data and laser point cloud data to obtain suspected defect areas.

[0018] In practice, taking a highway as an example, on a section of a highway from K100+200 to K100+500, an inspection vehicle conducts routine patrols at a speed of 60 kilometers per hour. Multimodal sensors mounted on the vehicle's roof begin working synchronously: a visible light camera captures color images of the road surface, an infrared thermal imager captures images of the road surface's temperature distribution, and a lidar scans the road surface and generates dense 3D point cloud data. The system first processes the visible light image, using image recognition algorithms to segment the image area. For example, at K100+300, due to dust kicked up by vehicles ahead, a blurred failure area appears in the center of the image. The system then extracts the temperature data and 3D elevation data corresponding to this blurred area from the simultaneously acquired infrared image and lidar point cloud data. This data is used to "paint" the texture and outline of the failure area, forming a compensated image area. Simultaneously, in the clear, effective areas of the image, the system combines infrared data (e.g., detecting a temperature significantly lower than the surrounding area) and point cloud data (e.g., detecting a slight elevation dip) to mark several suspected defect areas. In the compensation area, the system primarily relies on temperature anomalies from infrared data and deformation information from point cloud data, using perspective calculations to mark suspected points. The entire process is completed in real time while the vehicle is in motion, quickly identifying multiple suspected points on the road segment that require further attention.

[0019] The steps for identifying the distance data between suspected defect areas and the inspection vehicle, and for calculating resource allocation for multimodal sensors based on the distance data, are as follows: The spatial location of suspected disease areas is determined to obtain the spatial coordinate data of the suspected disease areas; Based on spatial coordinate data, the distance data between the suspected disease area and the inspection vehicle is obtained; Based on distance data, multiple suspected disease areas were classified into near-field, mid-field, and far-field areas. Based on the near-field region, improve the data accuracy of visible light image data, and reduce the data accuracy of infrared light image data and laser point cloud data; Based on the midfield area, the accuracy of visible light image data, infrared light image data, and laser point cloud data is kept constant according to preset standard values; Based on the far-field region, the data accuracy of visible light image data is reduced, while the data accuracy of infrared light image data and laser point cloud data is improved.

[0020] In practice, taking a highway as an example, the inspection vehicle continues to travel along the road, and the system spatially locates all the suspected defect areas that have just been identified. Using onboard GPS, inertial measurement units, and laser point cloud data, the precise distance of each suspected area relative to the inspection vehicle is calculated. For example, a suspected crack located approximately 5 meters in front of the vehicle is marked as a "near-field area"; a suspected pothole located approximately 20 meters in front of the vehicle is marked as a "mid-field area"; and a suspected network crack located approximately 50 meters in front of the vehicle is marked as a "far-field area." Then, based on this classification result, the system dynamically adjusts the data processing strategies of each sensor. For the near-field crack at 5 meters, the system instructs the visible light camera to use higher resolution for close-up local photography to see texture details, while temporarily reducing the processing frequency of infrared and laser point cloud data for the same area to save computing power. For the mid-field pothole at 20 meters, all sensors maintain a preset standard accuracy for data acquisition and processing. For the far-field network crack at 50 meters, the system reduces the resolution of the visible light image (because the details are difficult to distinguish) and instead increases the scanning frequency and data accuracy of the infrared thermal imager and lidar, relying on temperature differences and macroscopic deformation characteristics to maintain effective monitoring of the area.

[0021] The steps for performing trajectory identification on each suspected disease area based on the video frame time period, obtaining the actual length and development trajectory of each suspected disease area, and assigning a unique ID to each suspected disease are as follows: The video frame time period when each suspected disease area is discovered is obtained. The video frame time period is extended forward and backward by a preset number of video frames from the discovery frame. Within a video frame time period, extract the optical flow data of the suspected disease area and the current vehicle speed. Based on the current vehicle speed and the optical flow data, obtain the motion trajectory of the suspected disease area. For each video frame after the discovery frame, the appearance features of the disease are identified to obtain data on changes in appearance features; Based on vehicle speed, the rate of change of observation angle is obtained. Based on the rate of change of observation angle and the data on changes in appearance features, the actual length and development trajectory of each suspected disease area are obtained. Determine whether the motion trajectory matches the data on changes in appearance features; If the motion trajectory is determined to match the appearance feature change data, a unique ID is assigned to the same suspected disease area spanning multiple frames.

[0022] In application, taking a highway as an example, when the system first identifies a suspected longitudinal crack at K100+350 in frame 1205 of the video stream, it automatically extracts a video segment of 15 frames before and after the detection frame (i.e., frames 1190 to 1220). Within this short sequence of 31 frames, the system extracts the optical flow data of the suspected crack area and, combined with the current speed of the inspection vehicle (60 km / h, approximately 16.7 m / s), calculates the theoretical trajectory of the crack within the image sequence. Simultaneously, for each frame after frame 1205, the system continuously identifies the appearance features of the area, such as the crack's edge shape and color contrast, obtaining a series of minute appearance change data. Based on the vehicle speed, the system calculates the rate of rapid change of the viewing angle, and, combined with the continuous evolution of appearance features, reversely calculates that the actual physical length of the crack on the road surface is approximately 2.1 meters, and its development trajectory is basically parallel to the road direction. The system then compared the calculated optical flow trajectory with the observed appearance features and found a high degree of consistency, thus determining that the suspected defects appearing in these 31 frames were the same entity. Therefore, the system assigned it a globally unique ID, such as "Crack-20231027-001," to prevent it from being misclassified as multiple independent short cracks in subsequent processing.

[0023] The steps for constructing a disease identification model, performing high-precision classification and measurement of suspected disease areas, screening suspected disease areas, and obtaining target disease areas are as follows: A disease identification model was constructed, and high-precision visible light images, high-precision infrared images, and high-precision point cloud data of suspected disease areas were extracted based on the disease identification model. Based on high-precision visible light images, high-precision infrared images, and high-precision point cloud data, cross-validation of images is performed on suspected disease areas to obtain multi-source fusion parameters for the suspected disease areas; Based on the multi-source fusion parameters, the authenticity of the disease is evaluated to obtain the disease authenticity value of the suspected disease area; Determine whether the true severity value of the disease exceeds the preset true severity threshold. If it is determined that the true severity value of the disease exceeds the true severity threshold, then mark the suspected disease area as the target disease area.

[0024] In application, taking a highway as an example, the system's built-in lightweight defect identification model is activated to conduct in-depth analysis of suspected defect areas with assigned IDs. For the area with ID "Crack-20231027-001", the model first extracts high-precision data segments for that area from the raw data stream: a magnified and enhanced visible light close-up image, a frame of infrared temperature distribution map with higher thermal sensitivity, and a denser 3D laser point cloud data. Next, the model cross-validates these high-precision data. It measures the visual width of the crack from the visible light image, observes whether the low-temperature anomaly in the linear area of ​​the crack is continuous from the infrared image, and calculates whether there is a linear depression depth at the location of the crack from the point cloud data. This information is fused to generate a set of multi-source fusion parameters, including "visual width 0.8 cm", "strong low-temperature continuity", and "depression depth 0.5 cm". Then, the model evaluates the authenticity of this set of parameters based on evaluation rules trained on historical data. The rules may consider areas that simultaneously meet the following three conditions—"width greater than 0.5 cm", "low-temperature continuity exceeding 80%", and "clear indentation"—to be highly likely to be genuine cracks. After calculation, the area's degree of damage is scored 92 (out of 100). The system's preset threshold for degree of damage is 85. Since 92 is greater than 85, the system officially marks this suspected area as a "target damage area," locking its ID and all associated data, awaiting further detailed quantitative evaluation.

[0025] The steps for identifying the geometric parameters of the target disease area and assessing the severity of the disease based on these parameters to obtain disease risk parameters are as follows: Based on the multi-source fusion parameters corresponding to the target disease area, geometric identification is performed on the multi-source fusion parameters to obtain the disease geometric parameters, which include gap width, pit depth and area ratio. Based on infrared light image data, the associated diseases in the target disease area are identified and deeply mined to obtain the associated hidden diseases and the depth of the diseases. By combining associated hidden diseases, disease depth, disease geometric parameters, actual length, and development trajectory, the severity of the target disease area is assessed to obtain a disease severity value. Based on the geometric parameters of the disease and infrared image data, the disease type is obtained. A risk assessment is then conducted by combining the disease type and the severity value to obtain the disease risk parameters.

[0026] In application, taking a highway as an example, for the confirmed target defect area "Crack-20231027-001", the system begins precise geometric measurement and depth assessment. Based on previously fused high-precision multi-source data, the geometric recognition algorithm accurately measured the maximum width of the crack to be 8.2 mm and the average depth to be 5.1 mm, with its affected area accounting for approximately 0.15% of the lane area. Simultaneously, the system retrieves the original infrared image data of the area for further analysis. It was found that within approximately 10 cm on both sides of the visible crack, there was a subtle gradient change in pavement temperature, suggesting the possible existence of "hidden" damage areas within the pavement structure layer caused by water ingress or loosening. Combining the crack's geometric parameters (width 8.2 mm, depth 5.1 mm), estimated actual length (2.1 meters), development trajectory (longitudinal), and newly discovered related hidden defect signs, the system activates a severity assessment algorithm. This algorithm integrates these factors and outputs a defect severity value, such as 75 (range 0-100, higher values ​​indicate greater severity). Next, based on the geometric characteristics of the crack (linear, with a certain width and depth) and infrared characteristics (linear low-temperature zone), the system determines the disease type as "moderately developed longitudinal temperature crack". Finally, combining the "longitudinal temperature crack" type and its severity value of 75, and referring to the risk assessment matrix, the system generates the final disease risk parameters, such as "risk level: medium to high; recommended inspection cycle: shortened to 1 month".

[0027] Based on visible light and infrared image data, the road surface material is obtained. Combining the road surface material with damage risk parameters, the steps for generating maintenance and treatment decisions are as follows: Based on visible light image data, a road surface image is obtained. Color and texture recognition are performed on the road surface image to obtain the color and texture features of the road surface. The first material parameters are generated by combining the color and texture features. Based on infrared light image data, a road surface temperature distribution image is obtained. Based on the road surface temperature distribution image, the road surface thickness is identified to obtain the road surface thickness distribution parameters. Based on the thickness distribution parameters, a second material parameter is generated. By combining the first material parameter and the second material parameter, the material of the road surface is identified, and the road surface material is obtained. Based on the road surface material, the material information and treatment process for road maintenance are obtained. Based on the disease risk parameters, the quantity of materials, maintenance area and maintenance depth for road maintenance are obtained. Based on material information, processing technology, material quantity, curing area, and curing depth, a curing treatment decision is generated.

[0028] In application, taking a highway as an example, the system analyzes normal visible light image data of the target road section. The color recognition module determines the road surface is generally dark black, while the texture recognition module identifies uniform granular texture and a few polishing marks. Combining these features, the system generates the first material parameter, initially inferring "asphalt concrete." Next, the system analyzes a large area of ​​the original infrared temperature distribution image of the road section. It finds that the temperature rise rate and distribution pattern of the road surface under sunlight match the typical thermophysical characteristics of asphalt pavement, and the temperature distribution is relatively uniform, without obvious large areas of low temperature caused by base layer delamination. Therefore, the system estimates the road thickness to be approximately 12 to 15 centimeters, generating the second material parameter. Combining the first and second material parameters, the system ultimately confirms the road material as "approximately 13 centimeters thick dense asphalt concrete (AC-13)." Based on this material information, the system matches commonly used maintenance materials and processes from the knowledge base, such as "using modified asphalt sealant for crack filling." Meanwhile, based on the risk parameters of the defect (medium-high risk level, crack length 2.1 meters, width 8.2 millimeters), the system calculated that approximately 2.5 kg of sealant was needed, the maintenance area to be treated was approximately 0.017 square meters, and the grouting depth should reach 5 cm. Finally, the system integrated this information to generate a complete maintenance treatment decision: "At K100+350, for the longitudinal crack with ID Crack-20231027-001, pressure grouting with modified bitumen sealant is to be performed, with an estimated usage of 2.5 kg and a treatment depth of 5 cm. Construction should be arranged within one month." The steps for obtaining disease identification parameters for each target disease area to generate a self-learning training package, and then self-training the disease identification model based on the self-learning training package, are as follows: Obtain the disease identification parameters for each target disease area, and after standardizing and preprocessing these disease identification parameters, aggregate them to generate a self-learning training package; The disease identification model learns from the training data in the self-learning training package and records the learning results after the identification learning. The learning outcomes are compared with the training data, and the recognition errors generated during the comparison process are extracted. Based on the recognition error, record the erroneous recognition operations that generate errors during the training process, and generate error correction processing methods for the erroneous recognition operations; The error correction process is incorporated into the disease identification model to optimize it.

[0029] In practice, taking a highway as an example, after a complete inspection task, the system begins self-learning optimization. It collects complete defect identification parameters for all finally confirmed "target defect areas" from the task. For example, besides "Crack-20231027-001," it includes parameter sets for several other potholes and repair damage areas. The system performs standardized preprocessing on these parameter sets, such as unifying all geometric units to millimeters and normalizing severity values ​​to between 0 and 1, and then packages them into a self-learning training package named "20231027_Highway GXX Inspection." Subsequently, the defect identification model loads this training package as new training data for a round of identification learning. The model attempts to identify these data using its current internal algorithm and records the learning results, such as the success rate of identifying cracks on new types of repair edges. After learning is complete, the system compares the model's learning results (such as the identified defect types and geometric parameters) with the real data labeled manually or by more advanced algorithms in the training package, extracting the identification error. For example, the model might misidentify a subtle lateral texture as a crack, which is a misidentification. The system records this error and the data characteristics at the time of the error. For this error, the system generates a corresponding error correction method, such as "when the width of a suspected linear area is consistently less than 0.3 cm and the infrared temperature difference is less than 0.5 degrees Celsius, its weight in classifying it as a crack should be reduced." Finally, the system incorporates this correction method into the defect identification model as parameter adjustments or rule additions, thereby completing a continuous optimization of the model and enabling it to more accurately distinguish between real cracks and road surface textures in future detections.

[0030] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A lightweight deep learning-based real-time dynamic detection method for highway pavement defects, characterized in that, include: Multimodal sensors are deployed on the inspection vehicle to acquire visible light image data, infrared light image data, and laser point cloud data of the road surface, and lightweight data processing is performed to quickly locate suspected defect areas. Identify the distance data between the suspected disease area and the inspection vehicle, and allocate computational resources to the multimodal sensor based on the distance data; The video frame time period when each suspected disease area is discovered is obtained, and the trajectory of each suspected disease area is identified according to the video frame time period to obtain the actual length and development trajectory of each suspected disease area, and a unique ID is assigned to each suspected disease. A disease identification model is constructed to perform high-precision classification and measurement of the suspected disease areas, and the suspected disease areas are screened to obtain the target disease areas; Identify the geometric parameters of the disease in the target disease area, and assess the severity of the disease based on the geometric parameters to obtain disease risk parameters; Based on the visible light image data and the infrared light image data, the road surface material is obtained. Combining the road surface material and the disease risk parameters, a maintenance treatment decision is generated. Obtain the disease identification parameters for each target disease area to generate a self-learning training package, and perform self-training on the disease identification model based on the self-learning training package to continuously optimize the disease identification model.

2. The lightweight deep learning-based real-time dynamic detection method for highway pavement defects according to claim 1, characterized in that, The steps for acquiring visible light image data, infrared image data, and laser point cloud data of the road surface, and performing lightweight data processing to quickly locate suspected defect areas are as follows: Multimodal sensors are deployed on the inspection vehicle to acquire visible light image data, infrared light image data, and laser point cloud data of the road surface; Based on the visible light image data, region identification is performed on the visible light image data to obtain valid image regions and invalid image regions; Based on the failed image area, target area data of the same area in the infrared light image data and the laser point cloud data are extracted to supplement the failed image area, thereby obtaining a compensated image area. Based on the effective image area, combined with the infrared image data and the laser point cloud data, image recognition is performed to obtain the suspected disease area. Based on the compensated image area, the suspected disease area is obtained by performing perspective identification according to the infrared light image data and the laser point cloud data.

3. The lightweight deep learning-based real-time dynamic detection method for highway pavement defects according to claim 2, characterized in that, The steps of identifying the distance data between the suspected defect area and the inspection vehicle, and calculating resource allocation for the multimodal sensor based on the distance data, are as follows: The suspected diseased area is spatially located to obtain its spatial coordinate data; Based on the spatial coordinate data, the distance between the suspected disease area and the inspection vehicle is obtained; Based on the distance data, the suspected disease areas are classified into near-field, mid-field, and far-field areas. Based on the near-field region, the data accuracy of the visible light image data is improved, while the data accuracy of the infrared light image data and the laser point cloud data is reduced. Based on the aforementioned mid-field region, the accuracy of the visible light image data, the infrared light image data, and the laser point cloud data is kept constant according to preset standard values; Based on the far-field region, the data accuracy of the visible light image data is reduced, while the data accuracy of the infrared light image data and the laser point cloud data is improved.

4. The lightweight deep learning-based real-time dynamic detection method for highway pavement defects according to claim 3, characterized in that, The steps of performing trajectory identification on each suspected disease area based on the video frame time period to obtain the actual length and development trajectory of each suspected disease area, and assigning a unique ID to each suspected disease, are as follows: The video frame time period when each suspected diseased area is discovered is obtained, and the video frame time period is extended forward and backward by a preset number of video frames based on the discovery frame. During the video frame time period, the optical flow data of the suspected disease area and the current vehicle speed are extracted. Based on the current vehicle speed and the optical flow data, the motion trajectory of the suspected disease area is obtained. For each video frame following the discovery frame, identify the appearance features of the disease to obtain appearance feature change data; Based on the vehicle speed, the rate of change of observation angle is obtained. Based on the rate of change of observation angle and the appearance feature change data, the actual length and development trajectory of each suspected disease area are obtained. Determine whether the motion trajectory matches the appearance feature change data; If the motion trajectory is determined to match the appearance feature change data, a unique ID is assigned to the same suspected disease area spanning multiple frames.

5. The lightweight deep learning-based real-time dynamic detection method for highway pavement defects according to claim 4, characterized in that, The steps of constructing a disease identification model, performing high-precision classification and measurement of the suspected disease areas, and filtering the suspected disease areas to obtain the target disease areas are as follows: A disease identification model is constructed, and high-precision visible light images, high-precision infrared images, and high-precision point cloud data of the suspected disease areas are extracted based on the disease identification model. Based on the high-precision visible light image, high-precision infrared light image, and high-precision point cloud data, cross-validation of the suspected disease area is performed to obtain the multi-source fusion parameters of the suspected disease area. Based on the multi-source fusion parameters, the authenticity of the disease is evaluated to obtain the disease authenticity value of the suspected disease area; Determine whether the true severity value of the disease exceeds a preset true severity threshold. If the true severity value of the disease exceeds the true severity threshold, then mark the suspected disease area as the target disease area.

6. The lightweight deep learning-based real-time dynamic detection method for highway pavement defects according to claim 5, characterized in that, The steps of identifying the geometric parameters of the disease in the target disease area and assessing the severity of the disease based on these geometric parameters to obtain disease risk parameters are as follows: Based on the multi-source fusion parameters corresponding to the target disease area, geometric identification is performed on the multi-source fusion parameters to obtain disease geometric parameters, which include gap width, pit depth and area ratio. Based on the infrared image data, the target disease area is identified and deeply mined to obtain the associated hidden diseases and the depth of the diseases. By combining the associated hidden diseases, the disease depth, the disease geometric parameters, the actual length, and the development trajectory, the severity of the target disease area is assessed to obtain a disease severity value. Based on the disease geometric parameters and the infrared image data, the disease type is obtained. A risk assessment is then performed by combining the disease type and the disease severity value to obtain disease risk parameters.

7. A lightweight deep learning-based real-time dynamic detection method for highway pavement defects according to claim 6, characterized in that, Based on the visible light image data and the infrared light image data, the road surface material is obtained. Combining the road surface material with the damage risk parameters, the steps for generating maintenance and treatment decisions are as follows: Based on the visible light image data, a road surface image is obtained. Color and texture recognition are performed on the road surface image to obtain the color features and texture features of the road surface. The first material parameter is generated by combining the color features and the texture features. Based on the infrared light image data, a road surface temperature distribution image is obtained. Based on the road surface temperature distribution image, the road surface thickness is identified to obtain the road surface thickness distribution parameters. Based on the thickness distribution parameters, a second material parameter is generated. By combining the first material parameter and the second material parameter, the material of the road surface is identified to obtain the road surface material; Based on the road surface material, the material information and treatment process for road maintenance are obtained. According to the disease risk parameters, the quantity of materials, maintenance area and maintenance depth for road maintenance are obtained. Based on the material information, the processing technology, the quantity of material, the curing area, and the curing depth, a curing treatment decision is generated.

8. A lightweight deep learning-based real-time dynamic detection method for highway pavement defects according to claim 7, characterized in that, The steps of obtaining disease identification parameters for each target disease area to generate a self-learning training package, and self-training the disease identification model based on the self-learning training package, are as follows: Obtain the disease identification parameters for each target disease area, and then collect these disease identification parameters after standardized data preprocessing to generate a self-learning training package; The disease identification model learns by recognizing the training data in the self-learning training package and records the learning results after recognition and learning. The learning outcomes are compared with the training data, and the recognition errors generated during the comparison process are extracted. Based on the recognition error, record the erroneous recognition operations that generate errors during the training process, and generate error correction processing methods for the erroneous recognition operations; The error correction process is then incorporated into the disease identification model to optimize it.

Images