A road surface disease detection method and system based on environment perception
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2026-01-26
- Publication Date
- 2026-06-05
AI Technical Summary
Existing methods for detecting road surface defects lack accuracy and robustness in complex environments and cannot dynamically select or integrate the optimal detection model based on environmental conditions.
External meteorological data and road surface image features are acquired through the environmental perception module. The shared feature extraction network and multiple environmental expert detection heads in the multi-expert detection model are used to perform weighted fusion or gating selection by combining the probability distribution of environmental state and confidence level, and the final road surface defect detection results are output.
It maintains high detection accuracy under different environmental conditions, achieving robustness and practicality of the system. It has the ability to optimize performance in a closed loop, and its structure is flexible and highly scalable.
Smart Images
Figure CN122156953A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pavement distress detection technology, and in particular to a pavement distress detection method and system based on environmental perception. Background Technology
[0002] With the continuous development and progress of society, the transportation sector has achieved rapid development, with a rapid increase in road mileage and a continuous expansion of traffic volume. At the same time, road damage is inevitable, making road defect detection an increasingly heavy task.
[0003] Current methods for detecting road surface defects typically rely on a single deep learning model. In practical applications, the environments in which images are acquired vary greatly (e.g., sunny days, shadows, post-rain reflections), making it difficult for the same model to maintain high accuracy in all environments. Existing methods mainly improve generalization through data augmentation or training robust models, but still suffer from the following drawbacks: recognition rates drop significantly in strong shadows or water accumulation; enhanced generalization leads to compromised best performance in a single scene; and there is a lack of explicit modeling and adaptation to environmental conditions.
[0004] For example, the method described in publication CN118840322A reduces the number of parameters and improves the accuracy of small target detection by introducing depthwise separable convolution and attention mechanisms. However, this method does not consider the impact of environmental changes, resulting in a significant decrease in model performance in complex environments such as shadows or water accumulation. Similarly, the method proposed in publication CN118038283A utilizes radar imaging and historical reference information to enhance detection reliability, but it still relies on a fixed model strategy and cannot dynamically adjust the detection process according to environmental conditions, making it difficult to adapt to changing acquisition conditions.
[0005] Therefore, the core flaw of existing methods lies in their inability to dynamically select or fuse the optimal detection model based on environmental conditions, resulting in insufficient detection accuracy and robustness in complex environments. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a pavement distress detection method and system based on environmental perception, which can automatically select or fuse the optimal detection model according to the collected environment, thereby improving the accuracy and stability of pavement distress detection under different complex environments.
[0007] The objective of this invention can be achieved through the following technical solutions: The first aspect of this invention provides a method for detecting pavement defects based on environmental perception, comprising the following steps: S1. Obtain a road surface image with timestamps and location information; S2. Based on the timestamp and location information, acquire external meteorological data, and combine it with the environmental classification results obtained by environmental classification of the road surface image to generate an environmental state probability distribution; S3. Input the road surface image into the multi-expert detection model, extract image features through the shared feature extraction network in the multi-expert detection model, and input the image features into multiple environmental expert detection heads in the multi-expert detection model for processing to obtain the disease detection results and their confidence levels output by each environmental expert detection head; S4. Based on the probability distribution of the environmental state and the confidence level, perform weighted fusion or gating selection on the defect detection results output by multiple environmental expert detection heads, and output the final road defect detection result.
[0008] Furthermore, in S2, the external meteorological data includes at least one of light intensity, rainfall, cloud cover, and humidity.
[0009] Further, in S2, external meteorological data is acquired based on the timestamp and location information, and combined with the environmental classification results obtained from environmental classification of the road surface image, an environmental state probability distribution is generated. The specific process includes: S2.1 Based on the timestamp and location information, query the corresponding external meteorological data through the network meteorological interface; S2.2 Perform environmental feature analysis on the road surface image, the environmental feature analysis including at least one of the following: extracting the brightness histogram of the road surface image, calculating the image contrast statistical features, identifying the shadow area in the image and calculating the shadow area ratio, and analyzing the image reflection features; S2.3. Based on the environmental features obtained from the analysis in step S2.2, a pre-trained image environment classifier is used to determine the environmental category of the road surface image and output the environmental classification result. S2.4. The external meteorological data obtained in step S2.1 and the environmental classification results obtained in step S2.3 are fused and input into an environmental state probability prediction model to generate an environmental state probability distribution. The environmental state probability distribution represents the probability that the road surface image belongs to each preset environmental state category.
[0010] Furthermore, in S2.3, the pre-trained image environment classifier specifically includes the following pre-training process: collecting a large number of road surface images containing different environmental state labels as training datasets, extracting the environmental features of each road surface image to form a feature vector, using the corresponding environmental state labels as supervision signals, and using supervised learning methods to train the image environment classifier until it can accurately predict the environment category of the image based on the input environmental features.
[0011] Furthermore, in S2.4, the construction process of the environmental state probability prediction model includes: collecting a road surface image dataset labeled with real environmental states, and associating each image with the external meteorological data at the time of collection and the intermediate environmental classification results generated by the image environment classifier, thereby forming training samples; subsequently, using the fusion features of external meteorological data and environmental classification results as input, and the real environmental state labels as supervision signals, the probability prediction model is trained end-to-end through supervised learning methods, so that it learns to accurately output the probability distribution representing the probability of occurrence of various environmental states based on the input fusion features.
[0012] Furthermore, in S2.4, each preset environmental state category is specifically a normal environment, a shaded environment, and a post-rain environment. The normal environment corresponds to the conditions of uniform illumination, no significant shading, and dry road surface. The shaded environment corresponds to the conditions of insufficient illumination in some or all areas of the road surface due to obstruction by buildings, trees, or clouds. The post-rain environment corresponds to the conditions of the road surface having characteristics such as water accumulation or damp reflection after precipitation.
[0013] Furthermore, in S3, image features are extracted through a shared feature extraction network in the multi-expert detection model. The specific process includes: S3.1a. Preprocess the input road surface image, the preprocessing including image size normalization and pixel value standardization; S3.1b. Input the preprocessed road surface image into a shared feature extraction network, which is composed of multiple layers of interconnected convolutional layers, pooling layers, or Transformer coding blocks. S3.1c. The input image is transformed and abstracted layer by layer through the shared feature extraction network, and finally an image feature tensor containing high-level semantic information is output.
[0014] Furthermore, in S3.1b, the specific construction process of the shared feature extraction network includes: designing a network structure consisting of multiple layers of convolutional layers, pooling layers, or Transformer coding blocks, and pre-training it on a large general image dataset to learn a general feature extraction capability that can effectively represent the multi-level visual features of road surface images.
[0015] Further, in S3, the image features are input into multiple environmental expert detection heads in the multi-expert detection model for processing, to obtain the disease detection results and their confidence levels output by each environmental expert detection head. The specific process includes: S3.2a. The image feature tensor output by the shared feature extraction network is transformed through a shared feature adapter. The transformation includes feature dimension alignment and semantic adaptation to obtain adapted general features. S3.2b. The adapted general features are input in parallel to multiple environmental expert detection heads, wherein each environmental expert detection head corresponds to a preset environmental state and is trained on a specific environmental dataset for its corresponding environmental state. Furthermore, the correspondence between each environmental expert detection head and a preset environmental state (such as normal, shadow, after rain) is established and solidified during the configuration and training process in the model building phase. Specifically, when building a multi-expert detection model, an independent environmental expert detection head is assigned to each preset environmental state category, and this one-to-one correspondence is stored and maintained in the model parameters through a static mapping table or internal identifier. In the model training phase, each detection head is trained independently or jointly but with separate targets by using a specific environmental dataset labeled with the corresponding environmental state labels, so that its parameters learn the ability to process road surface image features under the corresponding environmental state. This correspondence is fixed after the model is deployed and used in the forward inference process.
[0016] S3.2c. Each environmental expert detection head independently processes the input general features. Each environmental expert detection head contains a classification branch for disease classification and a regression branch for locating disease areas, so as to simultaneously output disease category, location information, and confidence level representing the reliability of this detection.
[0017] Furthermore, based on the environmental state probability distribution and the confidence level, the pavement defect detection results output by multiple environmental expert detection heads are weighted and fused or gated to output the final pavement defect detection result. The specific process includes: S4.1 Obtain the environmental state probability distribution generated in step S2, and the disease detection results and their corresponding confidence levels output by each environmental expert detection head obtained in step S3; S4.2 For each environmental expert detection head, combine the probability value in the probability distribution of its corresponding environmental state with the confidence level output by the environmental expert detection head to calculate the combined weight of the environmental expert detection head. In step S4.2, the calculation rule for the combined weight is as follows: the probability value in the probability distribution of the environmental state corresponding to each environmental expert detection head is multiplied by the confidence level calculated by the environmental expert detection head for the output disease detection result to obtain the combined weight used to characterize the reliability of the detection head under the current comprehensive environment and the model's own judgment. When performing the weighted fusion operation in step S4.3, the specific weighted fusion algorithm is as follows: for the disease detection results output by each environmental expert detection head, the disease category probability vector contained therein is weighted and averaged according to its respective combined weight, and the boundary box coordinates of the disease area contained therein are weighted and fused according to their respective combined weights to generate the final disease detection result containing the unified category probability and location coordinates.
[0018] S4.3. Based on the combined weights of each environmental expert detection head calculated in step S4.2, perform weighted fusion or gating selection operations: If a weighted fusion operation is performed, the disease detection results output by each environmental expert detection head will be weighted and summed according to their respective combination weights to obtain the final disease detection result. If a gating selection operation is performed, the disease detection result output by the environmental expert detection head with the highest combination weight is selected as the final disease detection result.
[0019] A second aspect of the present invention provides a pavement defect detection system based on environmental perception, comprising: The data acquisition module is used to acquire road surface images with timestamps and location information; The environmental perception module is used to acquire external meteorological data based on the timestamp and location information, and combine the environmental classification results obtained by environmental classification of the road surface image to generate an environmental state probability distribution. The multi-expert detection module includes a shared feature extraction network and multiple environmental expert detection heads, used to extract features from the road surface image, and each environmental expert detection head outputs the defect detection results and their confidence levels. The fusion processing module is used to perform weighted fusion or gating selection on multiple pavement defect detection results based on the probability distribution of the environmental state and the confidence level, and output the final pavement defect detection result.
[0020] Furthermore, it also includes a model optimization module, which is used for: The system stores historical detection records, which include the road surface image, the real environment label corresponding to the road surface image, the final road surface defect detection result, and correction information fed back by humans or sensors. Using the historical detection records, the environment perception module and the environment expert detection head are periodically updated through an incremental learning method. When updating the environment expert detection head, the parameters of the shared feature extraction network are frozen, and training is performed only on newly added environment expert detection heads; and / or, Using the historical detection records, a student detection model is trained through a knowledge distillation method, wherein the outputs of multiple environmental expert detection heads in the multi-expert detection module serve as teacher signals for the knowledge distillation process.
[0021] Furthermore, the environmental perception module includes a network meteorological interface unit and an image analyzer, wherein: The network meteorological interface unit is used to query and obtain the external meteorological data based on the timestamp and location information; The image analyzer is used to perform environmental feature analysis on the road surface image and generate the environmental classification result based on the analysis result through a pre-trained classifier.
[0022] Furthermore, the data acquisition module includes an onboard industrial camera, a GPS module, and at least one environmental sensor, wherein: The vehicle-mounted industrial camera is used to acquire images of the road surface. The GPS module is used to generate the location information and combine it with the system time to generate the timestamp; The environmental sensors include light sensors, temperature and humidity sensors, or inertial measurement units, used to collect auxiliary environmental information; The multi-expert detection module includes a shared feature extraction network and at least three environmental expert detection heads, wherein: Expert inspection head A is used to detect road surface defects in images taken in normal environments; Expert inspection head B is used for defect detection on road surface images in shadowed environments; The expert inspection head C is used to detect road surface defects in post-rain environments using images. Each expert detection head is trained or optimized on the corresponding environmental dataset to obtain the best detection performance in a specific environment.
[0023] Compared with the prior art, the present invention has the following beneficial effects: This invention maintains high detection accuracy under different environmental conditions: by dynamically selecting or weighting multiple expert models through the environmental perception module, it effectively solves the problem of performance degradation of a single model in complex environments; by introducing a feedback mechanism of historical detection records, it realizes the periodic updating of the environmental perception module and the fusion strategy, and has the ability to optimize performance in a closed loop; in addition, this invention has a flexible structure and strong scalability, which facilitates the subsequent integration of more environmental expert models, thereby improving the overall robustness and practicality of the system. Attached Figure Description
[0024] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 This is a schematic diagram of the working structure of the environmental sensing module of the present invention; Figure 3 This is a schematic diagram of the multi-expert detection module structure of the present invention; Figure 4 This is a schematic diagram of the MLP fusion network structure of the present invention. Detailed Implementation
[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Component models, material names, connection structures, control methods, algorithms, and other features not explicitly described in this technical solution are considered common technical features disclosed in the prior art.
[0026] Example 1 This embodiment provides a pavement distress detection method based on environmental perception, including the following steps: S1. Obtain a road surface image with timestamps and location information; S2. Based on the timestamp and location information, acquire external meteorological data, and combine it with the environmental classification results obtained by environmental classification of the road surface image to generate an environmental state probability distribution; In specific implementation, in S2, the external meteorological data includes at least one of light intensity, rainfall, cloud cover, and humidity.
[0027] In specific implementation, in S2, external meteorological data is acquired based on the timestamp and location information, and combined with the environmental classification results obtained from environmental classification of the road surface image, an environmental state probability distribution is generated. The specific process includes: S2.1 Based on the timestamp and location information, query the corresponding external meteorological data through the network meteorological interface; S2.2 Perform environmental feature analysis on the road surface image, the environmental feature analysis including at least one of the following: extracting the brightness histogram of the road surface image, calculating the image contrast statistical features, identifying the shadow area in the image and calculating the shadow area ratio, and analyzing the image reflection features; S2.3. Based on the environmental features obtained from the analysis in step S2.2, a pre-trained image environment classifier is used to determine the environmental category of the road surface image and output the environmental classification result. S2.4. The external meteorological data obtained in step S2.1 and the environmental classification results obtained in step S2.3 are fused and input into an environmental state probability prediction model to generate an environmental state probability distribution. The environmental state probability distribution represents the probability that the road surface image belongs to each preset environmental state category.
[0028] In specific implementation, in S2.3, the pre-trained image environment classifier includes the following pre-training process: collecting a large number of road surface images containing different environmental state labels as training datasets, extracting the environmental features of each road surface image to form a feature vector, using the corresponding environmental state labels as supervision signals, and using supervised learning methods to train the image environment classifier until it can accurately predict the environment category of the image based on the input environmental features.
[0029] In specific implementation, in S2.4, the construction process of the environmental state probability prediction model includes: collecting a road surface image dataset with labeled real environmental states, and associating each image with the external meteorological data at the time of collection and the intermediate environmental classification results generated by the image environment classifier, thus forming training samples; subsequently, using the fusion features of the external meteorological data and the environmental classification results as input, and the real environmental state labels as supervision signals, the probability prediction model is trained end-to-end through supervised learning methods, so that it learns to accurately output the probability distribution representing the probability of occurrence of various environmental states based on the input fusion features.
[0030] In specific implementation, in S2.4, each preset environmental state category is specifically a normal environment, a shaded environment, and a post-rain environment. The normal environment corresponds to the condition of uniform lighting, no significant shading, and dry road surface. The shaded environment corresponds to the condition of insufficient lighting in some or all areas of the road surface due to obstruction by buildings, trees, or clouds. The post-rain environment corresponds to the condition of the road surface having characteristics such as water accumulation or damp reflection after precipitation.
[0031] The environmental classification results include environmental category determinations based on image brightness histograms, contrast statistics, shadow area proportions, and reflection features, which are used to guide the subsequent selection or weighting of multiple detection models.
[0032] S3. Input the road surface image into the multi-expert detection model, extract image features through the shared feature extraction network in the multi-expert detection model, and input the image features into multiple environmental expert detection heads in the multi-expert detection model for processing to obtain the disease detection results and their confidence levels output by each environmental expert detection head.
[0033] In specific implementation, in S3, image features are extracted through the shared feature extraction network in the multi-expert detection model. The specific process includes: S3.1a. Preprocess the input road surface image, the preprocessing including image size normalization and pixel value standardization; S3.1b. Input the preprocessed road surface image into a shared feature extraction network, which is composed of multiple layers of interconnected convolutional layers, pooling layers, or Transformer coding blocks. S3.1c. The input image is transformed and abstracted layer by layer through the shared feature extraction network, and finally an image feature tensor containing high-level semantic information is output.
[0034] In specific implementation, the construction process of the shared feature extraction network in S3.1b includes: designing a network structure composed of multiple layers of convolutional layers, pooling layers or Transformer coding blocks, and pre-training it on a large general image dataset to learn a general feature extraction capability that can effectively represent the multi-level visual features of road images.
[0035] In specific implementation, in S3, the image features are input into multiple environmental expert detection heads in the multi-expert detection model for processing, obtaining the disease detection results and their confidence levels output by each environmental expert detection head. (See [link to relevant documentation]). Figure 3 The specific process includes: S3.2a. The image feature tensor output by the shared feature extraction network is transformed through a shared feature adapter. The transformation includes feature dimension alignment and semantic adaptation to obtain adapted general features. S3.2b. The adapted general features are input in parallel to multiple environmental expert detection heads, wherein each environmental expert detection head corresponds to a preset environmental state and is trained on a specific environmental dataset for its corresponding environmental state. In practice, the correspondence between each environmental expert detection head and a preset environmental state (such as normal, shadow, after rain) is established and solidified during the configuration and training process in the model building phase. Specifically, when building a multi-expert detection model, an independent environmental expert detection head is assigned to each preset environmental state category, and this one-to-one correspondence is stored and maintained in the model parameters through a static mapping table or internal identifier. In the model training phase, each detection head is trained independently or jointly but with separate targets by using a specific environmental dataset labeled with the corresponding environmental state labels, so that its parameters learn the ability to process road surface image features under the corresponding environmental state. This correspondence is fixed after the model is deployed and used in the forward inference process.
[0036] S3.2c. Each environmental expert detection head independently processes the input general features. Each environmental expert detection head contains a classification branch for disease classification and a regression branch for locating disease areas, so as to simultaneously output disease category, location information, and confidence level representing the reliability of this detection.
[0037] In specific implementation, the shared feature extraction network serves as the backbone network of the multi-expert detection model and can be constructed using a convolutional neural network or a visual Transformer architecture; each environmental expert detection head is a classification-regression dual-branch detection head, which includes a classification branch for disease classification and a regression branch for locating disease areas, together constituting the dual-branch structure.
[0038] S4. Based on the probability distribution of the environmental state and the confidence level, perform weighted fusion or gating selection on the defect detection results output by multiple environmental expert detection heads, and output the final road defect detection result.
[0039] In specific implementation, based on the probability distribution of the environmental state and the confidence level, the pavement defect detection results output by multiple environmental expert detection heads are weighted and fused or gated to output the final pavement defect detection result. The specific process includes: S4.1 Obtain the environmental state probability distribution generated in step S2, and the disease detection results and their corresponding confidence levels output by each environmental expert detection head obtained in step S3; S4.2 For each environmental expert detection head, combine the probability value in the probability distribution of its corresponding environmental state with the confidence level output by the environmental expert detection head to calculate the combined weight of the environmental expert detection head. In step S4.2, the calculation rule for the combined weight is as follows: the probability value in the probability distribution of the environmental state corresponding to each environmental expert detection head is multiplied by the confidence level calculated by the environmental expert detection head for the output disease detection result to obtain the combined weight used to characterize the reliability of the detection head under the current comprehensive environment and the model's own judgment. When performing the weighted fusion operation in step S4.3, the specific weighted fusion algorithm is as follows: for the disease detection results output by each environmental expert detection head, the disease category probability vector contained therein is weighted and averaged according to its respective combined weight, and the boundary box coordinates of the disease area contained therein are weighted and fused according to their respective combined weights to generate the final disease detection result containing the unified category probability and location coordinates.
[0040] S4.3. Based on the combined weights of each environmental expert detection head calculated in step S4.2, perform weighted fusion or gating selection operations: If a weighted fusion operation is performed, the disease detection results output by each environmental expert detection head will be weighted and summed according to their respective combination weights to obtain the final disease detection result. If a gating selection operation is performed, the disease detection result output by the environmental expert detection head with the highest combination weight is selected as the final disease detection result.
[0041] The core of this embodiment is to construct a multi-expert collaborative decision-making system guided by environmental perception. First, the system synchronously acquires road surface images and their spatiotemporal context information, and integrates external meteorological data with the image's own visual features. A probabilistic prediction model comprehensively assesses the probability that the current environment belongs to various preset states, thus forming a quantified environmental state probability distribution. This distribution is not a simple classification but reflects the uncertainty under the interplay of multiple environmental factors. Second, a shared feature extraction network encodes the basic features of the input image. The general features extracted by this network are then distributed to multiple parallel environmental expert detection heads. Each expert detection head is specifically trained on a particular environmental dataset, enabling it to possess optimal disease identification capabilities in its corresponding environment and output detection results with its own confidence level. Finally, the decision-making mechanism uses the aforementioned environmental state probabilities as prior weights, combining them with the confidence levels output by each expert to calculate the combined confidence weight of each expert under the current comprehensive conditions. Based on this weight, the results of all experts are dynamically weighted and fused, or the result of the expert with the highest weight is directly adopted.
[0042] Example 2 This embodiment provides a road surface defect detection system based on environmental perception, such as... Figures 1 to 4 As shown, it includes a data acquisition module, an environmental perception module, a multi-expert detection module, and a fusion processing module, specifically: The data acquisition module is used to acquire road surface images with timestamps and location information; The environmental perception module is used to acquire external meteorological data based on the timestamp and location information, and combine the environmental classification results obtained by environmental classification of the road surface image to generate an environmental state probability distribution. The multi-expert detection module includes a shared feature extraction network and multiple environmental expert detection heads, used to extract features from the road surface image, and each environmental expert detection head outputs the defect detection results and their confidence levels. The fusion processing module is used to perform weighted fusion or gating selection on multiple pavement defect detection results based on the probability distribution of the environmental state and the confidence level, and output the final pavement defect detection result.
[0043] In specific implementation, the environmental perception module includes a network meteorological interface unit and an image analyzer, wherein: The network meteorological interface unit is used to query and obtain the external meteorological data based on the timestamp and location information; The image analyzer is used to perform environmental feature analysis on the road surface image and generate the environmental classification result based on the analysis result through a pre-trained classifier.
[0044] In specific implementation, the data acquisition module includes a vehicle-mounted industrial camera, a GPS module, and at least one environmental sensor, wherein: The vehicle-mounted industrial camera is used to acquire images of the road surface. The GPS module is used to generate the location information and combine it with the system time to generate the timestamp; The environmental sensors include light sensors, temperature and humidity sensors, or inertial measurement units, used to collect auxiliary environmental information; The environmental expert detection head is connected to the shared feature extraction network through an adapter structure. The adapter achieves the connection between the environmental expert detection head and the shared feature extraction network by performing feature dimension / semantic alignment and lightweight adaptation.
[0045] In specific implementation, the multi-expert detection module includes a shared feature extraction network and at least three environmental expert detection heads, wherein: Expert inspection head A is used to detect road surface defects in images taken in normal environments; Expert inspection head B is used for defect detection on road surface images in shadowed environments; The expert inspection head C is used to detect road surface defects in post-rain environments using images. Each expert detection head is trained or optimized on the corresponding environmental dataset to obtain the best detection performance in a specific environment.
[0046] In practical implementation, the road surface defect detection system based on environmental perception can periodically update the environmental perception module and fusion strategy based on historical detection records to achieve closed-loop performance optimization. The input image, environmental discrimination, detection results, and subsequent corrections (manual or sensor feedback) for each detection are stored in the database. The accuracy, false negative rate, and false positive rate of each model / fusion strategy are periodically calculated under different environments. The classifier is retrained or fine-tuned using the "image + actual environment label" from historical records, and the fusion weights are adjusted.
[0047] The core operating mechanism of the system provided in this embodiment lies in constructing an intelligent detection system that can dynamically adapt to the environment and continuously self-optimize through modular division of labor and closed-loop feedback. The entire system starts with data acquisition, with an onboard industrial camera, GPS, and various environmental sensors working collaboratively to simultaneously acquire a comprehensive data stream containing visual, spatiotemporal, and physical environmental information. The environmental perception module's internal network meteorological interface unit and image analyzer process data in parallel, acquiring macroscopic meteorological data and microscopic image features respectively. A probabilistic prediction model then fuses these two data, outputting a quantified probability distribution of the environmental state. This probability distribution is not a simple hard classification but a soft measure of the current complex environmental state. The multi-expert detection module is the system's specialized recognition engine. Its shared feature extraction network serves as a general feature calculation base, ensuring efficiency, while multiple environmental expert detection heads trained specifically for particular environments are connected via adapters, forming a flexibly expandable expert array. Each expert detection head works independently, outputting the defect detection results and confidence levels within its area of expertise for the same road surface image. The fusion processing module acts as an intelligent arbitrator. Based on the prior environmental probabilities provided by the environmental perception module and the confidence levels of the results from each expert detection head, it generates the final optimal detection result through weighted fusion or gating selection strategies. The system incorporates a closed-loop optimization mechanism based on historical detection records. The system continuously stores input data, intermediate discriminations, output results, and subsequent correction feedback into the database. By periodically analyzing performance indicators under different environments, it uses this data to retrain and adjust the weights of the classifier and fusion strategy in the environmental perception module. This enables the entire system to not only dynamically select the best recognition strategy for the current environment but also learn from historical experience, achieving periodic self-improvement and long-term stability of detection performance.
[0048] Example 3 Compared to Example 2, this example further includes a model optimization module, which is used for: The system stores historical detection records, which include the road surface image, the real environment label corresponding to the road surface image, the final road surface defect detection result, and correction information fed back by humans or sensors. Using the historical detection records, the environment perception module and the environment expert detection head are periodically updated through an incremental learning method. When updating the environment expert detection head, the parameters of the shared feature extraction network are frozen, and training is performed only on newly added environment expert detection heads; and / or, Using the historical detection records, a student detection model is trained through a knowledge distillation method, wherein the outputs of multiple environmental expert detection heads in the multi-expert detection module serve as teacher signals for the knowledge distillation process.
[0049] In specific implementation, the incremental learning and knowledge distillation methods for closed-loop optimization are as follows: S1. Incremental learning: When new environmental data is collected, the environmental perception module and expert detection head can be updated without forgetting the original knowledge. Specifically, the backbone network parameters can be frozen and only the newly added detection head can be trained, or regularization constraints and sample replay mechanisms can be used to avoid performance degradation in old scenes. S2. Knowledge Distillation: This method uses the detection results of multiple expert detection models as teacher signals to train student models, thereby reducing computational overhead while maintaining overall detection accuracy. Specifically, it can be performed using output distillation (mimicking the class probability distribution of expert models) or feature distillation (mimicking the intermediate feature representations of expert models). S3 combines incremental learning with knowledge distillation. This involves continuously introducing new environmental expert models through incremental learning and then integrating the knowledge from multiple expert models into a unified model through knowledge distillation, thereby achieving long-term performance improvement and efficient deployment.
[0050] The optimization mechanism in this embodiment is primarily achieved through two complementary technical paths. The first path is incremental learning, whose core is to expand the system's adaptability to new environments without compromising its existing capabilities. When the system encounters a new environmental pattern, this mechanism can seamlessly integrate new knowledge into the existing model by freezing shared feature extraction network parameters and training only the newly added expert detection heads, or by mixing old and new data for training and applying regularization constraints. The second path is knowledge distillation, whose core is to refine and compress the system's existing knowledge. It treats multiple professional environmental expert detection heads in the system as a powerful teacher committee, and by having a structured student model imitate the probability distribution or intermediate feature representations output by these teachers, it integrates the collective wisdom and expertise of multiple expert models into a more efficient unified model. Ultimately, these two paths work together to form a complete self-evolution strategy: incremental learning is responsible for vertically expanding the system's environmental adaptability by continuously adding new experts; while knowledge distillation is responsible for horizontally compressing and fusing the system's knowledge density, refining the capabilities of multiple experts into a single, efficient model. Through this periodically executed optimization loop, the system can not only continuously learn from real data after deployment, improving the overall detection accuracy and robustness in complex and ever-changing environments, but also ultimately produce a lightweight model that is more computationally efficient and easier to deploy.
[0051] The above description of the embodiments is provided to enable those skilled in the art to understand and use the invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the invention should be within the protection scope of the present invention.
Claims
1. A method for detecting pavement defects based on environmental perception, characterized in that, Includes the following steps: S1. Obtain a road surface image with timestamps and location information; S2. Based on the timestamp and location information, acquire external meteorological data, and combine it with the environmental classification results obtained by environmental classification of the road surface image to generate an environmental state probability distribution; S3. Input the road surface image into the multi-expert detection model, extract image features through the shared feature extraction network in the multi-expert detection model, and input the image features into multiple environmental expert detection heads in the multi-expert detection model for processing to obtain the disease detection results and their confidence levels output by each environmental expert detection head; S4. Based on the probability distribution of the environmental state and the confidence level, perform weighted fusion or gating selection on the defect detection results output by multiple environmental expert detection heads, and output the final road defect detection result.
2. The method for detecting pavement defects based on environmental perception according to claim 1, characterized in that, In S2, the external meteorological data includes at least one of light intensity, rainfall, cloud cover, and humidity.
3. The method for detecting pavement defects based on environmental perception according to claim 1, characterized in that, In S2, external meteorological data is acquired based on the timestamp and location information, and combined with the environmental classification results obtained from environmental classification of the road surface image, an environmental state probability distribution is generated. The specific process includes: S2.1 Based on the timestamp and location information, query the corresponding external meteorological data through the network meteorological interface; S2.2 Perform environmental feature analysis on the road surface image, the environmental feature analysis including at least one of the following: extracting the brightness histogram of the road surface image, calculating the image contrast statistical features, identifying the shadow area in the image and calculating the shadow area ratio, and analyzing the image reflection features; S2.
3. Based on the environmental features obtained from the analysis in step S2.2, a pre-trained image environment classifier is used to determine the environmental category of the road surface image and output the environmental classification result. S2.
4. The external meteorological data obtained in step S2.1 and the environmental classification results obtained in step S2.3 are fused and input into the environmental state probability prediction model to generate an environmental state probability distribution. The environmental state probability distribution represents the probability that the road surface image belongs to each preset environmental state category.
4. The method for detecting pavement defects based on environmental perception according to claim 1, characterized in that, In S3, image features are extracted through a shared feature extraction network in the multi-expert detection model. The specific process includes: S3.1a. Preprocess the input road surface image, the preprocessing including image size normalization and pixel value standardization; S3.1b. Input the preprocessed road surface image into a shared feature extraction network, which is composed of multiple layers of interconnected convolutional layers, pooling layers, or Transformer coding blocks. S3.1c. The input image is transformed and abstracted layer by layer through the shared feature extraction network, and finally an image feature tensor containing high-level semantic information is output.
5. The method for detecting pavement defects based on environmental perception according to claim 1, characterized in that, In S3, the image features are input into multiple environmental expert detection heads in the multi-expert detection model for processing, resulting in the disease detection results and confidence levels output by each environmental expert detection head. The specific process includes: S3.2a. The image feature tensor output by the shared feature extraction network is transformed through a shared feature adapter. The transformation includes feature dimension alignment and semantic adaptation to obtain adapted general features. S3.2b. The adapted general features are input in parallel to multiple environmental expert detection heads, wherein each environmental expert detection head corresponds to a preset environmental state and is trained on a specific environmental dataset for its corresponding environmental state. S3.2c. Each environmental expert detection head independently processes the input general features. Each environmental expert detection head contains a classification branch for disease classification and a regression branch for locating disease areas, so as to simultaneously output disease category, location information, and confidence level representing the reliability of this detection.
6. The method for detecting pavement defects based on environmental perception according to claim 1, characterized in that, Based on the probability distribution of the environmental state and the confidence level, the pavement defect detection results output by multiple environmental expert detection heads are weighted and fused or gated to output the final pavement defect detection result. The specific process includes: S4.1 Obtain the environmental state probability distribution generated in step S2, and the disease detection results and their corresponding confidence levels output by each environmental expert detection head obtained in step S3; S4.2 For each environmental expert detection head, combine the probability value in the probability distribution of its corresponding environmental state with the confidence level output by the environmental expert detection head to calculate the combined weight of the environmental expert detection head. S4.
3. Based on the combined weights of each environmental expert detection head calculated in step S4.2, perform weighted fusion or gating selection operations: If a weighted fusion operation is performed, the disease detection results output by each environmental expert detection head will be weighted and summed according to their respective combination weights to obtain the final disease detection result. If a gating selection operation is performed, the disease detection result output by the environmental expert detection head with the highest combination weight is selected as the final disease detection result.
7. A pavement defect detection system based on environmental perception, characterized in that, include: The data acquisition module is used to acquire road surface images with timestamps and location information; The environmental perception module is used to acquire external meteorological data based on the timestamp and location information, and combine the environmental classification results obtained by environmental classification of the road surface image to generate an environmental state probability distribution. The multi-expert detection module includes a shared feature extraction network and multiple environmental expert detection heads, used to extract features from the road surface image, and each environmental expert detection head outputs the defect detection results and their confidence levels. The fusion processing module is used to perform weighted fusion or gating selection on multiple pavement defect detection results based on the probability distribution of the environmental state and the confidence level, and output the final pavement defect detection result.
8. A pavement defect detection system based on environmental perception according to claim 7, characterized in that, It also includes a model optimization module, which is used for: The system stores historical detection records, which include the road surface image, the real environment label corresponding to the road surface image, the final road surface defect detection result, and correction information fed back by humans or sensors. Using the historical detection records, the environment perception module and the environment expert detection head are periodically updated through an incremental learning method. When updating the environment expert detection head, the parameters of the shared feature extraction network are frozen, and training is performed only on newly added environment expert detection heads; and / or, Using the historical detection records, a student detection model is trained through a knowledge distillation method, wherein the outputs of multiple environmental expert detection heads in the multi-expert detection module serve as teacher signals for the knowledge distillation process.
9. A pavement defect detection system based on environmental perception according to claim 7, characterized in that, The environmental perception module includes a network meteorological interface unit and an image analyzer, wherein: The network meteorological interface unit is used to query and obtain the external meteorological data based on the timestamp and location information; The image analyzer is used to perform environmental feature analysis on the road surface image and generate the environmental classification result based on the analysis result through a pre-trained classifier.
10. A pavement defect detection system based on environmental perception according to claim 7, characterized in that, The data acquisition module includes a vehicle-mounted industrial camera, a GPS module, and at least one environmental sensor, wherein: The vehicle-mounted industrial camera is used to acquire images of the road surface. The GPS module is used to generate the location information and combine it with the system time to generate the timestamp; The environmental sensors include light sensors, temperature and humidity sensors, or inertial measurement units, used to collect auxiliary environmental information; The multi-expert detection module includes a shared feature extraction network and at least three environmental expert detection heads, wherein: Expert inspection head A is used to detect road surface defects in images taken in normal environments; Expert inspection head B is used for defect detection on road surface images in shadowed environments; The expert inspection head C is used to detect road surface defects in post-rain environments using images. Each expert detection head is trained or optimized on the corresponding environmental dataset to obtain the best detection performance in a specific environment.