Expressway disease detection system and detection method based on ensemble learning

Abstract

The invention relates to the technical field of road detection and maintenance, in particular to an expressway disease detection system and method based on integrated learning, and the system comprises a data collection module, a model training module, a model deployment module, a real-time detection module, a result integration module and a user feedback module. Based on a deep learning detection model and a model integration method, the strong learning ability of the deep learning model is specifically utilized, and original data and enhanced data of multiple scenes and multiple time periods are used for training the model, so that the model has relatively high generalization performance and can keep stable detection performance during actual detection, and during actual deployment calculation, the detection efficiency is greatly improved. A plurality of reasoning models are deployed at the cloud simultaneously, multi-model parallel reasoning is realized by distributing reasoning tasks to different models, and finally multi-model reasoning results are summarized and integrated, so that the model reasoning efficiency is improved, the vulnerability of a single deep learning model is effectively slowed down, and the detection effect is more stable.

Description

Technical Field

[0001] The present invention relates to the technical field of road detection and maintenance, and in particular to a fast road disease detection system and a detection method based on ensemble learning. Background Art

[0002] Traditional road disease detection methods rely on manual inspections and regular maintenance, which is inefficient and unable to detect and warn in real time.

[0003] At present, the methods of road defect detection include vision-based detection, radar or ultrasonic detection, and automated detection technologies based on other sensors. Compared with other detection methods, vision-based detection has the advantages of low cost and high efficiency, but it also has disadvantages such as difficulty in developing the detection system, high requirements on the computing power of the detection equipment, and unstable detection results.

[0004] Existing vision-based detection systems are either based on traditional image processing methods and have low robustness; or the application of deep learning detection models requires high computing power, making them difficult to deploy or have high deployment costs. Summary of the invention

[0005] The purpose of the present invention is to provide a rapid road disease detection system and detection method based on ensemble learning, which improves the detection accuracy and stability through the integration of multiple deep learning models and non-maximum suppression algorithms, and realizes real-time, efficient and automated detection of road diseases.

[0006] To achieve the above object, the present invention provides a method for detecting fast road damage based on ensemble learning, comprising the following steps:

[0007] Step 1: Data collection and preprocessing;

[0008] Step 2: Integrate multiple deep learning models for training and optimization;

[0009] Step 3: Deploy the model;

[0010] Step 4: Real-time detection, using the non-maximum suppression algorithm to integrate the detection results of each model, and further optimize and screen the detection results;

[0011] Step 5: User feedback.

[0012] Optionally, during the execution of step 1, a collection strategy is selected through the data collection module to collect data covering different types of roads, different weather conditions and different time periods, and then the collected data is preprocessed by cropping, scaling, rotating and noise removal to generate training data sets and test data sets.

[0013] Optionally, the multiple deep learning models in step 2 are vertically combined with four models of different sizes and precisions of yolov8, s, n, m, and l, and horizontally combined with the first-stage fast detection yolo and the second-stage Faster R-CNN detection model, and the models are connected in a low-coupling manner.

[0014] Optionally, during the training and optimization process in step 2, a distributed training strategy is adopted, multiple GPUs or TPUs are used to accelerate the training process, and an early stopping strategy is used to prevent overfitting; the cross entropy loss function and L2 regularization are used to optimize the model, and learning rate decay and momentum optimization are used to improve training efficiency and model performance.

[0015] Optionally, the execution process of step 3 includes the following steps:

[0016] Step 3.1: Package the trained model into a Docker image and deploy it to the inference server in the cloud;

[0017] Step 3.2: Use load balancing technology to evenly distribute user requests to multiple inference servers;

[0018] Step 3.3: Register, manage, and update the model through a central server, providing version control and rollback mechanisms for the model.

[0019] Optionally, the execution process of step 4 includes three steps: user request processing, request distribution and parallel reasoning, and result integration and optimization. Specifically, the user uploads the road image to be detected through the front-end interface. After the central server receives the user request, it distributes the request to each inference server through the reverse proxy. The deep learning model on each inference server will simultaneously infer the input road image and generate preliminary candidate boxes and detection results respectively; finally, the result integration module will comprehensively process the detection results of each model through the non-maximum suppression algorithm to remove redundant candidate boxes and low-confidence detection results.

[0020] Optionally, during the execution of step 5, the user feedback module returns the test results to the user in the form of charts and text descriptions, and supports multiple view options. The user can choose different display methods to view the specific information of the test results.

[0021] The present invention also proposes a fast road disease detection system based on ensemble learning, which is used in the fast road disease detection method based on ensemble learning, and includes a data acquisition module, a model training module, a model deployment module, a real-time detection module, a result integration module and a user feedback module, and the modules are connected through a network to transmit and process data;

[0022] The data acquisition module is responsible for collecting and preprocessing road damage image data to generate training data sets and test data sets;

[0023] The model training module is responsible for the training and optimization of the deep learning model and generates multiple high-precision detection models.

[0024] The model deployment module is responsible for deploying the trained model to the inference server in the cloud and managing and updating the model.

[0025] The real-time detection module is responsible for receiving user requests, distributing detection tasks, performing parallel reasoning and result integration.

[0026] The result integration module is responsible for integrating and optimizing the detection results of each model and outputting the final detection result.

[0027] The user feedback module is responsible for feeding back the detection results to the user and providing user management and history query functions.

[0028] The present invention provides a fast road disease detection system and detection method based on integrated learning. The system includes a data acquisition module, a model training module, a model deployment module, a real-time detection module, a result integration module and a user feedback module. The method is based on a deep learning detection model and model integration, specifically utilizing the powerful learning ability of the deep learning model, using raw data and enhanced data of multiple scenes and multiple time periods to train the model, so that the model has higher generalization performance and can maintain stable detection performance during actual detection. During actual deployment and calculation, multiple inference models are deployed on the cloud at the same time, and multi-model parallel inference is realized by distributing inference tasks to different models. Finally, the multi-model inference results are summarized and integrated, thereby improving the model inference efficiency and effectively mitigating the vulnerability of a single deep learning model, so that the detection effect is more stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a schematic flow chart of the steps of a rapid road disease detection method based on ensemble learning of the present invention.

[0031] Figure 2 It is a flow chart of the Yolov8 algorithm used in a module of a rapid road disease detection method based on ensemble learning of the present invention.

[0032] Figure 3 It is a schematic diagram of an integrated learning algorithm framework of an expressway disease detection method based on integrated learning of the present invention.

[0033] Figure 4 It is a schematic diagram of the operation flow of a rapid road disease detection system based on ensemble learning of the present invention.

[0034] Figure 5 It is a functional module schematic diagram of a rapid road disease detection system based on ensemble learning of the present invention. DETAILED DESCRIPTION

[0035] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0036] See also Figure 1 The present invention proposes a method for detecting fast road defects based on ensemble learning, comprising the following steps:

[0037] S1: Data collection and preprocessing;

[0038] S2: Integrate multiple deep learning models for training and optimization;

[0039] S3: Deploy the model;

[0040] S4: Real-time detection, using the non-maximum suppression algorithm to integrate the detection results of each model, and further optimize and screen the detection results;

[0041] S5: User feedback.

[0042] The following is a further explanation based on the specific implementation steps:

[0043] The execution process of step S1 includes the following steps:

[0044] 1) Data collection equipment: Use sensors such as high-resolution cameras and lidar to collect road disease image data. Through different collection strategies, these devices can collect data under different weather conditions, different time periods and different road types to ensure the diversity and comprehensiveness of the data.

[0045] 2) Data preprocessing: Preprocess the collected data, including image cropping, scaling, denoising, etc. Use data enhancement techniques (such as rotation, flipping, brightness adjustment, etc.) to generate more diverse training data and improve the generalization ability of the model.

[0046] The preprocessed road crack data is divided into a training set and a test set to ensure the diversity and balance of the data set. Labeling tools such as LabelImg are used to label the diseased areas in the image to generate the label data required for training.

[0047] Step S2 includes model selection, model training and optimization processes:

[0048] Model selection: Choose the appropriate deep learning model architecture, including the latest object detection models such as YOLOv8's n, s, x models and Faster R-CNN. These models perform well in object detection tasks and can efficiently detect road damage.

[0049] The Yolov8 algorithm flow chart used by the model is as follows Figure 2 As shown, the algorithm is outlined as follows.

[0050] YOLOv8 (You Only Look Once version 8) is an improved real-time target detection algorithm. Its structure consists of three main parts: Backbone (backbone network), Neck (feature fusion module), and Head (detection head). Each part has its own specific function to achieve fast and accurate target detection.

[0051] 1. Backbone (CSPDarknet)

[0052] Backbone is responsible for extracting multi-scale features from the input image. YOLOv8 uses CSPDarknet (CrossStage Partial Darknet) as the backbone network, which is further divided into multiple stages:

[0053] Stem Layer: The initial convolutional module used to reduce the size of the input image from 640x640x3 to 320x320x64.

[0054] Stage 1: Contains a convolution module and a CSPLayer_2Conv to extract deeper features.

[0055] Stage 2: Through the combination of convolutional modules and CSPLayer_2Conv, the feature map size is further reduced and the deep features are enhanced.

[0056] Stage 3: The multi-scale information is further integrated through the SPPF (Spatial Pyramid Pooling Fast) module to generate a feature map of size 20x20x512xw.

[0057] The output size of the convolutional module is calculated as:

[0058]

[0059] Among them, Input Size: represents the size of the input feature map, that is, the size of the input data received by the convolutional layer (for example, the width or height of the image).

[0060] Kernel Size: Indicates the size of the convolution kernel (filter), usually the side length of a square matrix, such as 3x3, 5x5, etc.

[0061] Padding: It represents the number of zeros added around the input feature map. The purpose of padding is to control the output size or keep the input size unchanged.

[0062] Stride: Indicates the step size of the convolution kernel moving on the input feature map. A stride of 1 means that the convolution kernel moves 1 pixel each time, a stride of 2 means that it moves 2 pixels each time, and so on.

[0063] Output Size: Indicates the size of the feature map obtained after the convolution operation.

[0064] 2. Neck (YOLOv8PAFPN)

[0065] The Neck module uses an improved PAFPN (Path Aggregation Network), which is responsible for fusing features at different levels and providing them to the detection head for multi-scale target detection.

[0066] TopDown Layers: Upsample high-level semantic features layer by layer and concatenate them with feature maps from earlier layers.

[0067] BottomUp Layers: Downsampling is performed to generate larger-scale feature maps, allowing the network to process smaller objects.

[0068] 3.Head (YOLOv8HeadModule)

[0069] The Head module is responsible for converting the fused feature map into the final detection output, including the bounding box position, classification score, etc.

[0070] Decoupled Head: In order to improve detection accuracy, YOLOv8 introduces a decoupled head structure (DecoupledHead), which separates the bounding box regression and classification tasks.

[0071] Loss: A loss function based on the combination of CIOU+DFL and BCE (binary cross entropy) is used to handle regression and classification tasks respectively.

[0072] The expression of CIOU loss function is:

[0073]

[0074] Among them, b x , b y is the center point of the bounding box, c is the length of the diagonal, and v is a measure of the consistency of the aspect ratio of the bounding box.

[0075] YOLOv8 uses CSPDarknet as the backbone network and combines PAFPN with Decoupled Head and other technologies to achieve efficient and accurate target detection. This algorithm is not only optimized in terms of inference speed, but also significantly improves detection accuracy through multi-scale feature fusion and sophisticated loss function design, which is why this invention uses it.

[0076] Model training and optimization:

[0077] ① Use a pre-trained model for initialization to improve the initial performance of the model.

[0078] ② Set reasonable hyperparameters (such as learning rate, batch size, etc.) and adopt distributed training strategy for training.

[0079] ③ Use cross entropy loss function and L2 regularization and other techniques to optimize the model, and use methods such as learning rate decay and momentum optimization to improve training efficiency and model performance.

[0080] ④ Use sparse training for channel pruning and layer pruning to optimize the model structure, reduce the number of model parameters, and improve the model inference speed.

[0081] Specifically, we use the integrated prediction method to build a model library containing multiple models, and provide users with different reasoning paths to achieve the purpose of balancing reasoning performance and speed, and to make up for the different learning preferences of different models. The detection model library combines lightweight models and high-precision models. Vertically, it combines four models of different sizes and precisions of YOLOv8, s, n, m, and l. Horizontally, it combines the first-stage fast detection yolo and the second-stage high-precision FasterR-CNN detection model. In addition, the detection model is connected to the system platform in a low-coupling manner, which can easily add, delete, modify weights, and other operations on the detection model.

[0082] The execution process of step 3 is as follows:

[0083] 1) Cloud deployment: Encapsulate the trained model into a Docker image and deploy it to the cloud inference server. Containerization technology is used to ensure efficient operation and easy management of the model.

[0084] 2) Load balancing: Use load balancing technology to evenly distribute user detection requests to multiple inference servers to ensure high availability and response speed of the system.

[0085] 3) Model management: The central server is responsible for model registration, management, and updating, and provides model version control and rollback mechanisms to ensure system stability and reliability.

[0086] The present invention builds a backend prediction cluster of the system based on the idea of ​​microservices. Figure 3 As shown in the figure, each prediction model is independently deployed on multiple computing servers, and is registered and discovered on the central server. The central server exposes the detection interface to the user, and sets up a reverse proxy server to forward the user's detection request to the computing server where the model is located, and provides model management, registration and other basic non-computational model services. The backend computing cluster includes an inference server and a data server, where the inference server deploys one or more inference models. The inference model provides processing, single image inference, video inference and other services as a complete detection node. Different nodes of the same server are deployed and isolated using separate docker images. The central server and the computing server are connected through a lightweight http protocol, and dynamic management of nodes is achieved through a dynamic routing table.

[0087] The execution process of step S4 includes the following steps:

[0088] 1) User request processing: Users upload road images that need to be detected through the front-end interface, which can be a web application or a mobile application.

[0089] 2) Request distribution and parallel reasoning: After receiving the user request, the central server distributes the request to each reasoning server through the reverse proxy. The models on each server perform reasoning at the same time to generate preliminary detection results.

[0090] 3) Result integration and optimization: Use the non-maximum suppression algorithm to integrate the detection results of each model, remove redundant and low-confidence detection results, and ensure that the final output detection results have high accuracy and high reliability.

[0091] The specific process is as follows: the user uploads the road image to be detected through the front-end interface. After the central server receives the user's request, it distributes the request to each inference server through the reverse proxy. On each inference server, a variety of deep learning models including YOLO 5-8n-x series models, Faster R-CNN, DETR, etc. are deployed. These models will simultaneously infer the input road image and generate preliminary candidate boxes and detection results respectively. The generated candidate boxes may contain disease areas of different scales and categories. In order to improve the accuracy of detection, the result integration module will comprehensively process the detection results of each model through the non-maximum suppression (NMS) algorithm to remove redundant candidate boxes and low-confidence detection results. At this time, a weighted fusion strategy may also be used to weight the candidate boxes according to the confidence and detection effect of different models to generate the final detection result. The final optimization process may also include post-processing steps, such as applying morphological operations or region-based filtering to further improve the accuracy and robustness of the detection results.

[0092] In this step, a non-maximum suppression integrated learning scheme is used to combine the prediction results of multiple models as the final prediction output. When there are similar prediction results among multiple prediction results, the comprehensive prediction result will select the prediction result with the highest prediction score as the final output. When there are prediction results that are not included in each other among multiple prediction results, the comprehensive prediction result will take the prediction result with a higher prediction score and add it to the existing prediction set to achieve the complementarity of the prediction results. Therefore, under the same conditions, the prediction accuracy of the method of the present invention is higher than that of the scheme using a single prediction model, but the disadvantages are also obvious, that is, it brings greater deployment difficulty and greater computing load. However, the integrated prediction system platform in the present invention adopts a parallel deployment method of multiple models, and multiple models are inferred in parallel during reasoning, so that the above-mentioned shortcomings are greatly improved.

[0093] Step S5 includes the following process:

[0094] 1) Result display: The detection results of cracks and defects are returned to the user in the form of charts and text. The user can view the detection results and analyze the type and location of the defect through the front-end interface.

[0095] 2) User management: Provides a user management interface where users can view historical detection records, manage detection tasks, set detection parameters, etc., to improve user experience and system availability.

[0096] During the execution of step S5, the user feedback module not only returns the test results to the user in the form of intuitive charts and detailed text descriptions, but also supports multiple view options. Users can choose different display methods to view the specific information of the test results. In addition, the module provides comprehensive user management functions, allowing users to customize the display format and content of the test results and set personalized report templates. At the same time, the user feedback module also has a historical record query function. Users can access and browse previous test records at any time, compare test results in different time periods, generate trend analysis reports, and support one-click export of results, which is convenient for subsequent sharing and archiving.

[0097] Furthermore, the present invention also provides an expressway disease detection system based on ensemble learning, which is used in the expressway disease detection method based on ensemble learning, and includes a data acquisition module, a model training module, a model deployment module, a real-time detection module, a result integration module and a user feedback module, wherein the modules are connected through a network for data transmission and processing;

[0098] The data acquisition module is responsible for collecting and preprocessing road damage image data to generate training data sets and test data sets;

[0099] The model training module is responsible for the training and optimization of the deep learning model and generates multiple high-precision detection models.

[0100] The model deployment module is responsible for deploying the trained model to the inference server in the cloud and managing and updating the model.

[0101] The real-time detection module is responsible for receiving user requests, distributing detection tasks, performing parallel reasoning and result integration.

[0102] The result integration module is responsible for integrating and optimizing the detection results of each model and outputting the final detection result.

[0103] The user feedback module is responsible for feeding back the detection results to the user and providing user management and history query functions.

[0104] The operation process of the expressway disease detection system based on ensemble learning is as follows: Figure 4 The whole process from data collection to user feedback is described as shown in Figure 1. Figure 5 As shown in the figure, it includes various links such as data collection, preprocessing, model training, model deployment, real-time detection, result integration and user feedback.

[0105] In summary, the present invention has the following beneficial effects:

[0106] 1. High-precision detection and low false detection rate:

[0107] The present invention demonstrates high comprehensive test accuracy (mainly reflected in higher mAP50) in multiple complex test scenarios. By integrating multiple deep learning models (such as YOLOv8, Faster R-CNN, etc.), the detection accuracy and stability are effectively improved. The application of the non-maximum suppression algorithm (NMS) further reduces false detections and missed detections, ensuring that the system can maintain a low comprehensive false detection rate under different conditions. By aggregating the prediction results of different models, the patented solution can effectively balance the deviations of each model and improve the overall performance.

[0108] 2. Robustness and generalization ability:

[0109] Ensemble learning methods show stronger robustness and generalization ability when facing road damage of different types and conditions. The advantages and disadvantages of different models in different scenarios can complement each other. Ensemble learning can effectively reduce the false detection and missed detection that may occur in a single model. Especially for complex road damage detection tasks, ensemble learning can process images under different lighting, background and weather conditions, ensuring the stability and reliability of the model in practical applications.

[0110] 3. Efficient real-time detection and scalability:

[0111] The present invention adopts cloud deployment and load balancing technology to achieve efficient real-time detection and meet the processing requirements of large-scale concurrent requests. Through the powerful computing power of the cloud, the user's detection request can be quickly responded to, ensuring the low latency and high response speed of the system. The design of the model management system makes it easy to add, update and delete new models, ensuring that the system can be dynamically adjusted and expanded according to actual needs.

[0112] 4. Data enhancement and model optimization:

[0113] In data preprocessing, a variety of data enhancement techniques (such as rotation, cropping, scaling, noise addition, etc.) are used to improve the diversity of training data and the generalization ability of the model. In the model training process, the pre-trained model is used for initialization, and the model structure is optimized through sparse training and channel pruning technology, which reduces the number of model parameters and improves the model inference speed. These optimization techniques ensure the efficient operation of the model in actual deployment.

[0114] 5. User-friendly and widely used:

[0115] It provides intuitive display of test results and convenient user management interface, improves user experience and facilitates users to maintain and repair roads. The system design is targeted at actual application scenarios, making the method of the present invention more feasible than other deep learning applications, and provides a practical solution for the application of deep learning technology in the field of road disease detection.

[0116] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for detecting fast road defects based on ensemble learning, characterized in that: The following steps are involved: Step 1: Data collection and preprocessing; Step 2: Integrate multiple deep learning models for training and optimization; Step 3: Deploy the model; Step 4: Real-time detection, using the non-maximum suppression algorithm to integrate the detection results of each model, and further optimize and screen the detection results; Step 5: User feedback.

2. The method for detecting highway defects based on ensemble learning according to claim 1, characterized in that: During the execution of step 1, the data acquisition module selects a collection strategy to collect data covering different types of roads, different weather conditions and different time periods. The collected data is then preprocessed by cropping, scaling, rotating and removing noise to generate training data sets and test data sets.

3. The method for detecting defects on expressways based on ensemble learning as claimed in claim 2, characterized in that: In step 2, the various deep learning models are vertically combined with four models of different sizes and precisions, namely s, n, m, and l of yolov8, and horizontally combined with the first-stage fast detection yolo and the second-stage FasterR-CNN detection model. The models are connected in a low-coupling manner.

4. The method for detecting highway defects based on ensemble learning according to claim 3, characterized in that: In the training and optimization process of step 2, a distributed training strategy is adopted, multiple GPUs or TPUs are used to accelerate the training process, and an early stopping strategy is used to prevent overfitting. The cross entropy loss function and L2 regularization are used to optimize the model, and learning rate decay and momentum optimization are used to improve training efficiency and model performance.

5. The method for detecting defects on expressways based on ensemble learning according to claim 4, characterized in that: The execution process of step 3 includes the following steps: Step 3.1: Package the trained model into a Docker image and deploy it to the inference server in the cloud; Step 3.2: Use load balancing technology to evenly distribute user requests to multiple inference servers; Step 3.3: Register, manage, and update the model through a central server, providing version control and rollback mechanisms for the model.

6. The method for detecting defects on expressways based on ensemble learning according to claim 5, characterized in that: The execution process of step 4 includes three steps: user request processing, request distribution and parallel reasoning, and result integration and optimization. Specifically, the user uploads the road image to be detected through the front-end interface. After the central server receives the user request, it distributes the request to each inference server through the reverse proxy. The deep learning model on each inference server will simultaneously infer the input road image and generate preliminary candidate boxes and detection results respectively; finally, the result integration module will comprehensively process the detection results of each model through the non-maximum suppression algorithm to remove redundant candidate boxes and low-confidence detection results.

7. The method for detecting highway defects based on ensemble learning according to claim 6, characterized in that: During the execution of step 5, the user feedback module returns the test results to the user in the form of charts and text descriptions, and supports multiple view options. The user can choose different display methods to view the specific information of the test results.

8. A fast road disease detection system based on ensemble learning, used in the fast road disease detection method based on ensemble learning as claimed in claim 7, characterized in that: It includes data collection module, model training module, model deployment module, real-time detection module, result integration module and user feedback module. The modules are connected through the network for data transmission and processing. The data acquisition module is responsible for collecting and preprocessing road damage image data to generate training data sets and test data sets; The model training module is responsible for the training and optimization of the deep learning model and generates multiple high-precision detection models. The model deployment module is responsible for deploying the trained model to the inference server in the cloud and managing and updating the model. The real-time detection module is responsible for receiving user requests, distributing detection tasks, performing parallel reasoning and result integration. The result integration module is responsible for integrating and optimizing the detection results of each model and outputting the final detection result. The user feedback module is responsible for feeding back the detection results to the user and providing user management and history query functions.

Images