Highway maintenance decision management method based on multi-source data analysis

Abstract

The invention discloses an expressway maintenance decision management method based on multi-source data analysis, and the method comprises the following steps: 1, road region pre-segmentation and feature extraction: carrying out the preprocessing of an expressway monitoring video through employing a MobileNetV3 + DeepLabV3 model, carrying out the pre-segmentation of a road region, and only carrying out the feature extraction of the segmented road region; step 2, image splicing and complete image generation: splicing the segmented road area images; 3, highway maintenance demand analysis: designing a cascade network architecture, adopting a YOLOv8 model for detecting large-scale damage in the first layer, adopting a-Net + + model for finely segmenting micro-cracks in the second layer, adopting a Vision Transform model for analyzing the damage depth in the third layer, and combining the output of the three layers of networks to accurately evaluate the maintenance demand of the highway; 4, performing maintenance engineering analysis and duration prediction; 5, analyzing historical traffic flow; and step 6, maintenance time decision and influence evaluation.

Description

Technical Field

[0001] This invention relates to the field of highway maintenance decision-making technology, and in particular to a highway maintenance decision-making and management method based on multi-source data analysis. Background Technology

[0002] In the information age, the efficient operation of highways relies heavily on big data technology. Through refined management of maintenance work, big data analytics has become a key means to improve maintenance quality, extend facility lifespan, and reduce operating costs. Highway maintenance decision-making and management methods based on multi-source data analysis, through automated and intelligent data analysis techniques, can achieve rapid response and accurate identification of highway maintenance needs, thereby improving the efficiency of maintenance work.

[0003] In the maintenance needs analysis phase, existing technologies mainly rely on calculating the structural similarity between the road surface display area and traffic safety facility area between a complete highway image and the original complete highway image to determine maintenance needs. However, this method has limitations in capturing localized damage such as minute cracks and potholes. Therefore, a highway maintenance decision management method based on multi-source data analysis is proposed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a highway maintenance decision management method based on multi-source data analysis.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A highway maintenance decision-making and management method based on multi-source data analysis includes the following steps: Step 1: Road area pre-segmentation and feature extraction: The video surveillance of the highway is acquired through the highway's own video surveillance system. The video surveillance is preprocessed using the MobileNetV3+DeepLabV3 model to pre-segment the road area. Feature extraction is performed only on the key road areas that are segmented. Step 2: Image stitching and complete image generation: Based on the extracted feature points, the segmented road area images are stitched together to generate a complete highway image. During the stitching process, semantic segmentation constraints are applied. Step 3: Highway maintenance needs analysis: Design a cascaded network architecture. The first layer uses the YOLOv8 model to detect large-scale damage, the second layer uses the -Net++ model to finely segment micro-cracks, and the third layer uses the VisionTransformer model to analyze the damage depth. Combining the outputs of the three-layer network, the maintenance needs of the highway are accurately assessed. Step 4: Maintenance Project Analysis and Duration Prediction: Based on historical maintenance data and big data analysis, a maintenance project model is constructed to predict the duration required for different maintenance tasks. The predicted duration is adjusted by comprehensively considering weather, brightness, and the complexity and urgency of the maintenance tasks. Step 5: Historical Traffic Flow Analysis: Using the highway's own video surveillance system, locate and identify all vehicles on the highway requiring maintenance in each time period, calculate traffic flow, divide the day into multiple time periods, and analyze the changes in traffic flow in each time period. Step Six: Maintenance Time Decision and Impact Assessment: Define the construction impact index f, consider the lane closure ratio, construction period and detour route saturation, determine the weight of each factor using the AHP (Analog-Hybrid Analysis) method, and select the period with the least traffic impact for maintenance based on the construction impact index f and the historical traffic flow for each time period.

[0006] The above further includes: Furthermore, the specific steps for the road area pre-segmentation and feature extraction are as follows: Video surveillance acquisition: Real-time acquisition of video surveillance footage through video surveillance systems along the highway; Preprocessing: The acquired surveillance video is input into the MobileNetV3+DeepLabV3 model. First, the efficient convolutional neural network of MobileNetV3 is used to extract features from the video frames. Then, the road area is accurately segmented through the dilated convolution and conditional random field optimization of DeepLabV3. Segmentation results: After preprocessing, the MobileNetV3+DeepLabV3 model outputs the segmentation results of the road region; Key region extraction: Further extract key regions from the segmentation results; Feature point selection: In the segmented key road regions, the SIFT algorithm is used to extract feature points from the segmented road region image; Feature description: For each selected feature point, calculate its feature descriptor, which is extracted based on local binary patterns; Feature matching: Using feature descriptors, the FLANN algorithm is used to match feature points in the current frame with feature points in the previous frame or reference frame.

[0007] Furthermore, the specific steps for image stitching and complete image generation are as follows: Concatenation Transformation Matrix Calculation: After feature point matching is completed, the concatenation transformation matrix is ​​calculated to align feature points in different images. The RANSAC algorithm is used to estimate the concatenation transformation matrix. The RANSAC algorithm estimates the transformation matrix by randomly selecting a set of feature point pairs and calculating the error of all feature point pairs under this transformation matrix. Then, the transformation matrix with the smallest error is selected as the final concatenation transformation matrix. This concatenation transformation matrix is ​​used to transform one image into the coordinate system of another image. The concatenation transformation matrix is ​​represented as follows: ; in, These are the parameters of the affine transformation. These are the parameters of the translation transformation. A value of 0 or 1 indicates a scaling factor. Image stitching and fusion: After obtaining the stitching transformation matrix, one image is transformed into the coordinate system of another image and then stitched together. Let the two images to be stitched be and , and the mask for the overlapping region be M. Then the fused image is represented as . ; in, It is a weighting factor that is dynamically adjusted based on the pixel value differences in the overlapping areas; Semantic segmentation constraints: Semantic segmentation is performed on each image to obtain semantic information. Then, during the stitching process, the consistency of semantic information at the stitching point is checked. If the consistency is not consistent, the stitching transformation matrix is ​​adjusted or the image at the stitching point is corrected.

[0008] Furthermore, in the highway maintenance needs analysis, the specific steps for designing and training the cascaded network architecture are as follows: The first layer uses preprocessed image data to train the YOLOv8 model, enabling it to identify and locate large-scale damage on highways. The YOLOv8 model outputs detection boxes containing the location, type, and size of the damage. The second layer uses microcrack labeled data to train the U-Net++ model to segment the microcracks. The image region containing large-scale damage output by the YOLOv8 model is used as input, and the U-Net++ model outputs the fine segmentation result of the microcracks. The third layer uses a dataset containing damage depth annotations to train the Vision Transformer model, enabling it to analyze the damage depth based on the microcrack segmentation results and other relevant information. The microcrack segmentation results output by the U-Net++ model are used as input, and combined with meteorological data and maintenance records, the Vision Transformer model outputs the damage depth analysis results and the corresponding damage level assessment.

[0009] Furthermore, by combining the output of the three-layer network, a precise assessment of highway maintenance needs is conducted, including the following steps: The outputs of the YOLOv8 model, U-Net++ model, and Vision Transformer model are fused to form a comprehensive assessment of highway maintenance needs. The damage degree D is determined by the damage area A, damage depth h, and damage level L. The damage degree D is expressed as... ,in, , and These are the weighting coefficients; Based on the assessment results of maintenance needs, the damaged areas are prioritized to determine which areas require priority maintenance. A maintenance plan should be developed based on the type, location, and extent of damage to the affected area.

[0010] Furthermore, the maintenance project analysis and duration prediction includes the following steps: Data preparation: Collect and organize historical maintenance data, including maintenance task type, required duration, execution time, weather conditions, brightness, and the complexity and urgency of the maintenance task; Feature selection and processing: From the collected data, the following features were selected as input to the random forest regression model: Maintenance task types; Meteorological conditions; brightness; The complexity and urgency of the maintenance tasks; Building a Random Forest Regression Model: Using Python's scikit-learn library, build a random forest regression model; Adjusting the forecast duration: After obtaining the initial forecast duration, the forecast duration is adjusted by comprehensively considering the impact of weather, brightness, and the complexity and urgency of maintenance tasks on the forecast duration, and an adjustment coefficient is set for each factor; The final adjusted forecast duration is ,in, This is the adjusted duration. This indicates the preliminary estimated duration. This is expressed as a rainy day adjustment factor. This represents the brightness adjustment factor. This is expressed as an adjustment factor for complexity and urgency.

[0011] Furthermore, the specific steps of the historical traffic flow analysis are as follows: Time period division: Dividing a 24-hour day into several time periods; Vehicle location and identification: Using the highway video surveillance system, all vehicles on the road sections requiring maintenance are located and identified in real time during each time period; Traffic flow calculation: After completing the vehicle location and identification, calculate the traffic flow for each time period; Traffic flow change analysis: After obtaining the traffic flow data for each time period, I began to analyze the changes in traffic flow. By plotting the curve of traffic flow changes over time, I learned about the peak and off-peak periods of traffic flow, as well as the differences in traffic flow between different time periods.

[0012] Furthermore, the specific steps for the maintenance timing decision and impact assessment are as follows: Definition of Construction Impact Index f: The Construction Impact Index f is an indicator that comprehensively assesses the impact of construction on traffic. Its calculation formula is as follows: ; in, It is the construction impact index. It refers to the number of factors considered. The weight of the i-th factor is determined using the Analytic Hierarchy Process (AHP). It is the specific value of the i-th factor, which depends on the construction situation; Determining the weights of each factor: The Analytic Hierarchy Process (AHP) is a decision analysis tool used to determine the relative importance of each factor and construct a hierarchical model. This hierarchical model includes an objective layer, a criterion layer, and a solution layer. The objective layer minimizes the impact of construction on traffic. The criterion layer includes lane closure ratio, construction period, and detour route saturation. The solution layer includes different construction time and location options. For each criterion in the criterion layer, a judgment matrix is ​​constructed. This judgment matrix is ​​used to compare the relative importance of different factors under that criterion. For each judgment matrix, its weight vector is calculated, and then the weight of each factor is calculated. Calculate the construction impact index f: Calculate the construction impact index based on the weight of each factor; Select the period with the least impact on traffic for maintenance: After obtaining the construction impact index f, select the period with the least impact on traffic for maintenance based on the historical traffic flow of each time period.

[0013] The present invention has the following beneficial effects: 8. In this invention, a cascaded network architecture is designed. The first layer, YOLOv8, detects large-scale damage. The second layer, U-Net++, refines the segmentation of microcracks. The third layer, Vision Transformer, analyzes the damage depth, realizing full-scale damage identification from "meter level" to "centimeter level", and improving the microcrack detection rate.

[0014] 9. In this invention, MobileNetV3+DeepLabV3 is used to pre-segment the road area, and feature extraction and matching are performed only on key road areas to avoid background interference. Semantic constraints are used to reduce redundant feature points and improve splicing accuracy. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the steps of a highway maintenance decision management method based on multi-source data analysis proposed in this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] See also Figure 1 As shown, this invention is a highway maintenance decision management method based on multi-source data analysis, comprising the following steps: Step 1: Road Area Pre-segmentation and Feature Extraction: Acquire surveillance video through the highway's own video surveillance system, and preprocess the highway surveillance video using the MobileNetV3+DeepLabV3 model to pre-segment the road area (including road surface, markings, guardrails, etc.), and extract features only from the segmented key road areas; Step 2: Image stitching and complete image generation: Based on the extracted feature points, the segmented road area images are stitched together to generate a complete highway image. During the stitching process, semantic segmentation constraints are used to ensure the coherence and accuracy of the image content during stitching. Step 3: Highway Maintenance Needs Analysis: Design a cascaded network architecture. The first layer uses the YOLOv8 model to detect large-scale damage, such as potholes and cracks. The second layer uses the -Net++ model to finely segment micro-cracks and improve detection accuracy. The third layer uses the Vision Transformer model to analyze damage depth, achieving full-scale damage identification from "meter-level" to "centimeter-level". Combining the outputs of the three-layer network, the maintenance needs of the highway are accurately assessed. Step 4: Maintenance Project Analysis and Duration Prediction: Based on historical maintenance data and big data analysis, a maintenance project model is constructed to predict the duration required for different maintenance tasks. The predicted duration is adjusted by comprehensively considering weather, brightness, and the complexity and urgency of the maintenance tasks. Step 5: Historical Traffic Flow Analysis: Using the highway's own video surveillance system, locate and identify all vehicles on the highway requiring maintenance in each time period, calculate traffic flow, divide the day into multiple time periods, and analyze the changes in traffic flow in each time period. Step Six: Maintenance Time Decision and Impact Assessment: Define the construction impact index f, consider the lane closure ratio, construction period (peak / off-peak) and detour route saturation, determine the weight of each factor using the AHP (Analog-Hybrid Analysis) method, and select the period with the least traffic impact for maintenance based on the construction impact index f and the historical traffic flow for each time period.

[0018] In one embodiment, the specific steps of the road area pre-segmentation and feature extraction are as follows: Video surveillance acquisition: Real-time video surveillance is acquired through video surveillance systems along the highway. These videos are typically high-resolution and real-time, clearly capturing road conditions. Preprocessing: The acquired surveillance video is input into the MobileNetV3+DeepLabV3 model. First, the efficient convolutional neural network of MobileNetV3 is used to extract features from the video frames. Then, the road area is accurately segmented through the dilated convolution and conditional random field (CRF) optimization of DeepLabV3. Segmentation results: After preprocessing, the MobileNetV3+DeepLabV3 model outputs the segmentation results of the road region. The segmentation results are presented in the form of a binary image or a probability map, where the road region is marked as the foreground and other regions are marked as the background. Key region extraction: From the segmentation results, key regions such as road surface, markings, and guardrails are further extracted; Feature point selection: In the segmented key road regions, the SIFT algorithm is used to extract feature points from the segmented road region image. These feature points are road corners, edge points, texture feature points, etc. Feature description: For each selected feature point, calculate its feature descriptor, which is extracted based on Local Binary Pattern (LBP); Feature matching: Using feature descriptors, the FLANN algorithm is used to match feature points in the current frame with feature points in the previous frame or reference frame.

[0019] In one embodiment, the specific steps of image stitching and complete image generation are as follows: Image stitching transformation matrix calculation: After feature point matching is completed, the stitching transformation matrix is ​​calculated to align feature points in different images. The RANSAC (Random Sample Consensus) algorithm is used to estimate the stitching transformation matrix. RANSAC estimates the transformation matrix by randomly selecting a set of feature point pairs and calculating the error of all feature point pairs under this transformation matrix. Then, the transformation matrix with the smallest error is selected as the final stitching transformation matrix. This stitching transformation matrix is ​​used to transform one image into the coordinate system of another image, thereby achieving image stitching. The stitching transformation matrix is ​​represented as follows: ; in, These are the parameters of the affine transformation. These are the parameters of the translation transformation. The value is 0 (in homogeneous coordinates), and 1 is the scale factor; Image stitching and fusion: After obtaining the stitching transformation matrix, one image is transformed into the coordinate system of another image and then stitched together. Let the two images to be stitched be and , and the mask for the overlapping region be M. Then the fused image is represented as . ; in, It is a weighting factor that is dynamically adjusted based on the pixel value differences in the overlapping areas; Semantic segmentation constraints: Semantic segmentation is performed on each image to obtain semantic information such as road areas, markings, and guardrails. Then, during the stitching process, the consistency of semantic information at the stitching point is checked. If there is a discrepancy, the stitching transformation matrix is ​​adjusted or the image at the stitching point is corrected to ensure that the stitched image is semantically coherent.

[0020] In one embodiment, the specific steps for designing and training the cascaded network architecture in highway maintenance demand analysis are as follows: The first layer uses preprocessed image data to train the YOLOv8 model, enabling it to identify and locate large-scale damage on highways. The YOLOv8 model outputs detection boxes containing the location, type, and size of the damage. The second layer uses microcrack labeled data to train the U-Net++ model, enabling it to accurately segment microcracks. The image region containing large-scale damage output by the YOLOv8 model is used as input, and the U-Net++ model outputs fine segmentation results of microcracks, including information such as the location, shape and length of the cracks. The third layer uses a dataset containing damage depth annotations to train the Vision Transformer model, enabling it to accurately analyze the damage depth based on the microcrack segmentation results and other relevant information. The microcrack segmentation results output by the U-Net++ model are used as input, and combined with meteorological data (such as rainfall, temperature, etc.) and maintenance records (such as maintenance time, material type, etc.), the Vision Transformer model outputs the damage depth analysis results and the corresponding damage level assessment.

[0021] In one embodiment, the maintenance needs of highways are accurately assessed by combining the outputs of the three-layer network, including the following steps: The outputs of the YOLOv8 model, U-Net++ model, and Vision Transformer model are fused to form a comprehensive assessment of highway maintenance needs. The damage degree D is determined by the damage area A, damage depth h, and damage level L. The damage degree D is expressed as... ,in, , and The weighting coefficients are determined through methods such as expert scoring and data fitting. Based on the assessment results of maintenance needs, the damaged areas are prioritized to determine which areas require priority maintenance. Based on the type, location, and extent of damage to the damaged area, develop a suitable maintenance plan, including the selection of maintenance materials and the scheduling of maintenance time.

[0022] In one embodiment, the maintenance project analysis and duration prediction includes the following steps: Data preparation: Collect and organize historical maintenance data, including maintenance task type (such as crack repair, pothole filling, guardrail replacement, etc.), required duration, execution time, weather conditions (such as temperature, humidity, rainfall, etc.), brightness (which can be obtained through monitoring video or sensors), and the complexity and urgency of the maintenance task (which can be obtained through expert scoring or questionnaires, etc.). Feature selection and processing: From the collected data, the following features were selected as input to the random forest regression model: Maintenance task type (converted into numerical features through encoding); Meteorological conditions (such as temperature and humidity) can be processed through normalization or standardization. Brightness (also normalized); The complexity and urgency of the maintenance tasks (converted into numerical characteristics through expert scoring). Building a Random Forest Regression Model: Using Python's scikit-learn library, we build a random forest regression model. The specific steps are as follows: Python from sklearn.ensemble import RandomForestRegressor from sklearn.model_selection import train_test_split from sklearn.metrics import mean_squared_error import numpy as np # Assume X is the input feature matrix and y is the output duration vector. X = # Input feature matrix (including maintenance task type, weather conditions, brightness, complexity, and urgency) y = # Output duration vector # Split the dataset into training and test sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # Building a Random Forest Regression Model model = RandomForestRegressor(n_estimators=100, random_state=42) model.fit(X_train, y_train) # Predict test set duration y_pred = model.predict(X_test) # Calculate the mean squared error (MSE) mse = mean_squared_error(y_test, y_pred) print(f"Mean Squared Error: {mse}"); Adjusting the forecast duration: After obtaining the initial forecast duration, adjustments are made by comprehensively considering the impact of weather, brightness, and the complexity and urgency of the maintenance task on the forecast duration. For example, if a maintenance task is predicted to be carried out in rainy weather, additional waterproofing measures and drying time may be needed, thus extending the required duration. Similarly, insufficient brightness may require additional lighting equipment, which will also affect the duration. Furthermore, the complexity and urgency of the maintenance task will also affect resource allocation and work progress. An adjustment coefficient is set for each factor, for example:

[0023] Rainy day adjustment factor: 1.2 (assuming a 20% increase in duration on rainy days) Brightness adjustment factor: 1.1 (assuming that insufficient brightness requires an additional 10% of the duration) Complexity and urgency adjustment coefficient: determined based on expert scores (e.g., if the score range is 1-5, the adjustment coefficient can be 1+0.1*(score-3), that is, no adjustment is made when the score is 3, the duration is increased when the score is higher than 3, and the duration is decreased when the score is lower than 3). The final adjusted forecast duration is ,in, This is the adjusted duration. This indicates the preliminary estimated duration. This is expressed as a rainy day adjustment factor. This represents the brightness adjustment factor. This is expressed as an adjustment factor for complexity and urgency.

[0024] In one embodiment, the specific steps of the historical traffic flow analysis are as follows: Time Period Division: Divide the 24 hours of a day into several time periods, with the length of each time period adjusted according to actual needs. For example, to capture changes in traffic flow more precisely, choose one hour as a time period, or divide the time into unequal lengths based on peak and off-peak traffic periods;

[0025] Vehicle location and identification: Using the highway video surveillance system, all vehicles on the road sections requiring maintenance are located and identified in real time during each time period; Traffic flow calculation: After the vehicle location and identification are completed, the traffic flow in each time period is calculated. The traffic flow is obtained by counting the total number of vehicles passing through the maintenance section in each time period. This is achieved by analyzing the video footage frame by frame or by using the vehicle counting function provided by the video surveillance system. Traffic flow change analysis: After obtaining the traffic flow data for each time period, I began to analyze the changes in traffic flow. By plotting the curve of traffic flow changes over time, I learned about the peak and off-peak periods of traffic flow, as well as the differences in traffic flow between different time periods.

[0026] In one embodiment, the specific steps for the maintenance timing decision and impact assessment are as follows: Definition of Construction Impact Index f: The Construction Impact Index f is an indicator that comprehensively assesses the impact of construction on traffic. Its calculation formula is as follows: ; in, It is the construction impact index. It refers to the number of factors considered. The weight of the i-th factor is determined using the Analytic Hierarchy Process (AHP). It is the specific value of the i-th factor, which depends on the construction situation; In this example, consider the following three factors: Lane closure ratio ( ); Construction period (peak / off-peak) ); Detour path saturation ( ); Determining the weights of each factor: The Analytic Hierarchy Process (AHP) is a decision analysis tool used to determine the relative importance of each factor and construct a hierarchical model. This hierarchical model includes an objective layer, a criterion layer, and a solution layer. The objective layer minimizes the impact of construction on traffic. The criterion layer includes lane closure ratio, construction period, and detour route saturation. The solution layer includes different construction time and location options. For each criterion in the criterion layer, a judgment matrix is ​​constructed. This judgment matrix is ​​used to compare the relative importance of different factors under that criterion. The elements of the judgment matrix are usually obtained through expert scoring or questionnaires, representing the importance of one factor relative to another. For each judgment matrix, its weight vector is calculated, and then the weight of each factor is calculated. In this example, the following weights have been obtained through AHP analysis: Lane closure ratio weight ; Construction period weight ; Detour path saturation weight ; Calculate the construction impact index f: Calculate the construction impact index based on the weight of each factor; Assume the following specific construction conditions: The lane closure ratio is 20% (i.e.) ); The construction period coincides with peak hours (assuming the impact coefficient for peak hours is 1.5, i.e.) ); The detour path saturation is 80% (i.e.) ).

[0027] The calculation process for the construction impact index f is as follows: f = 0.5 × 0.2 + 0.3 × 1.5 + 0.2 × 0.8 f=0.71; Select the period with the least traffic impact for maintenance: After obtaining the construction impact index f, combine it with the historical traffic flow for each time period to select the period with the least traffic impact for maintenance. Assume the following historical traffic flow data (in hours): 00:00-06:00: Low traffic volume, averaging 100 vehicles per hour; 06:00-09:00: High traffic volume (peak hours), averaging 1000 vehicles per hour; 09:00-17:00: The average traffic volume is 500 vehicles per hour; 17:00-20:00: High traffic volume (evening rush hour), averaging 800 vehicles per hour; 20:00-24:00: Traffic flow gradually decreases, averaging 300 vehicles per hour.

[0028] Combining the construction impact index f and historical traffic flow data, maintenance is carried out during periods with relatively low traffic flow and a low construction impact index. In this example, since the traffic flow is lowest between 00:00 and 06:00, and the impact of construction on traffic is relatively small during this period (although the construction impact index f is not zero, it is relatively low), this period can be selected for maintenance.

[0029] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A highway maintenance decision-making and management method based on multi-source data analysis, characterized in that, Includes the following steps: Step 1: Road area pre-segmentation and feature extraction: The video surveillance system of the highway itself is used to acquire the surveillance video. The MobileNetV3+DeepLabV3 model is used to preprocess the highway surveillance video to pre-segment the road area. Feature extraction is performed only on the segmented road area. Step 2: Image stitching and complete image generation: Based on the extracted feature points, the segmented road area images are stitched together to generate a complete highway image. During the stitching process, semantic segmentation constraints are applied. Step 3: Highway maintenance needs analysis: Design a cascaded network architecture. The first layer uses the YOLOv8 model to detect large-scale damage, the second layer uses the -Net++ model to finely segment micro-cracks, and the third layer uses the VisionTransformer model to analyze the damage depth. Combining the outputs of the three-layer network, the maintenance needs of the highway are accurately assessed. Step 4: Maintenance Project Analysis and Duration Prediction: Based on historical maintenance data and big data analysis, a maintenance project model is constructed to predict the duration required for different maintenance tasks. The predicted duration is adjusted by comprehensively considering weather, brightness, and the complexity and urgency of the maintenance tasks. Step 5: Historical Traffic Flow Analysis: Using the highway's own video surveillance system, locate and identify all vehicles on the highway requiring maintenance in each time period, calculate traffic flow, divide the day into multiple time periods, and analyze the changes in traffic flow in each time period. Step Six: Maintenance Time Decision and Impact Assessment: Define the construction impact index f, consider the lane closure ratio, construction period and detour route saturation, determine the weight of each factor using the AHP (Analog-Hybrid Analysis) method, and select the period with the least traffic impact for maintenance based on the construction impact index f and the historical traffic flow for each time period.

2. The highway maintenance decision-making and management method based on multi-source data analysis according to claim 1, characterized in that, The specific steps of the road area pre-segmentation and feature extraction are as follows: Video surveillance acquisition: Real-time acquisition of video surveillance footage through video surveillance systems along the highway; Preprocessing: The acquired surveillance video is input into the MobileNetV3+DeepLabV3 model. First, the efficient convolutional neural network of MobileNetV3 is used to extract features from the video frames. Then, the road area is accurately segmented through the dilated convolution and conditional random field optimization of DeepLabV3. Segmentation results: After preprocessing, the MobileNetV3+DeepLabV3 model outputs the segmentation results of the road region; Key region extraction: Further extract key regions from the segmentation results; Feature point selection: In the segmented key road regions, the SIFT algorithm is used to extract feature points from the segmented road region image; Feature description: For each selected feature point, calculate its feature descriptor, which is extracted based on local binary patterns; Feature matching: Using feature descriptors, the FLANN algorithm is used to match feature points in the current frame with feature points in the previous frame or reference frame.

3. The highway maintenance decision-making and management method based on multi-source data analysis according to claim 1, characterized in that, The specific steps for image stitching and complete image generation are as follows: Concatenation Transformation Matrix Calculation: After feature point matching is completed, the concatenation transformation matrix is ​​calculated to align feature points in different images. The RANSAC algorithm is used to estimate the concatenation transformation matrix. The RANSAC algorithm estimates the transformation matrix by randomly selecting a set of feature point pairs and calculating the error of all feature point pairs under this transformation matrix. Then, the transformation matrix with the smallest error is selected as the final concatenation transformation matrix. This concatenation transformation matrix is ​​used to transform one image into the coordinate system of another image. The concatenation transformation matrix is ​​represented as follows: ； in, These are the parameters of the affine transformation. These are the parameters of the translation transformation. A value of 0 or 1 indicates a scaling factor. Image stitching and fusion: After obtaining the stitching transformation matrix, one image is transformed into the coordinate system of another image and then stitched together. Let the two images to be stitched be and , and the mask for the overlapping region be M. Then the fused image is represented as . ； in, It is a weighting factor that is dynamically adjusted based on the pixel value differences in the overlapping areas; Semantic segmentation constraints: Semantic segmentation is performed on each image to obtain semantic information. Then, during the stitching process, the consistency of semantic information at the stitching point is checked. If the consistency is not consistent, the stitching transformation matrix is ​​adjusted or the image at the stitching point is corrected.

4. The highway maintenance decision-making and management method based on multi-source data analysis according to claim 1, characterized in that, In the highway maintenance needs analysis, the specific steps for designing and training the cascaded network architecture are as follows: The first layer uses preprocessed image data to train the YOLOv8 model, enabling it to identify and locate large-scale damage on highways. The YOLOv8 model outputs detection boxes containing the location, type, and size of the damage. The second layer uses microcrack labeled data to train the U-Net++ model to segment the microcracks. The image region containing large-scale damage output by the YOLOv8 model is used as input, and the U-Net++ model outputs the fine segmentation result of the microcracks. The third layer uses a dataset containing damage depth annotations to train the Vision Transformer model, enabling it to analyze the damage depth based on the microcrack segmentation results and other relevant information. The microcrack segmentation results output by the U-Net++ model are used as input, and combined with meteorological data and maintenance records, the Vision Transformer model outputs the damage depth analysis results and the corresponding damage level assessment.

5. The highway maintenance decision-making and management method based on multi-source data analysis according to claim 1, characterized in that, By combining the outputs of the three-layer network, a precise assessment of highway maintenance needs is performed, including the following steps: The outputs of the YOLOv8 model, U-Net++ model, and Vision Transformer model are fused to form a comprehensive assessment of highway maintenance needs. The damage degree D is determined by the damage area A, damage depth h, and damage level L. The damage degree D is expressed as... ,in, , and These are the weighting coefficients; Based on the assessment results of maintenance needs, the damaged areas are prioritized to determine which areas require priority maintenance. A maintenance plan should be developed based on the type, location, and extent of damage to the affected area.

6. The highway maintenance decision-making and management method based on multi-source data analysis according to claim 1, characterized in that, The maintenance project analysis and duration prediction include the following steps: Data preparation: Collect and organize historical maintenance data, including maintenance task type, required duration, execution time, weather conditions, brightness, and the complexity and urgency of the maintenance task; Feature selection and processing: From the collected data, the following features were selected as input to the random forest regression model: Maintenance task types; Meteorological conditions; brightness; The complexity and urgency of the maintenance tasks; Building a Random Forest Regression Model: Using Python's scikit-learn library, build a random forest regression model; Adjusting the forecast duration: After obtaining the initial forecast duration, the forecast duration is adjusted by comprehensively considering the impact of weather, brightness, and the complexity and urgency of maintenance tasks on the forecast duration, and an adjustment coefficient is set for each factor; The final adjusted forecast duration is ,in, This is the adjusted duration. This indicates the preliminary estimated duration. This is expressed as a rainy day adjustment factor. This represents the brightness adjustment factor. This is expressed as an adjustment factor for complexity and urgency.

7. The highway maintenance decision-making and management method based on multi-source data analysis according to claim 1, characterized in that, The specific steps involved in the historical traffic flow analysis are as follows: Time period division: Dividing a 24-hour day into several time periods; Vehicle location and identification: Using the highway video surveillance system, all vehicles on the road sections requiring maintenance are located and identified in real time during each time period; Traffic flow calculation: After completing the vehicle location and identification, calculate the traffic flow for each time period; Traffic flow change analysis: After obtaining the traffic flow data for each time period, I began to analyze the changes in traffic flow. By plotting the curve of traffic flow changes over time, I learned about the peak and off-peak periods of traffic flow, as well as the differences in traffic flow between different time periods.

8. The highway maintenance decision-making and management method based on multi-source data analysis according to claim 1, characterized in that, The specific steps for determining the maintenance time and assessing its impact are as follows: Definition of Construction Impact Index f: The Construction Impact Index f is an indicator that comprehensively assesses the impact of construction on traffic. Its calculation formula is as follows: ； in, It is the construction impact index. It refers to the number of factors considered. The weight of the i-th factor is determined using the Analytic Hierarchy Process (AHP). It is the specific value of the i-th factor, which depends on the construction situation; Determining the weights of each factor: The Analytic Hierarchy Process (AHP) is a decision analysis tool used to determine the relative importance of each factor and construct a hierarchical model. This hierarchical model includes an objective layer, a criterion layer, and a solution layer. The objective layer minimizes the impact of construction on traffic. The criterion layer includes lane closure ratio, construction period, and detour route saturation. The solution layer includes different construction time and location options. For each criterion in the criterion layer, a judgment matrix is ​​constructed. This judgment matrix is ​​used to compare the relative importance of different factors under that criterion. For each judgment matrix, its weight vector is calculated, and then the weight of each factor is calculated. Calculate the construction impact index f: Calculate the construction impact index based on the weight of each factor; Select the period with the least impact on traffic for maintenance: After obtaining the construction impact index f, select the period with the least impact on traffic for maintenance based on the historical traffic flow of each time period.

Images