Road facility intelligent inspection and maintenance scheduling method based on deep learning

Abstract

The application provides a kind of road facility intelligent inspection and maintenance scheduling method based on deep learning, it is related to road facility maintenance technical field, including: obtaining multidimensional historical inspection data for damage identification;Collect historical maintenance records and classify, eliminate abnormal time points, determine the importance of each dimension time sequence and the optimal maintenance time in combination with maintenance cost;Based on damage information and optimal maintenance time, a multi-objective optimization problem is constructed, and a maintenance scheduling scheme is obtained by solving and iteratively optimizing the genetic algorithm, which can improve the efficiency of road facility maintenance, reduce maintenance cost and prolong the service life of the facility.

Description

Technical Field

[0001] This invention relates to the field of road infrastructure maintenance technology, and in particular to a method for intelligent inspection and maintenance scheduling of road infrastructure based on deep learning. Background Technology

[0002] With the continuous expansion of urban road networks and the increasingly serious problem of aging road facilities, the maintenance and management of road facilities has become an important part of urban management. Traditional road facility inspection and maintenance mainly rely on manual methods, including regular inspections, manual recording of problems and arranging repairs.

[0003] The maintenance and management of road facilities mainly relies on regular inspections and repair reporting systems. Relevant departments regularly dispatch staff for inspections or rely on public repair reports to identify problems. Repair work is then arranged according to budget and human resources. Some cities have also begun to use mobile devices to collect road image data and conduct preliminary damage identification through simple means. However, existing technologies still lack comprehensive utilization of multi-dimensional inspection data, resulting in insufficient accuracy in damage identification, failure to fully consider the development patterns and temporal characteristics of various types of damage, difficulty in determining the optimal repair time, often leading to premature repairs that waste resources or delayed repairs that cause safety hazards, and difficulty in balancing multiple objectives such as repair costs, service quality, and safety risks with limited resources. There is also a lack of adaptive optimization mechanisms based on historical data, making it impossible to continuously improve maintenance strategies based on actual repair results. Summary of the Invention

[0004] This invention provides a method for intelligent inspection and maintenance scheduling of road facilities based on deep learning, which can at least solve some of the problems existing in the prior art.

[0005] A first aspect of this invention provides a method for intelligent inspection and maintenance scheduling of road facilities based on deep learning, comprising:

[0006] Multi-dimensional historical inspection data is acquired and labeled to obtain road facility labeling data. Damage identification is performed on the road facility labeling data to obtain damage information.

[0007] Historical maintenance records are collected and classified according to the damage information. The maintenance time interval sequence corresponding to each type of damage is extracted. Abnormal maintenance time points in the maintenance time interval sequence are identified and removed to obtain a corrected maintenance time interval sequence. The maintenance time prediction features are determined by combining the pre-acquired maintenance cost data. The temporal importance of each dimension of the maintenance time prediction features is calculated and the optimal maintenance time for each type of damage is determined.

[0008] Based on the damage information and the optimal repair time, multi-objective constraints are determined and a multi-objective optimization problem is constructed. The multi-objective optimization problem is solved by a genetic algorithm to obtain an optimized repair scheme. The fitness of the optimized repair scheme is evaluated based on the historical repair records and iteratively optimized to obtain the optimal repair strategy and generate a maintenance scheduling scheme.

[0009] In one alternative implementation,

[0010] By acquiring multi-dimensional historical inspection data and labeling it to obtain road facility labeling data, and then performing damage identification on the road facility labeling data to obtain damage information, including:

[0011] Historical inspection data is obtained by combining the appearance image data, contour data and internal structure data corresponding to road inspection. Crack propagation features are extracted from the historical inspection data and fatigue damage features are obtained by calculating the cumulative damage degree. Road facility annotation data is obtained by annotating the data based on the fatigue damage features and the crack propagation features.

[0012] Extract the physical features and damage features corresponding to the road facility annotation data, calculate the category loss value based on the physical features and the damage features, determine the damage contribution of each feature at different scales based on the category loss value, and obtain the damage information by weighted fusion of the physical features and the damage features based on the damage contribution.

[0013] In one alternative implementation,

[0014] Based on the physical features and the damage features, a category loss value is calculated, and based on the category loss value, the damage contribution of each feature at different scales is determined, including:

[0015] The physical feature probability distribution is obtained by modeling the physical feature using a Gaussian mixture distribution, and the uncertainty corresponding to the physical feature probability distribution is calculated to obtain the first uncertainty;

[0016] The damage features are modeled by a pre-set Bayesian probabilistic network to obtain a damage feature probability distribution. The uncertainty corresponding to the damage feature probability distribution is calculated to obtain a second uncertainty. The first uncertainty and the second uncertainty are adaptively weighted and fused based on a pre-set cross-entropy loss to obtain a category loss value.

[0017] Based on the category loss value, a loss gradient matrix is ​​constructed and the sensitivity scores of the physical feature and the damage feature are calculated. The sensitivity scores are then input into a pre-set feature filtering network to obtain a feature importance vector.

[0018] Based on the feature importance vector, the probability distributions of the physical features and the probability distributions of the damage features are resampled to determine the confidence intervals, and the damage contribution corresponding to the physical features and the damage features is calculated.

[0019] In one alternative implementation,

[0020] Based on the damage information, the damage is classified, and a maintenance time interval sequence corresponding to each type of damage is extracted. Abnormal maintenance time points in the maintenance time interval sequence are identified and removed to obtain a corrected maintenance time interval sequence, including:

[0021] Based on the damage information, a damage classification standard is constructed. Historical maintenance information matching the damage classification standard is extracted from the historical maintenance records and classified to obtain initial damage types. Within each initial damage type, a preset damage threshold is used for filtering to obtain maintenance record categories. The time difference between adjacent maintenance times in each maintenance record category is extracted to obtain a maintenance time interval sequence.

[0022] Calculate the degree of fluctuation corresponding to the maintenance time interval sequence and initialize a dynamic time window based on the degree of fluctuation. Divide the maintenance time interval sequence into segments according to the dynamic time window and calculate the statistical characteristics and trend characteristics of each segment.

[0023] The statistical features and trend features are decomposed over time, and maintenance pattern indicators are constructed based on the changing patterns of the components obtained from the decomposition. The deviation between the current maintenance time interval sequence and the maintenance pattern indicators is calculated, and a dynamic threshold judgment criterion is constructed. Based on the dynamic threshold judgment criterion, abnormal maintenance time points in the maintenance time interval sequence are identified and removed to obtain a corrected maintenance time interval sequence.

[0024] In one alternative implementation,

[0025] Based on pre-acquired maintenance cost data, maintenance time prediction characteristics are determined. The temporal importance of each dimension of these characteristics is calculated, and the optimal maintenance time for each type of damage is determined, including:

[0026] Statistical features corresponding to the corrected maintenance time interval sequence are extracted and periodic features corresponding to the corrected maintenance time interval sequence are calculated by Fourier transform. The statistical features and periodic features are combined to obtain maintenance time prediction features. Maintenance benefit indicators are calculated based on the maintenance cost data and the corresponding performance parameter difference before and after maintenance.

[0027] Based on the maintenance efficiency index, the sensitivity score corresponding to the maintenance time prediction feature is calculated. The importance weight is initialized based on the sensitivity score. The temporal correlation corresponding to the maintenance time prediction feature is analyzed by a neural network to obtain the temporal importance of each dimension feature.

[0028] Principal component analysis is performed on the maintenance time prediction features to reduce dimensionality and obtain the main feature components. The influence of the main feature components on the maintenance benefit index is evaluated by the XGBoost algorithm. The comprehensive importance weight is determined based on the influence and the time series importance.

[0029] The maintenance benefit index is set as the optimization objective. The baseline maintenance time is obtained by iteratively solving the problem based on the particle swarm optimization algorithm, combined with the comprehensive importance weight and the pre-set maintenance cost upper limit. The type correction coefficient is initialized according to the pre-acquired damage type. The optimal maintenance time is calculated based on the type correction coefficient and the baseline maintenance time.

[0030] In one alternative implementation,

[0031] Based on the damage information and the optimal repair time, multi-objective constraints are determined and a multi-objective optimization problem is constructed. An optimized repair scheme is obtained by solving the multi-objective optimization problem using a genetic algorithm. The fitness of the optimized repair scheme is evaluated based on the historical repair records, and iterative optimization is performed to obtain the optimal repair strategy and generate a maintenance scheduling scheme, including:

[0032] Based on the damage information and the optimal repair time, a spatiotemporal constraint matrix is ​​constructed, and based on the spatiotemporal constraint matrix, a cost constraint threshold and a repair quality constraint threshold are calculated to obtain the multi-objective constraint conditions.

[0033] Extract the location information and damage type from the damage information, set up nodes and connect the nodes in combination with the pre-set initial maintenance plan to obtain a heterogeneous graph network. For the edges in the heterogeneous graph network, extract the edge feature vectors and construct the edge weight matrix through the attention mechanism, and fuse the edge feature vectors in combination with the message passing mechanism to obtain the fused feature vector.

[0034] The historical maintenance records are processed by a temporal graph convolutional network, self-connections are added to construct an adjacency matrix and a degree matrix is ​​calculated, and a temporal feature matrix is ​​calculated based on the degree matrix and the fused feature vector.

[0035] Decision variables are determined based on the time-series feature matrix and the multi-objective constraints, and the weight coefficients of the multi-objective constraints are set. A multi-objective optimization problem is constructed by combining the variables. The NSGA-III algorithm is executed to optimize the multi-objective optimization problem. Population selection is performed based on the reference point selection strategy, and the optimized maintenance scheme is obtained by combining the local search mechanism.

[0036] The fitness value of the optimized maintenance plan is calculated based on the historical maintenance records, and the parameters in the NSGA-III algorithm are updated based on the fitness value. The optimization is repeated until convergence, and the optimal maintenance strategy is obtained and a maintenance scheduling plan is generated.

[0037] In one alternative implementation,

[0038] The NSGA-III algorithm is applied to optimize the multi-objective optimization problem. Based on a reference point selection strategy, population selection is performed, and a local search mechanism is combined to obtain an optimized maintenance scheme, including:

[0039] Obtain the natural attenuation coefficients corresponding to different road facilities, and fit the damage growth curves according to the historical evolution trend of each damage type to calculate the development function values ​​and damage weight coefficients of various types of damage. Determine the road facility condition index threshold constraints and maintenance time constraints based on the development function values ​​and the damage weight coefficients.

[0040] Historical data on road facility damage is collected and performance degradation characteristics are calculated and determined. Based on the performance degradation characteristics, the damage accumulation law and performance recovery effect are analyzed, and life cycle analysis is performed to obtain life cycle analysis results. A set of reference points is generated based on the life cycle analysis results and preset balance parameters.

[0041] The road facility damage development rate is obtained, and the local search direction and search step size are determined based on the road facility damage development rate. The NSGA-III algorithm is used to optimize the multi-objective optimization problem by combining the road facility condition index threshold constraint and the maintenance time constraint. Population selection is performed based on the reference point set, and a local search mechanism is executed by combining the local search direction and the search step size to obtain the optimized maintenance scheme.

[0042] A second aspect of this invention provides a deep learning-based intelligent inspection and maintenance scheduling system for road facilities, comprising:

[0043] The first unit is used to acquire multi-dimensional historical inspection data and perform data annotation to obtain road facility annotation data, and to perform damage identification on the road facility annotation data to obtain damage information.

[0044] The second unit is used to collect historical maintenance records and classify them according to the damage information, extract the maintenance time interval sequence corresponding to each type of damage, determine the abnormal maintenance time points in the maintenance time interval sequence and remove them to obtain the corrected maintenance time interval sequence, combine the pre-acquired maintenance cost data to determine the maintenance time prediction features, calculate the temporal importance of each dimension of the maintenance time prediction features and determine the optimal maintenance time for each type of damage.

[0045] The third unit is used to determine multi-objective constraints and construct a multi-objective optimization problem based on the damage information and the optimal maintenance time. The multi-objective optimization problem is solved by a genetic algorithm to obtain an optimized maintenance scheme. The fitness of the optimized maintenance scheme is evaluated based on the historical maintenance records and iteratively optimized to obtain the optimal maintenance strategy and generate a maintenance scheduling scheme.

[0046] A third aspect of the present invention provides an electronic device, comprising:

[0047] A processor and a memory for storing processor-executable instructions, wherein the processor is configured to invoke instructions stored in the memory to perform the aforementioned method.

[0048] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0049] This invention achieves accurate identification and classification of road facility damage through intelligent processing and analysis of multi-dimensional historical inspection data, improving inspection efficiency and accuracy, reducing manual inspection costs, and enhancing the scientific and systematic nature of inspection work. By eliminating outliers and calculating feature importance in the maintenance time interval sequence, combined with maintenance cost data, the optimal maintenance time for various types of damage is scientifically determined, avoiding resource waste or safety hazards caused by premature or late maintenance. This achieves reasonable allocation and improved utilization efficiency of maintenance resources. Through evaluation and iterative optimization of historical maintenance records, the practicality and adaptability of the maintenance scheduling plan are ensured, providing scientific decision support for road facility management departments and effectively extending the service life of road facilities. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the intelligent road facility inspection and maintenance scheduling method based on deep learning, as described in an embodiment of the present invention.

[0051] Figure 2 This is a flowchart illustrating the maintenance time optimization logic of the intelligent road facility inspection and maintenance scheduling method based on deep learning, as described in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0054] Figure 1 This is a flowchart illustrating the intelligent road facility inspection and maintenance scheduling method based on deep learning, as described in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0055] Multi-dimensional historical inspection data is acquired and labeled to obtain road facility labeling data. Damage identification is performed on the road facility labeling data to obtain damage information.

[0056] Historical maintenance records are collected and classified according to the damage information. The maintenance time interval sequence corresponding to each type of damage is extracted. Abnormal maintenance time points in the maintenance time interval sequence are identified and removed to obtain a corrected maintenance time interval sequence. The maintenance time prediction features are determined by combining the pre-acquired maintenance cost data. The temporal importance of each dimension of the maintenance time prediction features is calculated and the optimal maintenance time for each type of damage is determined.

[0057] Based on the damage information and the optimal repair time, multi-objective constraints are determined and a multi-objective optimization problem is constructed. The multi-objective optimization problem is solved by a genetic algorithm to obtain an optimized repair scheme. The fitness of the optimized repair scheme is evaluated based on the historical repair records and iteratively optimized to obtain the optimal repair strategy and generate a maintenance scheduling scheme.

[0058] In one alternative implementation,

[0059] By acquiring multi-dimensional historical inspection data and labeling it to obtain road facility labeling data, and then performing damage identification on the road facility labeling data to obtain damage information, including:

[0060] Historical inspection data is obtained by combining the appearance image data, contour data and internal structure data corresponding to road inspection. Crack propagation features are extracted from the historical inspection data and fatigue damage features are obtained by calculating the cumulative damage degree. Road facility annotation data is obtained by annotating the data based on the fatigue damage features and the crack propagation features.

[0061] Extract the physical features and damage features corresponding to the road facility annotation data, calculate the category loss value based on the physical features and the damage features, determine the damage contribution of each feature at different scales based on the category loss value, and obtain the damage information by weighted fusion of the physical features and the damage features based on the damage contribution.

[0062] High-definition vehicle-mounted cameras are used to acquire surface images of road facilities, including visible light images of bridges, tunnels, guardrails, traffic signs, and drainage systems, with a resolution exceeding 4K. A 3D laser scanner is used to collect contour data of the road facilities, achieving millimeter-level accuracy. Internal structural data is collected using ground-penetrating radar to detect rebar distribution, internal cavities, and aquifers, reaching a depth of 50 centimeters. The acquired data is combined according to timestamps and geographic location information to form a historical inspection dataset. For example, for a 200-meter-long reinforced concrete bridge, 120 surface images, approximately 5 million points in the contour point cloud, and 45 sets of internal structural data slices are collected.

[0063] Historical inspection data was analyzed and processed to extract crack propagation features. The surface images were preprocessed, including image enhancement, noise removal, and geometric correction, and an adaptive threshold segmentation method was used to identify crack regions. Extracted crack features included attributes such as crack width, length, depth, orientation, and network morphology. For example, a typical diagonal crack on the bridge pier surface was recorded as having an average width of 3 mm, a length of 1.8 m, a maximum depth of 25 mm, an orientation angle of 43 degrees, and a linear crack morphology with branches.

[0064] Based on Miner's linear cumulative damage theory, and combining historical load data and material performance parameters, fatigue damage assessments are conducted for each facility component, taking into account factors such as vehicle load frequency, temperature variation cycles, and material aging. The cumulative damage is expressed as a percentage. For example, a bridge main beam with a cumulative damage of 42% indicates that the component has consumed 42% of its design service life. For metal guardrails, the cumulative damage may be even higher; for instance, a section of guardrail with a cumulative damage of 76% requires priority for repair or replacement.

[0065] The data annotation process is semi-automated, combining expert experience with computer-aided annotation tools. Road facilities are categorized and annotated based on crack propagation and fatigue damage characteristics, including damage types such as structural cracks, surface spalling, corrosion, and deformation; damage severity such as mild, moderate, and severe; and development stages such as initial, expanding, and stable. For example, the annotation data for a tunnel lining shows: damage type is water seepage crack, damage severity is moderate, development stage is expanding, cumulative damage is 61%, and the estimated remaining service life is 4.2 years.

[0066] This study analyzes labeled data of road facilities to extract physical and damage features. Physical features include the material of the facility (such as concrete, steel, aluminum alloy), structural type, thickness, and strength. Damage features include crack density, corrosion degree, deformation, and surface spalling area. Feature extraction employs a ResNet deep convolutional neural network, using a pre-trained model for transfer learning and fine-tuning for the road facility scenario. The network contains 50 convolutional layers and 3 fully connected layers, with standardized multimodal data as input and a 256-dimensional feature vector as output.

[0067] A multi-task loss function was constructed to calculate the category loss value, considering classification accuracy, location accuracy, and damage prediction bias. Model parameters were optimized through backpropagation. Different weight coefficients were set for different damage types and facility types to reflect their importance in facility safety. For example, a weight coefficient of 2.0 was set for main beam cracks affecting bridge structural safety; a weight coefficient of 1.6 was set for tunnel seepage damage; and a weight coefficient of 0.7 was set for paint peeling on guardrail surfaces. In practical applications, the category loss value of a bridge test sample was 0.28, indicating a high consistency between the model prediction and the actual annotations.

[0068] Feature importance analysis was employed to evaluate the contribution of each feature to damage identification and prediction at different scales. The contribution of each feature was quantified through feature ranking and ablation experiments, with values ​​ranging from 0 to 1. In a real-world case study, for structural cracks in concrete bridge piers, at the macroscopic scale, the contribution of concrete strength was 0.83, and the contribution of structural stress distribution was 0.91; at the microscopic scale, the contribution of crack edge sharpness was 0.88, and the contribution of depth gradient was 0.76. For corrosion of metal railings, the contribution of material electrochemical properties was 0.95, and the contribution of surface color change was 0.87.

[0069] A weighted fusion strategy is adopted to integrate features based on their contribution. Physical and damage features are normalized, weighted coefficients are assigned according to contribution, and a linear weighted combination is performed. The fused features more comprehensively reflect the condition of road facilities. For example, for a damage sample of a bridge bearing, the fused damage information shows: damage type is bearing rubber aging, severity is moderate, development trend is accelerated, recommended repair time window is within 45 days, and recommended repair solution is bearing replacement. For seepage points in the tunnel wall, the fused information shows that the seepage volume is moderate, affecting an area of ​​approximately 2 square meters, and recommends chemical grouting for sealing.

[0070] In this embodiment, by combining and processing appearance image data, contour data, and internal structure data, the surface and internal state of the road structure can be comprehensively reflected from different dimensions, significantly improving the completeness of data expression and the accuracy of feature extraction. By jointly calculating the crack propagation characteristics and cumulative damage degree in historical inspection data, the fatigue evolution law of the road can be more accurately depicted, improving the timeliness and predictive ability of damage identification. Based on physical features and damage features, the category loss value is calculated and a weighted fusion mechanism of multi-scale damage contribution is introduced, making the feature fusion result more consistent with the actual stress and damage distribution characteristics of road facilities.

[0071] In one alternative implementation,

[0072] Based on the physical features and the damage features, a category loss value is calculated, and based on the category loss value, the damage contribution of each feature at different scales is determined, including:

[0073] The physical feature probability distribution is obtained by modeling the physical feature using a Gaussian mixture distribution, and the uncertainty corresponding to the physical feature probability distribution is calculated to obtain the first uncertainty;

[0074] The damage features are modeled by a pre-set Bayesian probabilistic network to obtain a damage feature probability distribution. The uncertainty corresponding to the damage feature probability distribution is calculated to obtain a second uncertainty. The first uncertainty and the second uncertainty are adaptively weighted and fused based on a pre-set cross-entropy loss to obtain a category loss value.

[0075] Based on the category loss value, a loss gradient matrix is ​​constructed and the sensitivity scores of the physical feature and the damage feature are calculated. The sensitivity scores are then input into a pre-set feature filtering network to obtain a feature importance vector.

[0076] Based on the feature importance vector, the probability distributions of the physical features and the probability distributions of the damage features are resampled to determine the confidence intervals, and the damage contribution corresponding to the physical features and the damage features is calculated.

[0077] A Gaussian mixture model (GMM) is used to model the extracted physical features. For each type of road infrastructure, such as bridges, tunnels, and guardrails, a GMM model with multiple Gaussian kernels is constructed based on its material, structural type, and other physical parameters. The GMM model consists of a weighted sum of multiple single Gaussian distributions, each representing a possible physical state. Model parameters are estimated using the expectation-maximization algorithm and iteratively optimized until convergence. For example, for the physical features of a reinforced concrete bridge, including concrete strength, elastic modulus, and steel reinforcement density, a GMM model with five Gaussian kernels is constructed, with weights of 0.25, 0.2, 0.2, 0.15, and 0.2 for each kernel. The mean and covariance matrices are estimated from the training data.

[0078] Based on the obtained probability distribution of physical characteristics, the corresponding uncertainty is calculated to obtain the first uncertainty. The uncertainty calculation employs the information entropy method, quantifying uncertainty by calculating the dispersion of the probability distribution. A higher information entropy value indicates a higher level of uncertainty in the physical characteristics. For example, the information entropy of the probability distribution of a bridge main girder is 1.73, indicating a moderate degree of uncertainty in the physical properties of this component. In contrast, the information entropy of the probability distribution of a bridge bearing that already shows obvious signs of aging reaches 2.86, indicating a higher degree of uncertainty in its physical properties, requiring more careful evaluation.

[0079] Damage feature probabilistic modeling employs a pre-configured Bayesian neural network. This network consists of five layers, including three hidden layers, each containing 128, 256, and 128 neurons respectively. The network parameters are not fixed weights but random variables following a specific probability distribution, trained using variational inference methods. The network input consists of extracted damage features, such as crack density, corrosion degree, and deformation; the output is the probability distribution of these damage features. For example, for tunnel lining seepage problems, the input damage features include seepage point density, wetted area, and calcification degree; a probability distribution representing the degree of seepage damage can be obtained using a Bayesian network.

[0080] Based on the probability distribution of damage characteristics, the corresponding uncertainty is calculated to obtain the second uncertainty. The information entropy method is used for quantification to obtain a numerical representation of the uncertainty of the damage characteristics. For example, the information entropy of the probability distribution of a guardrail's corrosion damage is 1.25, indicating that the uncertainty of the corrosion state is relatively low and the assessment result is relatively reliable. However, for the tunnel water leakage problem, the information entropy of its damage characteristic probability distribution is 2.32, indicating that there is some uncertainty in the assessment of the water leakage damage state, and more detection methods may be needed to improve the accuracy of the assessment.

[0081] The first and second uncertainties are fused using a pre-set cross-entropy loss function to obtain a category loss value. The fusion process employs a dynamic weight allocation mechanism, adaptively adjusting the weights of physical features and damage features in the final fusion result based on their relative magnitudes. Specifically, when the uncertainty of physical features is low, their weight is increased; when the uncertainty of damage features is low, the weight of damage features is increased. For example, for a crack in a bridge main beam, the uncertainty of physical features is 1.56, and the uncertainty of damage features is 1.82. After adaptive weight fusion, the weight of physical features is 0.58, the weight of damage features is 0.42, and the final category loss value is 0.35.

[0082] Based on the calculated category loss value, gradients are calculated for the model parameters to form a loss gradient matrix. Specifically, the backpropagation algorithm is used to calculate the gradients of each parameter, and these gradients are then integrated into a gradient matrix through matrix operations. Based on the gradient matrix, sensitivity scores for physical features and damage features are calculated. The sensitivity score is measured by the absolute value of the gradient, reflecting the degree of influence of the feature on the final classification result. For example, for damage assessment of tunnel lining, the sensitivity score for concrete strength features is 0.82, while the sensitivity score for crack width features is 0.94, indicating that crack width features have a greater impact on the damage assessment result.

[0083] After the sensitivity scores are calculated, they are input into a pre-set feature selection network to obtain a feature importance vector. The feature selection network uses a fully connected neural network structure with two hidden layers, each containing 64 and 32 neurons respectively, and uses ReLU as the activation function. The network input is the sensitivity score of each feature, and the output is a standardized feature importance vector. Each element in the vector ranges from 0 to 1, representing the importance of the corresponding feature. For example, for damage assessment of bridge bearings, the resulting feature importance vector shows: bearing deformation importance is 0.95, rubber aging importance is 0.87, bearing surface crack importance is 0.76, and bearing corrosion importance is 0.68.

[0084] Based on feature importance vectors, the probability distributions of physical features and damage features are resampled to determine confidence intervals and calculate the damage contribution corresponding to the physical and damage features. The resampling process employs the Monte Carlo method, assigning sampling weights according to the feature importance vectors, with features of higher importance receiving more sampling opportunities. Through multiple resampling operations, the performance of each feature in different samples is statistically analyzed to determine its 95% confidence interval. The width of the confidence interval reflects the stability of the feature; a narrower interval indicates greater stability. For example, the 95% confidence interval for the concrete strength feature of a bridge main beam crack is [25.3 MPa, 28.7 MPa], with a relatively small interval width, indicating that the feature is relatively stable; while the 95% confidence interval for the crack depth feature is [12.5 mm, 28.2 mm], with a larger interval width, indicating that the feature has some uncertainty.

[0085] The contribution of each feature to road infrastructure damage is calculated based on feature importance and confidence intervals. The contribution calculation considers three aspects: feature importance, feature stability, and the degree of deviation of the feature value from typical damage modes. For example, for a guardrail damage case, the damage contribution of the metal corrosion feature is 0.85, the damage contribution of the deformation feature is 0.72, and the damage contribution of the surface coating feature is 0.65.

[0086] In this embodiment, a Gaussian mixture distribution is used to model physical features, which more realistically depicts the statistical distribution of continuous physical features and effectively reflects the multi-peak characteristics of road structures under different states, improving the fitting ability of physical feature modeling. A Bayesian network is used to probabilistically model damage features, which can maintain the consistency and interpretability of conditional probabilities among features even when there are complex dependencies between features, significantly enhancing the ability to express randomness and uncertainty. By adaptively weighting and fusing the uncertainty of physical features and damage features, the loss contribution can be dynamically adjusted according to the feature confidence, avoiding classification bias caused by excessive uncertainty of a single feature, and improving the stability and generalization ability of damage judgment. By constructing a loss gradient matrix and calculating feature sensitivity scores, the influence of different features on the classification results can be quantified. Combined with the feature importance vector obtained by the feature selection network, the interference of redundant information on model training is reduced.

[0087] In one alternative implementation,

[0088] Based on the damage information, the damage is classified, and a maintenance time interval sequence corresponding to each type of damage is extracted. Abnormal maintenance time points in the maintenance time interval sequence are identified and removed to obtain a corrected maintenance time interval sequence, including:

[0089] Based on the damage information, a damage classification standard is constructed. Historical maintenance information matching the damage classification standard is extracted from the historical maintenance records and classified to obtain initial damage types. Within each initial damage type, a preset damage threshold is used for filtering to obtain maintenance record categories. The time difference between adjacent maintenance times in each maintenance record category is extracted to obtain a maintenance time interval sequence.

[0090] Calculate the degree of fluctuation corresponding to the maintenance time interval sequence and initialize a dynamic time window based on the degree of fluctuation. Divide the maintenance time interval sequence into segments according to the dynamic time window and calculate the statistical characteristics and trend characteristics of each segment.

[0091] The statistical features and trend features are decomposed over time, and maintenance pattern indicators are constructed based on the changing patterns of the components obtained from the decomposition. The deviation between the current maintenance time interval sequence and the maintenance pattern indicators is calculated, and a dynamic threshold judgment criterion is constructed. Based on the dynamic threshold judgment criterion, abnormal maintenance time points in the maintenance time interval sequence are identified and removed to obtain a corrected maintenance time interval sequence.

[0092] Based on the acquired damage information, a damage classification standard is constructed, and a hierarchical clustering algorithm is used to divide the damage feature space into different categories. The damage classification standard includes three dimensions: damage type, damage severity, and damage location. Damage type covers major damage forms such as structural cracks, surface spalling, deformation, and corrosion; damage severity is divided into three levels: minor, moderate, and severe; damage location is further subdivided according to the structural characteristics of the road infrastructure. For example, for bridge infrastructure, the location can be divided into main beams, piers, bridge deck, and bearings; for tunnel infrastructure, the location can be divided into lining, entrances, and drainage systems. The combination of these three dimensions forms a complete damage classification standard system. For example, a typical damage classification standard is "Bridge main beam - moderate severity - structural crack".

[0093] Historical maintenance information matching the damage classification criteria is extracted from historical maintenance records and then categorized. These records include information such as maintenance time, maintenance content, maintenance cost, and maintenance results. Text matching and semantic analysis techniques are used to match the maintenance records with the damage classification criteria. The matching process employs the TF-IDF algorithm to calculate text similarity and combines it with a word embedding model to capture semantic information. Records with a matching score exceeding 0.75 are considered valid matches. After matching, the initial damage type classification results are obtained. For example, for the damage type "bridge main beam - structural crack," 127 relevant maintenance records were extracted from historical maintenance records.

[0094] Within each initial damage type, filtering is performed based on preset damage thresholds. These thresholds are set according to road infrastructure safety standards and expert experience, including crack width thresholds, deformation thresholds, and corrosion degree thresholds. For example, for structural cracks in bridge main beams, the crack width threshold is set to 2 mm and the length threshold to 500 mm; for guardrail deformation, the deformation threshold is set to 50 mm. Through threshold filtering, the initial damage type is further subdivided into different maintenance record categories. For instance, "bridge main beam - structural cracks" can be subdivided into three maintenance record categories: "minor cracks less than 1 mm wide," "moderate cracks 1-2 mm wide," and "severe cracks greater than 2 mm wide."

[0095] For each maintenance record category, the time difference between adjacent maintenance times is extracted to obtain a maintenance time interval sequence. The time difference is calculated in days, forming a numerical sequence representing the maintenance cycle. For example, the maintenance time interval sequence for a severe structural crack in the main beam of a bridge is [186, 192, 177, 215, 183, 354, 188, 176], in days, indicating that this type of damage usually requires maintenance approximately every six months. However, there is a significant outlier of 354 days in the sequence.

[0096] The volatility of the maintenance interval sequence is calculated and quantified using the coefficient of variation (COP), which is the ratio of the standard deviation to the mean. The COP reflects the relative dispersion of the sequence; greater volatility indicates a more unstable maintenance cycle, requiring a larger time window to capture maintenance patterns. Based on the calculated volatility, a dynamic time window is initialized. The time window size follows an adaptive principle: for every 0.1 increase in volatility, the window size increases by 10%. For example, for a bridge main girder crack maintenance interval sequence with a volatility of 0.32, the initial time window is set to 132% of the baseline window size, approximately 730 days.

[0097] The maintenance interval sequence is segmented according to a dynamic time window using a sliding window technique. The window size is determined based on the aforementioned calculations, and the sliding step size is 25% of the window size. Statistical and trend characteristics are calculated for each segment. Statistical characteristics include descriptive statistics such as mean, median, standard deviation, and interquartile range; trend characteristics include linear trend coefficient, periodicity intensity, and seasonality index. The calculated characteristics collectively describe the distribution characteristics and trends of the maintenance intervals. For example, for a segment of the maintenance intervals for a bridge bearing, the calculated statistical characteristics are: mean 182 days, median 185 days, standard deviation 15.6 days, and interquartile range 22 days; the trend characteristics are: linear trend coefficient -0.03, indicating a slight shortening trend in maintenance intervals during this stage; and periodicity intensity 0.68, indicating a significant periodicity.

[0098] The statistical and trend characteristics are decomposed over time using a seasonal trend decomposition method, dividing the time series into trend, seasonal, and residual components. The trend component reflects long-term trends, the seasonal component reflects periodic patterns, and the residual component contains random fluctuations and outlier information. Based on the variation patterns of each component obtained from the decomposition, a maintenance regularity index is constructed. This index comprehensively considers trend stability, periodic stability, and residual fluctuation, and is calculated using a weighted average. The weights are assigned as follows: trend stability 0.4, periodic stability 0.4, and residual fluctuation 0.2. For example, the maintenance time interval series of a tunnel drainage system, after time series decomposition, yields: a trend stability index of 0.82, a periodic stability index of 0.76, a residual fluctuation index of 0.68, and a comprehensive maintenance regularity index of 0.77, indicating that the facility's maintenance exhibits a relatively stable regularity.

[0099] The deviation between the current maintenance time interval sequence and maintenance pattern indicators is calculated using a dynamic time warping algorithm. A smaller deviation indicates that the current maintenance pattern conforms more closely to historical patterns; a larger deviation indicates potential anomalies in the current maintenance. Based on the calculated deviation, a dynamic threshold criterion is constructed. The dynamic threshold is adaptively adjusted according to the dispersion of historical data. For example, the dynamic threshold is set to the historical deviation mean plus 1.5 times the standard deviation. For instance, if the historical deviation mean of a guardrail maintenance record is 0.25 and the standard deviation is 0.08, then the dynamic threshold is 0.37.

[0100] Anomalies in the maintenance time interval sequence are identified based on a dynamic threshold criterion. Maintenance time points with deviations exceeding the dynamic threshold are marked as anomalies. Anomalies may represent emergency repairs, unplanned repairs, or data recording errors. By removing these anomalies, a corrected maintenance time interval sequence is obtained. For example, for the aforementioned maintenance time interval sequence for structural cracks in the main beam of the bridge [186, 192, 177, 215, 183, 354, 188, 176], the calculated deviations are [0.21, 0.18, 0.26, 0.31, 0.22, 0.89, 0.19, 0.24], respectively. The dynamic threshold is 0.42. The 6th maintenance time interval, 354 days, is identified as an anomaly and removed, resulting in the corrected sequence [186, 192, 177, 215, 183, 188, 176].

[0101] In this embodiment, by constructing damage classification criteria based on damage information and matching and filtering them with historical maintenance records, refined hierarchical management of maintenance records can be achieved, improving the scientificity and consistency of damage type identification and maintenance record classification. By dynamically adjusting the time window based on the degree of fluctuation, the time scale can be adaptively determined according to the maintenance frequency and fluctuation characteristics, effectively avoiding the time scale distortion problem caused by fixed windows, and more accurately reflecting the periodic and random characteristics of maintenance activities. By performing time-series decomposition of the statistical and trend characteristics of each time period, the long-term trend and short-term fluctuation in maintenance behavior can be effectively separated, and maintenance pattern indicators that characterize the changing patterns of maintenance can be constructed. By calculating the deviation between the current maintenance time interval sequence and the maintenance pattern indicators and introducing dynamic threshold judgment criteria, abnormal maintenance time points can be adaptively identified and eliminated, effectively eliminating interference caused by data anomalies or human intervention, and significantly improving the stability and reliability of maintenance time series analysis results.

[0102] In one alternative implementation,

[0103] Based on pre-acquired maintenance cost data, maintenance time prediction characteristics are determined. The temporal importance of each dimension of these characteristics is calculated, and the optimal maintenance time for each type of damage is determined, including:

[0104] Statistical features corresponding to the corrected maintenance time interval sequence are extracted and periodic features corresponding to the corrected maintenance time interval sequence are calculated by Fourier transform. The statistical features and periodic features are combined to obtain maintenance time prediction features. Maintenance benefit indicators are calculated based on the maintenance cost data and the corresponding performance parameter difference before and after maintenance.

[0105] Based on the maintenance efficiency index, the sensitivity score corresponding to the maintenance time prediction feature is calculated. The importance weight is initialized based on the sensitivity score. The temporal correlation corresponding to the maintenance time prediction feature is analyzed by a neural network to obtain the temporal importance of each dimension feature.

[0106] Principal component analysis is performed on the maintenance time prediction features to reduce dimensionality and obtain the main feature components. The influence of the main feature components on the maintenance benefit index is evaluated by the XGBoost algorithm. The comprehensive importance weight is determined based on the influence and the time series importance.

[0107] The maintenance benefit index is set as the optimization objective. The baseline maintenance time is obtained by iteratively solving the problem based on the particle swarm optimization algorithm, combined with the comprehensive importance weight and the pre-set maintenance cost upper limit. The type correction coefficient is initialized according to the pre-acquired damage type. The optimal maintenance time is calculated based on the type correction coefficient and the baseline maintenance time.

[0108] Statistical characteristics were extracted from the modified maintenance interval series, including mean, median, standard deviation, skewness, kurtosis, coefficient of variation, and interquartile range. For different types of road infrastructure, these statistical characteristics reflect the central tendency and fluctuations in maintenance patterns. For example, the modified maintenance interval series of a highway guardrail has the following statistical characteristics: mean 182 days, median 178 days, standard deviation 16.5 days, skewness 0.21, kurtosis 2.85, coefficient of variation 0.091, and interquartile range 23 days. These values ​​indicate that the maintenance cycle of this guardrail is relatively stable, concentrated around six months, with relatively small fluctuations.

[0109] The periodic characteristics of the maintenance interval sequence are calculated using Fourier transform. Fourier transform decomposes the time series into periodic components of different frequencies, and significant periodic patterns are identified by analyzing the spectrum. For road infrastructure maintenance, periodic characteristics typically include the primary period, secondary period, and seasonality index. The primary period represents the length of the most significant maintenance interval, the secondary period represents the length of the less significant period, and the seasonality index reflects the degree of influence of seasonal factors on the maintenance cycle. For example, after performing a Fourier transform on the maintenance interval sequence of a certain bridge bearing, the primary period was identified as 365 days, the secondary period as 182 days, and the seasonality index as 0.72. This indicates that the maintenance of this bridge bearing exhibits a clear annual cycle, with a semi-annual secondary cycle, and is significantly affected by seasonal factors.

[0110] By combining statistical and periodic features, a maintenance time prediction feature vector is formed. This feature vector contains 13 dimensions, corresponding to 7 statistical features and 6 periodic features (primary period, secondary period, seasonality index, harmonic intensity, phase consistency, and periodic stability). Each feature is standardized to have a mean of 0 and a standard deviation of 1 to eliminate the influence of dimensions. For example, the standardized maintenance time prediction feature vector for a tunnel drainage system is [-0.25, -0.18, 0.35, 0.12, -0.42, 0.38, 0.27, 0.95, 0.43, 0.68, 0.51, 0.33, 0.76].

[0111] The maintenance benefit index is calculated based on maintenance cost data and the corresponding difference in performance parameters before and after maintenance. The maintenance benefit index is defined as the ratio of performance improvement to maintenance cost. Performance improvement is measured by the difference in performance parameters before and after maintenance, including aspects such as structural strength, functional integrity, and service life of the facility. For example, for the maintenance of a bridge main beam, the structural strength score before maintenance was 72 points, and after maintenance it was 92 points, an improvement of 20 points; the maintenance cost was 45,000 yuan, then the maintenance benefit index is 20 / 45,000, or 4.44 × 10^-4 points / yuan. Maintenance benefit indices for different facilities can be standardized for comparison.

[0112] Sensitivity scores for maintenance time prediction features are calculated based on maintenance efficiency indicators. These scores are obtained through perturbation analysis, which involves applying small perturbations to each feature dimension and observing the degree of change in the maintenance efficiency indicators. The perturbation amplitude is set at 5% of the feature standard deviation. Perturbations are applied in both positive and negative directions for each feature, and the average rate of change of the maintenance efficiency indicators is calculated as the sensitivity score. For example, the sensitivity analysis results for guardrail maintenance time prediction features show that the sensitivity score for the mean feature is 0.82, the sensitivity score for the principal period feature is 0.95, and the sensitivity score for the coefficient of variation feature is 0.37, indicating that the principal period and mean features have a significant impact on maintenance efficiency.

[0113] Importance weights are initialized based on sensitivity scores. These importance weights are obtained through normalization of the sensitivity scores, ensuring the sum of all weights equals 1. The initial importance weight of each feature is equal to its sensitivity score divided by the sum of the sensitivity scores of all features. For example, the initial importance weight distribution for the maintenance feature of a road sign is: mean 0.15, median 0.12, standard deviation 0.08, major period 0.22, minor period 0.17, etc.

[0114] The temporal correlation of maintenance time prediction features is analyzed using a neural network to determine the temporal importance of each feature dimension. The neural network employs a Long Short-Term Memory (LSTM) structure, containing two LSTM layers (64 neurons each) followed by a fully connected layer. The network input is the temporal data of the maintenance time prediction features, and the output is the predicted maintenance benefit index. The temporal importance of each feature is determined by calculating its contribution to the network output. For example, the temporal importance analysis results for the maintenance features of a bridge expansion joint are as follows: principal period 0.28, mean 0.23, seasonality index 0.19, median 0.15, standard deviation 0.09, and other features totaling 0.06.

[0115] Principal component analysis (PCA) is used to reduce the dimensionality of maintenance time prediction features, yielding the main feature components. PCA retains principal components that explain more than 90% of the variance, typically reducing 13-dimensional features to 5-7 principal components. Each principal component is a linear combination of the original features, expressing the correlation structure between features. For example, PCA results for tunnel lining maintenance features show that the first 5 principal components explain 92.3% of the total variance. The first principal component is mainly composed of periodic features, the second principal component is mainly composed of central tendency features, and the third principal component is mainly composed of volatility features.

[0116] The influence of principal components on maintenance efficiency indicators was evaluated using the XGBoost algorithm. The XGBoost model contained 100 decision trees with a maximum depth of 5 and a learning rate of 0.05. Five-fold cross-validation was used to evaluate the model's performance. Feature importance analysis was performed to calculate the contribution of each principal component to the prediction, representing its influence. For example, the XGBoost analysis results for guardrail maintenance showed that the influence of the first principal component was 0.42, the second principal component was 0.28, the third principal component was 0.15, the fourth principal component was 0.10, and the fifth principal component was 0.05.

[0117] The overall importance weight is determined based on both temporal importance and degree of influence. The overall importance weight is calculated by a weighted average of temporal importance and degree of influence, with a weight ratio of 3:7. The overall importance weight reflects the combined importance of a feature in both the time-series pattern and its impact on benefits. For example, the overall importance weight of a traffic sign pole is: main period 0.32, mean 0.25, seasonality index 0.18, median 0.12, standard deviation 0.08, and other features totaling 0.05.

[0118] The maintenance benefit index is set as the optimization objective, and the solution is iteratively obtained using the particle swarm optimization algorithm combined with comprehensive importance weights and a pre-set maintenance cost ceiling. The particle swarm optimization algorithm parameters are set as follows: number of particles 50, maximum number of iterations 200, inertia weight 0.7, and individual and swarm learning factors both 1.5. The algorithm search space is the possible maintenance time range, and the time point that maximizes maintenance benefit is found through iterative optimization. For example, in the optimization process of a bridge main beam, 50 candidate maintenance time schemes are initially randomly generated. After 173 iterations, convergence is achieved, resulting in a baseline maintenance time of 168 days and an expected maintenance benefit index of 5.82 × 10^-4 fen / yuan, satisfying the constraint of a maintenance cost ceiling of 50,000 yuan.

[0119] The type correction factor is initialized based on the pre-acquired damage type. Different damage types have different sensitivities to repair time, which are adjusted using the type correction factor. The type correction factor is determined based on historical data analysis; for example, the correction factor for structural cracks is 0.85, for surface spalling it is 1.15, for deformation it is 0.92, and for corrosion it is 1.08. A correction factor less than 1 indicates that this type of damage needs to be repaired sooner, while a correction factor greater than 1 indicates that repair can be appropriately postponed.

[0120] The optimal maintenance time is calculated based on the type correction factor and the baseline maintenance time. The optimal maintenance time equals the baseline maintenance time multiplied by the type correction factor, and then rounded to obtain the actual executable maintenance time. For example, if a guardrail's damage type is surface corrosion, the type correction factor is 1.08, and the baseline maintenance time is 185 days, then the optimal maintenance time is 185 × 1.08 = 199.8 days, which is rounded to 200 days.

[0121] In this embodiment, by extracting statistical features from the corrected maintenance time interval sequence and combining them with Fourier transform to obtain periodic features, the stability and periodic regularity of maintenance behavior can be simultaneously characterized, significantly improving the completeness of the maintenance time sequence feature expression and the accuracy of periodic identification. By combining maintenance cost data and performance change data to construct maintenance benefit indicators, the dynamic balance between "cost input and performance recovery" can be comprehensively considered, enhancing the economic rationality and practical guiding value of maintenance decisions. Principal component analysis is used for dimensionality reduction, and the influence of the main feature components on the maintenance benefit indicators is evaluated using the XGBoost algorithm. This not only effectively reduces feature redundancy but also strengthens the ability to characterize complex nonlinear relationships. By comprehensively determining the comprehensive importance weight by combining the influence of features with the importance of the time sequence, the optimization process can be ensured to take into account both long-term trends and short-term fluctuations, thereby improving the robustness and adaptability of the prediction results. With the maintenance benefit indicators as the optimization target and introducing the particle swarm optimization algorithm, iterative solutions are performed under the constraints of comprehensive importance weight and maintenance cost upper limit, enabling optimization in the global scope to obtain a benchmark maintenance time that balances cost constraints and maximizes benefits.

[0122] Figure 2 This is a flowchart illustrating the maintenance time optimization logic of the intelligent road facility inspection and maintenance scheduling method based on deep learning, as described in an embodiment of the present invention.

[0123] In one alternative implementation,

[0124] Based on the damage information and the optimal repair time, multi-objective constraints are determined and a multi-objective optimization problem is constructed. An optimized repair scheme is obtained by solving the multi-objective optimization problem using a genetic algorithm. The fitness of the optimized repair scheme is evaluated based on the historical repair records, and iterative optimization is performed to obtain the optimal repair strategy and generate a maintenance scheduling scheme, including:

[0125] Based on the damage information and the optimal repair time, a spatiotemporal constraint matrix is ​​constructed, and based on the spatiotemporal constraint matrix, a cost constraint threshold and a repair quality constraint threshold are calculated to obtain the multi-objective constraint conditions.

[0126] Extract the location information and damage type from the damage information, set up nodes and connect the nodes in combination with the pre-set initial maintenance plan to obtain a heterogeneous graph network. For the edges in the heterogeneous graph network, extract the edge feature vectors and construct the edge weight matrix through the attention mechanism, and fuse the edge feature vectors in combination with the message passing mechanism to obtain the fused feature vector.

[0127] The historical maintenance records are processed by a temporal graph convolutional network, self-connections are added to construct an adjacency matrix and a degree matrix is ​​calculated, and a temporal feature matrix is ​​calculated based on the degree matrix and the fused feature vector.

[0128] Decision variables are determined based on the time-series feature matrix and the multi-objective constraints, and the weight coefficients of the multi-objective constraints are set. A multi-objective optimization problem is constructed by combining the variables. The NSGA-III algorithm is executed to optimize the multi-objective optimization problem. Population selection is performed based on the reference point selection strategy, and the optimized maintenance scheme is obtained by combining the local search mechanism.

[0129] The fitness value of the optimized maintenance plan is calculated based on the historical maintenance records, and the parameters in the NSGA-III algorithm are updated based on the fitness value. The optimization is repeated until convergence, and the optimal maintenance strategy is obtained and a maintenance scheduling plan is generated.

[0130] A spatiotemporal constraint matrix is ​​constructed based on damage information and optimal repair time. This matrix is ​​a two-dimensional array where rows represent damage points of different road facilities, and columns represent constraints in the time and spatial dimensions. Time-dimensional constraints include the earliest possible start time for repair, the latest required completion time, and the repair duration; spatial-dimensional constraints include geographical coordinates, traffic impact range, and construction space requirements. For example, for damage to a highway guardrail, the spatiotemporal constraint matrix elements include: the earliest possible start time for repair is May 15th, the latest required completion time is June 30th, the repair duration is 3 days, the geographical coordinates are latitude and longitude values, the traffic impact range is one lane in one direction, and the construction space requirement is a shoulder plus an emergency lane. By encoding the spatiotemporal constraints of different facilities, the spatiotemporal constraint matrix provides the basic constraints for subsequent optimization.

[0131] Cost constraint thresholds and maintenance quality constraint thresholds are calculated based on a spatiotemporal constraint matrix. The cost constraint thresholds are determined according to the maintenance budget and historical cost data, including upper limits for material costs, labor costs, equipment costs, and total costs. The maintenance quality constraint thresholds are determined based on facility safety standards and performance requirements, including post-repair strength requirements, service life requirements, and functional recovery requirements. These two types of thresholds together constitute multi-objective constraints. For example, the cost constraint threshold for a bridge expansion joint is set at a total cost not exceeding 38,000 yuan; the maintenance quality constraint thresholds are set at a post-repair strength not less than 95% of the original design strength, a service life of not less than 5 years, and a functional recovery rate of not less than 90%.

[0132] The location and damage types are extracted from the damage information, and nodes are set up in conjunction with a pre-set initial maintenance plan. Node types include damage point nodes, maintenance resource nodes, and time nodes. Damage point nodes represent the specific location of damage to road facilities, with attributes including damage type, severity, and location coordinates. Maintenance resource nodes represent available maintenance teams, equipment, and materials, with attributes including resource type, capacity parameters, and available time. Time nodes represent time window limitations, with attributes including start and end times and duration limits. For example, for a road facility maintenance plan within a region, a total of 127 damage point nodes, 15 maintenance resource nodes, and 35 time nodes are set up.

[0133] Connecting the nodes yields a heterogeneous graph network. The edges in the graph represent relationships between different types of nodes, including spatially related edges, resource-matching edges, and time-dependent edges. Spatially related edges connect damaged nodes that are geographically close, with edge weights inversely proportional to distance. Resource-matching edges connect damaged nodes to suitable maintenance resource nodes, with edge weights representing resource suitability. Time-dependent edges connect damaged nodes to time nodes, with edge weights representing the strictness of time constraints. The heterogeneous graph network encompasses all constraints and optimization space of the maintenance scheduling problem.

[0134] For edges in heterogeneous graph networks, an attention mechanism is used to extract edge feature vectors. This mechanism employs a multi-head self-attention structure with eight attention heads, each with a dimension of 64, collectively capturing the complex interactions between edges and connected nodes. The edge feature vector contains a combined representation of edge type encoding, weight values, and endpoint node features. An edge weight matrix is ​​constructed, where matrix elements represent the importance weights of edges, and softmax normalization is applied to ensure the sum of the weights equals 1. For example, for an edge from a damage point to a repair resource, the feature vector calculated by the attention mechanism has a dimension of 512 and a weight value of 0.73, indicating that the resource has high adaptability for processing this damage point.

[0135] A fused feature vector is obtained by fusing edge feature vectors using a message passing mechanism. The message passing process includes message generation, message aggregation, and feature updating. The message generation step combines source node features with edge features to form the message to be passed. The message aggregation step performs a weighted sum of messages from different source nodes. The feature updating step combines the aggregated message with the original features of the target node to update the node representation. After three rounds of message passing, a fused feature vector containing graph structure information is obtained. For example, for a bridge damage point node, the initial feature dimension is 256. After message passing, the fused feature vector still has a dimension of 256, but it includes related maintenance resource matching information and time constraint information.

[0136] Historical maintenance records are processed using a temporal graph convolutional network. The network consists of two graph convolutional layers and one temporal convolutional layer. The graph convolutional layers have a 3×3 kernel, and the temporal convolutional layer has a 1×5 kernel. The network input is a sequence of temporal graphs constructed from historical maintenance records, where each graph represents the maintenance status of a time slice. Self-connections are added to construct an adjacency matrix, with a self-connection weight set to 0.5 to indicate the degree of retention of node information. A degree matrix is ​​calculated, where the diagonal elements are the sum of the weights of the edges connecting nodes, used for normalization. Based on the degree matrix and fused feature vectors, a temporal feature matrix is ​​calculated, capturing the temporal patterns and spatial dependencies of maintenance activities. For example, for historical maintenance records of tunnel facilities in a certain area, after processing by the temporal graph convolutional network, a 127×128 temporal feature matrix is ​​obtained, where 127 represents the number of damage points, and 128 represents the temporal feature dimension of each point.

[0137] Decision variables are determined based on the time-series feature matrix and multi-objective constraints. These variables include the maintenance schedule for each damage point, the type of allocated maintenance resources, and the selection of maintenance schemes. Weight coefficients are set for the multi-objective constraints, and the importance of different objectives is determined according to the strategic preferences of the road infrastructure management department. For example, in a highway management scenario, the weight of total maintenance cost is 0.4, maintenance quality is 0.3, traffic impact minimization is 0.2, and resource utilization efficiency is 0.1. A multi-objective optimization problem is constructed by combining these factors, with the optimization objective being the maximization of a weighted objective function value.

[0138] The NSGA-III algorithm is applied to optimize the multi-objective optimization problem. The NSGA-III algorithm parameters are set as follows: population size 200, maximum number of iterations 500, crossover probability 0.85, and mutation probability 0.15. A systematic layout strategy is adopted for reference points, uniformly generating 45 reference points in the objective space to guide the population evolution in different directions within the multi-objective space. A reference point selection strategy is used to select the population, ensuring a balance between population diversity and convergence. The vertical distance method is used to calculate the correlation degree of reference points, associating each solution with the nearest reference point and selecting the population based on the correlation degree. A local search mechanism is combined to further optimize promising solutions. The local search uses simulated annealing with an initial temperature of 100 and a cooling coefficient of 0.95. After optimization, a set of optimized maintenance schemes consisting of a series of non-dominated solutions is obtained. For example, for a road infrastructure maintenance problem with 87 damage points, after optimization by the NSGA-III algorithm, 31 non-dominated solutions are obtained, representing compromise solutions for different maintenance strategies.

[0139] The fitness value of the optimized maintenance plan is calculated based on historical maintenance records. The fitness calculation considers three aspects: feasibility, cost-effectiveness, and resource balance. The feasibility score reflects the degree to which the plan meets the constraints; the cost-effectiveness score reflects the input-output ratio; and the resource balance score reflects the uniformity of resource allocation. A weighted combination yields the comprehensive fitness value. The parameters in the NSGA-III algorithm, including crossover probability, mutation probability, and reference point distribution, are updated based on the fitness value. The optimization process is repeated until the convergence criteria are met: the improvement of the optimal solution over 50 consecutive generations is less than 0.1%, or the maximum number of iterations is reached. The optimal maintenance strategy is obtained, and a maintenance scheduling plan is generated, including detailed information such as the specific maintenance time for each damage point, allocated maintenance resources, expected maintenance costs, and quality indicators.

[0140] In this embodiment, by constructing a spatiotemporal constraint matrix based on damage information and optimal maintenance time, multi-dimensional factors such as damage location, time, and type can be uniformly represented in the spatiotemporal constraint space. This ensures that the maintenance optimization process simultaneously satisfies time coordination and spatial rationality, significantly improving the overall synergy and executability of the maintenance plan. By calculating cost constraint thresholds and maintenance quality constraint thresholds to construct multi-objective constraints, the optimization process can be further ensured to achieve a dynamic balance between cost and quality, avoiding over-maintenance or resource waste. By extracting edge feature vectors through an attention mechanism and combining them with a message passing mechanism to fuse multi-dimensional correlation features, the high correlation between key nodes can be highlighted, thereby improving the expressive power of complex maintenance relationship modeling. By combining the temporal feature matrix with multi-objective constraints, an optimization problem containing multi-dimensional objectives such as maintenance cost, quality, and time coordination is constructed. The NSGA-III algorithm is used for multi-objective evolutionary optimization, which can maintain Pareto optimal balance among multiple objectives. By combining the reference point selection strategy and the local search mechanism, the population search is made more efficient, and the optimization results converge faster and more stably.

[0141] In one alternative implementation,

[0142] The NSGA-III algorithm is applied to optimize the multi-objective optimization problem. Based on a reference point selection strategy, population selection is performed, and a local search mechanism is combined to obtain an optimized maintenance scheme, including:

[0143] Obtain the natural attenuation coefficients corresponding to different road facilities, and fit the damage growth curves according to the historical evolution trend of each damage type to calculate the development function values ​​and damage weight coefficients of various types of damage. Determine the road facility condition index threshold constraints and maintenance time constraints based on the development function values ​​and the damage weight coefficients.

[0144] Historical data on road facility damage is collected and performance degradation characteristics are calculated and determined. Based on the performance degradation characteristics, the damage accumulation law and performance recovery effect are analyzed, and life cycle analysis is performed to obtain life cycle analysis results. A set of reference points is generated based on the life cycle analysis results and preset balance parameters.

[0145] The road facility damage development rate is obtained, and the local search direction and search step size are determined based on the road facility damage development rate. The NSGA-III algorithm is used to optimize the multi-objective optimization problem by combining the road facility condition index threshold constraint and the maintenance time constraint. Population selection is performed based on the reference point set, and a local search mechanism is executed by combining the local search direction and the search step size to obtain the optimized maintenance scheme.

[0146] The natural decay coefficients (FDCs) for different road infrastructures were obtained by fitting long-term monitoring data. These FDCs reflect the natural aging rate of the infrastructure under conditions of no external interference. They were obtained using an exponential decay model based on years of historical monitoring data. Significant differences exist in the FDCs for different types of road infrastructure. For example, the FDC for concrete bridges is 0.023 / year, for steel bridges it is 0.031 / year, for asphalt pavement it is 0.048 / year, for cement concrete pavement it is 0.026 / year, for tunnel lining it is 0.019 / year, for guardrails it is 0.035 / year, and for traffic signs it is 0.042 / year. A higher FDC indicates a faster aging rate and requires more frequent inspections and maintenance.

[0147] Damage growth curves were fitted based on the historical evolution trends of each damage type. The fitting method employed piecewise polynomial regression, dividing the damage development process into three stages: the initial stage, the development stage, and the acceleration stage. Different orders of polynomials were used for fitting in each stage. For example, for bridge crack damage, a first-order polynomial was used for the initial stage (0-2 years), a second-order polynomial for the development stage (2-5 years), and a third-order polynomial for the acceleration stage (over 5 years). The development function values ​​for each type of damage were calculated using the fitted curve equations. Taking a main beam crack on a highway bridge as an example, the initial stage function value growth rate was 0.05 / year, the development stage was 0.12 / year, and the acceleration stage was 0.28 / year, indicating that the crack propagation rate accelerated over time.

[0148] Damage weighting coefficients are calculated to reflect the degree of impact of different damage types on the overall performance of the facility. These coefficients are determined using the analytic hierarchy process (AHP), considering the impact of damage on the facility's safety, functionality, and durability. For example, for bridge facilities, the weighting coefficient for structural cracks is 0.35, for bearing damage it is 0.25, for steel reinforcement corrosion it is 0.20, for concrete spalling it is 0.15, and for expansion joint damage it is 0.05. A higher weighting coefficient indicates a greater impact of that type of damage on the overall performance of the facility, and a higher maintenance priority.

[0149] The threshold constraints for the road infrastructure condition index and the maintenance time constraints are determined based on the development function value and the damage weighting coefficient. The road infrastructure condition index is a comprehensive indicator measuring the overall condition of the infrastructure, ranging from 0 to 100, with 100 representing the best condition. The condition index threshold constraint sets a critical value to trigger maintenance; when the condition index falls below the threshold, maintenance is required. Different threshold constraints are set for facilities of different importance levels. For example, the condition index threshold for a super-important bridge is 75, for a first-class important bridge it is 70, and for a second-class important bridge it is 65. The maintenance time constraint includes the earliest time that maintenance can begin and the latest time that maintenance must be completed, which are calculated using the condition index decay curve and the warning value. For example, if a bridge currently has a condition index of 72, a decay rate of 3 / year, and a warning threshold of 65, then the latest time that maintenance must be completed is the current time plus (72-65) / 3 = 2.33 years, approximately 28 months later.

[0150] Historical data on road infrastructure damage was collected, and performance degradation characteristics were calculated. These characteristics were extracted using time series analysis and included four aspects: degradation rate, volatility, periodicity, and inflection point characteristics. The degradation rate represents the average speed of performance decline; volatility represents the stability of the degradation process; periodicity indicates the presence of seasonal variations; and the inflection point characteristic indicates the location and intensity of performance abrupt changes. For example, the performance degradation characteristic analysis of leakage damage to a tunnel lining showed a degradation rate of 5.2 points / year, a volatility coefficient of 0.18, a significant exacerbation during the rainy season, and an inflection point intensity of 8.5 points / year in the third year after use, indicating a significant increase in leakage in the third year.

[0151] Based on the analysis of performance degradation characteristics, the damage accumulation law and performance recovery effect are analyzed. The damage accumulation law is analyzed using Miner's linear cumulative damage theory to calculate the cumulative damage degree. The performance recovery effect analysis establishes a maintenance effect model by comparing the changes in performance indicators before and after historical maintenance. Life cycle analysis is performed using Monte Carlo simulation, combining the damage accumulation law and the performance recovery effect model to predict the life cycle performance of the facility under different maintenance strategies. The life cycle analysis results include remaining useful life, total life cycle cost, number of maintenance operations, and performance fluctuation range. For example, the life cycle analysis results for a bridge bearing show that the remaining useful life without maintenance is 4.2 years, the remaining useful life with a conventional maintenance strategy is 8.7 years, and the remaining useful life with a preventative maintenance strategy is 12.3 years, with total life cycle costs of 0, 250,000 yuan, and 320,000 yuan, respectively.

[0152] A set of reference points is generated based on the lifecycle analysis results and preset balance parameters. This set guides the diversity distribution of solutions in the NSGA-III algorithm. The balance parameters reflect the decision-maker's preference for different objectives, including cost preference coefficient, lifecycle preference coefficient, and performance preference coefficient. The reference points are generated using an adaptive grid method, uniformly distributing multiple points in the objective space to form N reference points. For example, under three objective dimensions (minimizing maintenance costs, maximizing lifecycle, and maximizing performance stability), with balance parameters set to [0.4, 0.4, 0.2], 45 reference points are generated, including 18 points biased towards the cost objective, 18 points biased towards the lifecycle objective, and 9 points biased towards the performance stability objective.

[0153] The damage development rate of road facilities is obtained to determine the local search direction and search step size. The damage development rate is calculated from the time derivative of the monitoring data and reflects the speed of damage propagation. For example, the fatigue crack of a steel structure bridge has a length growth rate of 2.5 mm / month, a width growth rate of 0.15 mm / month, and a depth growth rate of 0.8 mm / month during the monitoring period. The local search direction and search step size are determined based on the damage development rate. The faster the development rate, the more the search direction tends to be geared towards earlier repair, and the larger the search step size. For example, for damage points with a high development rate, the local search direction is biased towards an earlier repair time factor of 0.85, and the search step size is set to 15 days; for damage points with a low development rate, the local search direction is biased towards a later repair time factor of 1.12, and the search step size is set to 30 days.

[0154] The NSGA-III algorithm is applied to optimize a multi-objective optimization problem by combining a road infrastructure condition index threshold constraint and a maintenance time constraint. The NSGA-III algorithm configuration includes a population size of 200, a maximum number of iterations of 300, a crossover probability of 0.85, a mutation probability of 0.1, and 45 reference points. The optimization objectives are to minimize maintenance costs, maximize infrastructure lifespan, and maximize performance stability. Constraints include ensuring the condition index does not fall below the threshold, maintenance time meets the time window requirements, and maintenance resources do not exceed available limits. During the iteration process, non-dominated sorting and reference point association mechanisms are used to maintain population diversity while simultaneously approaching the Pareto front.

[0155] Population selection is performed based on a set of reference points to ensure a good distribution of the final solution set in the target space. The selection method uses reference point correlation calculation to associate each solution with the nearest reference point, and filters based on correlation counts to ensure a balanced number of solutions associated with each reference point. For example, in the 200th iteration, the 45 reference points average 4-5 solutions each, with the reference point with the fewest associations having 2 solutions and the reference point with the most associations having 7 solutions, resulting in a relatively balanced overall distribution.

[0156] A local search mechanism is implemented by combining the local search direction and search step size. The local search employs simulated annealing to fine-tune excellent solutions on the Pareto front. The initial temperature is set to 100°C, and the cooling coefficient is 0.95. For each solution, perturbation is applied in the local search direction in units of the search step size, and the acceptance probability is dynamically adjusted based on the temperature and changes in the objective function. Through local search, the quality of the solution is further optimized, making it closer to the true Pareto front. For example, during the local search for a certain solution, the maintenance time was adjusted from the originally planned 180 days to 165 days, and the maintenance cost increased by 2.3%, but the facility's expected lifespan increased by 7.5%, resulting in a better overall evaluation.

[0157] The solution yields multiple non-dominated solutions, representing the optimal strategies under different preferences. For example, the optimization results for a highway bridge group contain 31 non-dominated solutions. Among them, the cost-oriented solution has a maintenance cost of 1.57 million yuan and an average facility life extension of 5.3 years; the life-oriented solution has a maintenance cost of 2.13 million yuan and an average facility life extension of 8.7 years; and the balanced solution has a maintenance cost of 1.85 million yuan and an average facility life extension of 7.2 years. Based on actual needs and resource conditions, the most suitable maintenance solution is selected from these non-dominated solutions as the optimized maintenance solution.

[0158] In this embodiment, by obtaining the natural attenuation coefficients of different road facilities and fitting the damage growth curves with the historical evolution trends of various damage types, the degradation patterns of different facilities under actual operating conditions can be accurately characterized. By determining the threshold constraints of the road facility condition index and the maintenance time constraints based on the damage development function and damage weight, dynamic constraints can be introduced to ensure that the optimization results take into account both the facility health status and the rationality of maintenance timing. By collecting historical damage data and extracting performance degradation characteristics, the performance degradation and maintenance recovery patterns of facilities at different damage stages can be systematically analyzed. By generating a set of reference points by combining preset balance parameters, a reasonable target distribution reference is provided for the multi-objective optimization process, ensuring that the optimization direction achieves a balance between maintenance cost, life extension and performance recovery. By introducing the road facility damage development rate as an optimization guidance factor, the local search direction and search step size can be dynamically determined, realizing the adaptive adjustment of the optimization process.

[0159] A second aspect of this invention provides a deep learning-based intelligent inspection and maintenance scheduling system for road facilities, comprising:

[0160] The first unit is used to acquire multi-dimensional historical inspection data and perform data annotation to obtain road facility annotation data, and to perform damage identification on the road facility annotation data to obtain damage information.

[0161] The second unit is used to collect historical maintenance records and classify them according to the damage information, extract the maintenance time interval sequence corresponding to each type of damage, determine the abnormal maintenance time points in the maintenance time interval sequence and remove them to obtain the corrected maintenance time interval sequence, combine the pre-acquired maintenance cost data to determine the maintenance time prediction features, calculate the temporal importance of each dimension of the maintenance time prediction features and determine the optimal maintenance time for each type of damage.

[0162] The third unit is used to determine multi-objective constraints and construct a multi-objective optimization problem based on the damage information and the optimal maintenance time. The multi-objective optimization problem is solved by a genetic algorithm to obtain an optimized maintenance scheme. The fitness of the optimized maintenance scheme is evaluated based on the historical maintenance records and iteratively optimized to obtain the optimal maintenance strategy and generate a maintenance scheduling scheme.

[0163] A third aspect of the present invention provides an electronic device, comprising:

[0164] A processor and a memory for storing processor-executable instructions, wherein the processor is configured to invoke instructions stored in the memory to perform the aforementioned method.

[0165] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0166] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent inspection and maintenance scheduling of road facilities based on deep learning, characterized in that, The method comprises the following steps: acquiring multi-dimensional historical inspection data and performing data labeling to obtain road facility labeled data, and performing damage identification on the road facility labeled data to obtain damage information; collecting historical maintenance records and classifying them according to the damage information, extracting the maintenance time interval sequence corresponding to each type of damage, determining the abnormal maintenance time points in the maintenance time interval sequence and eliminating them to obtain a corrected maintenance time interval sequence, determining a maintenance time prediction feature based on the corrected maintenance time interval sequence, and calculating the time sequence importance of each dimension of the maintenance time prediction feature according to the pre-acquired maintenance cost data and determining the optimal maintenance time for each type of damage; determining multi-objective constraint conditions based on the damage information and the optimal maintenance time and constructing a multi-objective optimization problem, solving the multi-objective optimization problem by a genetic algorithm to obtain an optimized maintenance scheme, evaluating the fitness of the optimized maintenance scheme according to the historical maintenance records and iteratively optimizing to obtain an optimal maintenance strategy and generate a maintenance scheduling scheme, comprising: constructing a space-time constraint matrix based on the damage information and the optimal maintenance time, calculating the cost constraint threshold and the maintenance quality constraint threshold based on the space-time constraint matrix to obtain the multi-objective constraint conditions; extracting the location information and damage type in the damage information, setting nodes based on the pre-set initial maintenance scheme and connecting the nodes to obtain a heterogeneous graph network, for the edges in the heterogeneous graph network, extracting edge feature vectors by an attention mechanism and constructing an edge weight matrix, and fusing the edge feature vectors to obtain a fused feature vector by a message passing mechanism; processing the historical maintenance records by a time series graph convolution network, adding self-connections to construct an adjacency matrix and calculating a degree matrix, and calculating a time series feature matrix based on the degree matrix and the fused feature vector; determining decision variables based on the time series feature matrix and the multi-objective constraint conditions and setting the weight coefficients of the multi-objective constraint conditions, combining to construct a multi-objective optimization problem, executing an NSGA-III algorithm on the multi-objective optimization problem for optimization, selecting a population based on a reference point selection strategy and combining a local search mechanism to obtain an optimized maintenance scheme; calculating the fitness value of the optimized maintenance scheme based on the historical maintenance records and updating the parameters in the NSGA-III algorithm based on the fitness value, repeatedly optimizing until convergence, obtaining an optimal maintenance strategy and generating a maintenance scheduling scheme.

2. The method of claim 1, wherein, The method comprises the following steps: acquiring multi-dimensional historical inspection data and performing data labeling to obtain road facility labeled data, and performing damage identification on the road facility labeled data to obtain damage information; collecting road inspection corresponding apparent image data, set contour data and internal structure data to obtain historical inspection data, extracting crack propagation features in the historical inspection data and obtaining fatigue damage features by calculating cumulative damage degree, and performing data labeling based on the fatigue damage features and the crack propagation features to obtain road facility labeled data; The physical feature and the damage feature corresponding to the road facility labeling data are extracted, a category loss value is calculated based on the physical feature and the damage feature, and a damage contribution degree corresponding to each feature at different scales is determined based on the category loss value; and the physical feature and the damage feature are weighted and fused based on the damage contribution degree to obtain damage information.

3. The method of claim 2, wherein, The category loss value is calculated based on the physical feature and the damage feature, and the damage contribution degree corresponding to each feature at different scales is determined based on the category loss value. The physical feature is modeled by a Gaussian mixture distribution to obtain a physical feature probability distribution, and a first uncertainty is calculated based on the physical feature probability distribution; The damage feature is modeled by a pre-set Bayesian network probability network to obtain a damage feature probability distribution, a second uncertainty is calculated based on the damage feature probability distribution, and a category loss value is obtained by adaptively fusing the first uncertainty and the second uncertainty based on a pre-set cross-entropy loss; A loss gradient matrix is constructed based on the category loss value, and a sensitivity score of the physical feature and the damage feature is calculated, and the sensitivity score is input into a pre-set feature screening network to obtain a feature importance vector; The physical feature probability distribution and the damage feature probability distribution are resampled based on the feature importance vector to determine a confidence interval and to calculate a damage contribution degree corresponding to the physical feature and the damage feature.

4. The method of claim 1, wherein, According to the damage information, a classification is performed, a maintenance time interval sequence corresponding to each type of damage is extracted, an abnormal maintenance time point in the maintenance time interval sequence is determined and removed to obtain a corrected maintenance time interval sequence, which includes: According to the damage information, a damage classification standard is constructed, historical maintenance information matching the damage classification standard is extracted from the historical maintenance records and is classified to obtain an initial damage type, and each initial damage type is screened according to a pre-set damage threshold to obtain a maintenance record category and to extract a time difference between adjacent maintenance times in each maintenance record category to obtain a maintenance time interval sequence; A fluctuation degree corresponding to the maintenance time interval sequence is calculated, and a dynamic time window is initialized based on the fluctuation degree, the maintenance time interval sequence is segmented according to the dynamic time window, and statistical features and trend features in each segment are calculated; The statistical features and the trend features are time-decomposed, and a maintenance rule index is constructed according to a change law of a component obtained by the decomposition, a deviation degree between a current maintenance time interval sequence and the maintenance rule index is calculated, a dynamic threshold determination criterion is constructed, an abnormal maintenance time point in the maintenance time interval sequence is identified based on the dynamic threshold determination criterion, and the abnormal maintenance time point is removed to obtain a corrected maintenance time interval sequence.

5. The method of claim 1, wherein, Based on the corrected maintenance time interval sequence, a maintenance time prediction feature is determined, and a time sequence importance of each dimension of the maintenance time prediction feature is calculated based on pre-acquired maintenance cost data, and an optimal maintenance time of each type of damage is determined. extracting statistical features corresponding to the modified maintenance time interval sequence and calculating periodic features corresponding to the modified maintenance time interval sequence through Fourier transform, combining the statistical features and the periodic features to obtain maintenance time prediction features, and calculating a maintenance benefit index based on the maintenance cost data and the performance parameter difference before and after maintenance; calculating a sensitivity score corresponding to the maintenance time prediction features based on the maintenance benefit index, initializing importance weights based on the sensitivity score, and obtaining time sequence importance of each dimension feature through neural network analysis of time sequence correlation corresponding to the maintenance time prediction features; performing principal component analysis on the maintenance time prediction features to reduce dimension to obtain main feature components, evaluating the influence degree of the main feature components on the maintenance benefit index through the XGBoost algorithm, and determining comprehensive importance weights based on the influence degree and the time sequence importance; setting the maintenance benefit index as an optimization target, and iteratively solving based on the particle swarm optimization algorithm combined with the comprehensive importance weights and a pre-set upper limit of maintenance cost to obtain a benchmark maintenance time, initializing type correction coefficients according to pre-acquired damage types, and calculating an optimal maintenance time based on the type correction coefficients and the benchmark maintenance time.

6. The method of claim 1, wherein, performing NSGA-III algorithm optimization on the multi-objective optimization problem, selecting a population based on a reference point selection strategy and solving to obtain an optimized maintenance scheme including: obtaining natural attenuation coefficients corresponding to different road facilities and fitting damage growth curves according to historical evolution trends of each damage type to calculate development function values and damage weight coefficients of various damages, and determining road facility condition index threshold constraints and maintenance time constraints based on the development function values and the damage weight coefficients; collecting road facility damage historical data and calculating and determining performance degradation features, analyzing damage accumulation rules and performance recovery effects based on the performance degradation features and performing life cycle analysis to obtain life cycle analysis results, and generating a reference point set according to the life cycle analysis results and pre-set balance parameters; obtaining road facility damage development rates, determining a local search direction and a search step length based on the road facility damage development rates, performing NSGA-III algorithm optimization on the multi-objective optimization problem combined with the road facility condition index threshold constraints and the maintenance time constraints, selecting a population based on the reference point set and executing a local search mechanism combined with the local search direction and the search step length, and solving to obtain an optimized maintenance scheme.

7. A deep learning based intelligent road facility inspection and maintenance scheduling system for implementing the method of any one of the preceding claims 1-6, characterized in that, including: a first unit configured to obtain multi-dimensional historical inspection data and perform data labeling to obtain road facility labeled data, and perform damage identification on the road facility labeled data to obtain damage information; The second unit is configured to collect historical maintenance records, classify the historical maintenance records according to the damage information, extract a maintenance time interval sequence corresponding to each type of damage, determine an abnormal maintenance time point in the maintenance time interval sequence and eliminate the abnormal maintenance time point to obtain a corrected maintenance time interval sequence, determine a maintenance time prediction feature based on the corrected maintenance time interval sequence, calculate a time sequence importance of each dimension of the maintenance time prediction feature according to pre-acquired maintenance cost data, and determine an optimal maintenance time for each type of damage. The third unit is configured to determine a multi-objective constraint condition based on the damage information and the optimal maintenance time, construct a multi-objective optimization problem, solve the multi-objective optimization problem by using a genetic algorithm to obtain an optimized maintenance scheme, evaluate an adaptability of the optimized maintenance scheme according to the historical maintenance records, and iteratively optimize the optimized maintenance scheme to obtain an optimal maintenance strategy and generate a maintenance scheduling scheme.

8. An electronic device, comprising: The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 6. The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 6. ​ ​ 9. A computer-readable storage medium having stored thereon computer program instructions, wherein, ​

Images