Multilevel image intelligent damage assessment and prediction system for highway infrastructure

Abstract

This invention discloses a multi-level image-based intelligent damage assessment and prediction system for highway infrastructure, belonging to the fields of image processing and artificial intelligence technology. The system includes a multi-scale feature extraction module, a damage type intelligent identification module, a temporal damage evolution prediction module, and an adaptive maintenance decision module. It extracts multi-scale geometrically invariant features through topological manifold learning, achieves robust damage identification based on Lie group transformation theory, establishes a damage evolution prediction model using dynamical system theory, optimizes maintenance decisions using a game theory framework, and forms a closed-loop adaptive optimization system. This invention achieves comprehensive damage assessment, accurate prediction, and intelligent decision-making, significantly improving the intelligent level of highway infrastructure health monitoring and preventive maintenance capabilities.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and artificial intelligence, and particularly relates to a multi-level image intelligent damage assessment and prediction system for highway infrastructure, which is suitable for health monitoring and preventive maintenance management of highway pavement, bridges, tunnels and other infrastructure. BACKGROUND

[0002] Highway infrastructure is an important part of the transportation system, and its operating state directly affects traffic safety and transportation efficiency. With the continuous expansion of the highway network and the growth of the service life, the problem of infrastructure damage is increasingly prominent. Traditional manual inspection methods have low efficiency, high cost, strong subjectivity and other defects, and are difficult to meet the needs of large-scale road network management. In recent years, automatic damage detection technology based on image recognition has been widely studied, but existing technologies still have many shortcomings.

[0003] Existing image recognition methods mainly use single-scale feature extraction, which cannot fully capture damage features at different scales. Macroscopic images can reflect the overall condition of the road surface, but have limited ability to identify fine cracks; microscopic images can clearly show the texture of the material surface, but lack overall spatial information. This single-scale analysis method leads to insufficient recognition accuracy and is prone to missed detection and false detection.

[0004] A Chinese invention patent with the publication number CN115641334B discloses a road anti-collision facility damage identification method, which identifies the anti-collision barrel through an object recognition model and judges the damage by using the reflective area and the reflective intensity. This method mainly targets reflective film facilities and judges whether there is damage by comparing the reflective characteristics with a preset threshold. Although this method has certain effect in specific scenarios, it has obvious limitations. First, this method only judges damage based on reflective characteristics, and it is difficult to accurately evaluate facilities with non-reflective materials or degraded reflective performance. Second, this method only considers the damage state at the current time and lacks the ability to predict the evolution trend of damage, which cannot support preventive maintenance decisions. Third, this method uses fixed threshold judgment and cannot adaptively adjust according to environmental conditions and facility types, which lacks robustness in complex scenarios. Fourth, this method only focuses on damage identification of a single facility and does not consider the damage correlation and systematic degradation rules of multiple facilities in a region.

[0005] Most of the existing damage detection methods focus on the static identification of damage, lacking dynamic analysis of the damage evolution process and prediction of the future state. Pavement damage is a gradual development process, and early minor cracks will gradually expand under the action of traffic load and environmental factors, eventually evolving into serious structural damage. If maintenance is only carried out when the damage is obvious, not only the maintenance cost is high, but also it may affect traffic safety. Therefore, it is of great significance to establish a damage evolution prediction model to accurately predict the damage development trend for formulating preventive maintenance strategies.

[0006] The existing technology mainly adopts traditional convolutional neural network in feature extraction. Although this kind of method has achieved good results in target detection tasks, it is often difficult to extract essential features with geometric invariance when dealing with damage images with complex geometric structures. Damage presents different image features under different viewing angles, lighting conditions and collection distances, and traditional methods lack robustness to such geometric transformations, leading to performance degradation in practical applications.

[0007] Most of the existing maintenance decision methods are based on single objective optimization, such as only considering minimizing maintenance cost or maximizing road performance, without considering multiple mutually restrictive objectives such as cost, performance, traffic impact. In practical applications, maintenance decision is a multi-objective optimization problem, which needs to balance between different objectives. In addition, the existing methods lack adaptive adjustment mechanism for detection system, and cannot dynamically optimize resource allocation according to damage distribution and evolution trend.

[0008] In view of the above problems, it is urgent to develop a multi-level, intelligent and predictive highway infrastructure damage evaluation and prediction system, which can comprehensively analyze damage features from multiple scales, accurately predict damage evolution trend, and realize adaptive maintenance decision optimization, providing technical support for scientific management of highway infrastructure. SUMMARY

[0009] The purpose of the present application is to overcome the shortcomings of the prior art, provide a multi-level image intelligent damage evaluation and prediction system for highway infrastructure, realize comprehensive evaluation, accurate prediction and intelligent decision of highway infrastructure damage through multi-scale feature level extraction, damage recognition based on Lie group invariance, damage prediction driven by dynamic system theory and maintenance decision optimization under the framework of game theory, and form a deep coupling closed-loop adaptive optimization system.

[0010] To achieve the above object, the application adopts the following technical solutions. The application provides a highway infrastructure multi-level image intelligent damage assessment and prediction system, which comprises a multi-scale feature level extraction module, an intelligent damage type identification module, a time sequence damage evolution prediction module and a self-adaptive maintenance decision module. The multi-scale feature level extraction module acquires a multi-resolution image sequence of the highway infrastructure, constructs a feature representation space of three scales of macro, meso and micro based on a topological manifold learning method, maps high-dimensional image features to low-dimensional topological manifolds through a manifold embedding algorithm, and extracts geometric invariance feature vectors of each scale. The intelligent damage type identification module receives the geometric invariance feature vectors of each scale, constructs a rotation translation scale invariance representation of a damage mode based on a Lie group transformation theory, identifies a damage type through orbit decomposition of a transformation group, and generates a damage feature descriptor. The time sequence damage evolution prediction module receives the damage feature descriptor and a historical image sequence, establishes a differential equation model of damage propagation based on a dynamic system theory, and predicts a damage state distribution at a future time. The self-adaptive maintenance decision module receives the damage feature descriptor and the damage state distribution, constructs a multi-objective optimization model based on a game theory framework, solves an optimal maintenance scheme, generates a feedback control signal to adjust an image acquisition strategy and a feature extraction weight of the multi-scale feature level extraction module, and forms a closed-loop self-adaptive optimization system.

[0011] The beneficial effects of the present application are as follows: first, by the multi-scale feature level extraction module, the image information of macro, meso and micro three scales is fused, the damage characteristics of different levels are comprehensively captured, the limitations of single scale analysis are overcome, and the comprehensiveness and accuracy of damage identification are significantly improved. Second, the topological manifold learning method is used to extract image features, the low-dimensional feature representation with geometric fidelity is obtained by preserving the intrinsic geometric structure of the data, and compared with the traditional Euclidean space feature extraction method, it has stronger expression ability and robustness in processing complex geometric structure of damage image. Third, based on the Lie group transformation theory, the invariance representation of damage mode is constructed, the robust recognition of damage under different viewing angles, illumination and scales is realized, the defects of sensitivity to geometric transformation of existing methods are overcome, and the adaptability of the system in complex environment is improved. Fourth, the dynamic system theory is introduced to establish the damage evolution prediction model, the dynamics mechanism of damage propagation is revealed through phase space reconstruction, the accurate prediction of future damage state is realized, and a scientific basis is provided for preventive maintenance. Fifth, a multi-objective maintenance decision optimization framework based on game theory is constructed, scientific trade-off is realized among maintenance cost, road performance and traffic influence and other multiple objectives, the Pareto optimal maintenance scheme is obtained, and the rationality and economy of decision-making are improved. Sixth, the adaptive optimization of the detection system is realized through the feedback control mechanism, the sampling strategy and resource allocation are dynamically adjusted according to the damage distribution and evolution prediction result, a closed-loop optimization system is formed, and the intelligent level and operation efficiency of the system are significantly improved. Seventh, the modules form a deep coupling and collaborative relationship, the multi-scale feature extraction provides high-quality input for damage identification, the results of damage identification and evolution prediction jointly drive the maintenance decision, and the feedback of maintenance decision optimizes the feature extraction process, realizes the synergistic effect of 1+1>2, and the overall system performance is much higher than the sum of the independent operation of each module. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 Fig. 1 is a schematic diagram of the overall structure of the present application;

[0013] Figure 2 Fig. 4 is a detailed structure diagram of the multi-scale feature level extraction module of the present application;

[0014] Figure 3 Fig. 6 is a detailed structure diagram of the intelligent damage type recognition module of the present application;

[0015] Figure 4 Fig. 8 is a detailed structure diagram of the time sequence damage evolution prediction module of the present application;

[0016] Figure 5 Fig. 10 is a detailed structure diagram of the adaptive maintenance decision module of the present application. DETAILED DESCRIPTION

[0017] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] With reference to Figure 1 The highway infrastructure multi-level image intelligent damage assessment and prediction system comprises a multi-scale feature level extraction module 1, an intelligent damage type recognition module 2, a time series damage evolution prediction module 3 and a self-adaptive maintenance decision module 4. The four modules form a deep-coupled closed-loop collaborative system, and through multi-level coupling of parameters, states and logic, the system performance of mutual promotion, superposition of efficiency, and collaborative optimization is realized.

[0019] With reference to Figure 2 The multi-scale feature level extraction module 1 comprises an image acquisition unit, a topological feature extraction unit and a scale fusion unit. The core innovation of this module is to use topological manifold learning method to extract damage features with geometric invariance from multiple scales, providing high-quality feature representation for subsequent damage recognition and prediction.

[0020] The image acquisition unit obtains multi-source image data of highway infrastructure through visible light cameras, infrared cameras and laser scanning devices. The visible light camera obtains natural light images of the road surface, with a resolution of 4096x2160 pixels and a frame rate of 60fps, which can clearly capture the overall condition and obvious damage of the road surface. The infrared camera works in the 8-14 μm wave band and can identify hidden damages such as internal cavities and delamination by detecting the temperature distribution of the road surface. The laser scanning device uses line laser method with a scanning frequency of 2 kHz, which can obtain three-dimensional topographic data with an accuracy of 0.1 mm for precise measurement of crack width and depth. Through the fusion of multi-source images, an image pyramid containing macro-scale pixel-level images, meso-scale millimeter-level images and micro-scale micron-level images is constructed. The field of view of the macro image is 120°, covering an area of 4m wide, which is used to assess the overall condition of the road section. The field of view of the meso image is 30°, covering an area of 1m wide, which is used to identify typical damage types. The field of view of the micro image is 5°, covering an area of 10cm wide, which is used to analyze the material surface texture and micro cracks.

[0021] The topological feature extraction unit performs topological manifold learning on each level of the image pyramid to extract topological features that preserve local geometric structure. This unit first preprocesses the images at each scale, including denoising, contrast enhancement, and edge detection. Denoising employs nonlocal mean filtering, which removes noise while preserving edge information. Contrast enhancement uses adaptive histogram equalization, automatically adjusting image contrast for different lighting conditions. Edge detection uses the Canny operator, extracting damaged edges through dual threshold detection. The preprocessed image is converted into a high-dimensional feature vector, where each pixel's features include grayscale value, gradient magnitude, gradient direction, local binary mode, and Gabor filter response, forming a feature space of dimension D=128.

[0022] In a high-dimensional feature space, the topological feature extraction unit applies a topological manifold learning method to map the data to a low-dimensional topological manifold. This process is based on the manifold assumption, that is, high-dimensional image data actually lies on a low-dimensional manifold. First, it computes any two points in the feature space... and The geodesic distance between points is defined by the shortest path length on a manifold. Unlike the straight-line distance in Euclidean space, geodesic distance more accurately reflects the intrinsic geometric structure of the data. The geodesic distance is calculated using Dijkstra's shortest path algorithm on a k-nearest neighbor graph. For each data point, k=15 nearest neighbors are selected to construct a neighborhood graph, where the edge weights between adjacent points are their Euclidean distances. By calculating the shortest path on the neighborhood graph, the geodesic distance matrix between any two points is obtained. ,in Point and The geodesic distance between them.

[0023] Based on the geodesic distance matrix, the Laplacian eigenmap algorithm is used to project the data into a low-dimensional manifold space. This algorithm achieves dimensionality reduction by minimizing the following objective function: .

[0024] in, For low-dimensional embedding coordinate matrices, Let N be the coordinates of the i-th data point in the d-dimensional manifold space, where d = 32 is the dimension of the manifold, and N is the total number of data points. The adjacency weight matrix is ​​defined as follows:

[0025] .

[0026] in, express The set of k nearest neighbors, The Gaussian kernel width parameter is adaptively determined based on the local neighborhood density, and the calculation formula is as follows: The physical meaning of the objective function is to maintain the local neighborhood relationship of the high-dimensional space in the low-dimensional space, so that the originally adjacent data points are still close in the low-dimensional space.

[0027] The optimal solution of the objective function is obtained by solving a generalized eigenvalue problem: .

[0028] where, is the graph Laplacian matrix, is the degree matrix, satisfying , is the eigenvalue. Select the eigenvectors corresponding to the d smallest eigenvalues other than the smallest eigenvalue (usually 0) to form the low-dimensional embedding coordinate matrix . These eigenvectors correspond to the global coordinate system on the manifold and can reveal the intrinsic topological structure of the data.

[0029] Through the above manifold learning process, the high-dimensional features of the images of each scale are mapped to the low-dimensional topological manifold space to obtain macro features , meso features , and micro features . These features maintain the geometric structure of the original image while achieving dimension reduction and reducing the computational complexity of subsequent processing.

[0030] The scale fusion unit performs weighted fusion on the topological features of each scale to generate a multi-scale fusion feature vector. The fusion process is based on the attention mechanism and adaptively adjusts the fusion weights according to the feature response intensity of different damage types at each scale. The calculation formula of the multi-scale fusion feature vector is: .

[0031] where, , and are the fusion weights of the features of each scale, satisfying and . The fusion weights are calculated by the softmax function: .

[0032] where s represents the scale type, is the feature response intensity of scale s, which is determined by calculating the L2 norm of the feature vector, i.e. , is the temperature coefficient, which is used to control the smoothness of the weight distribution, and is preferably taken as , , . For crack damage, the feature response of the meso scale is the strongest, so is usually the largest. For pit and groove damage, the feature response of the macro scale is the strongest, so Generally the largest. For material spalling, micro-scale features respond most strongly, thus Generally the largest. Through this adaptive fusion mechanism, the system can automatically adjust the weight of each scale feature according to the damage type, improving the discriminability of feature representation.

[0033] The output of the multi-scale feature hierarchy extraction module is the geometric invariance feature vector of each scale , , and the multi-scale fusion feature vector These feature vectors serve as inputs to the damage type intelligent recognition module, achieving parameter-level deep coupling between modules.

[0034] Referring to Figure 3 , the damage type intelligent recognition module 2 includes a Lie group invariance encoding unit, a damage mode classification unit, and a damage quantification evaluation unit. The core innovation of this module is to construct a geometric invariance representation of damage mode based on Lie group transformation theory, achieving robust recognition of damage under different viewing angles, lighting, and scales.

[0035] The Lie group invariance encoding unit performs Lie group transformation on the geometric invariance feature vectors output by the multi-scale feature hierarchy extraction module 1, calculating the invariants of the features under the action of rotation group, translation group, and scale transformation group. Lie group is a class of mathematical objects that have both group structure and manifold structure, capable of describing the geometric transformation of images. In the damage recognition task, changes in camera viewing angle correspond to rotation transformation, changes in acquisition position correspond to translation transformation, and changes in acquisition distance correspond to scale transformation. By constructing a feature representation that is invariant to these transformations, the influence of geometric transformation on the recognition result can be eliminated.

[0036] Let the feature vector be transformed under the action of the Lie group to , where is a group element. The Lie group invariant satisfies , i.e., it remains unchanged under group transformation. For the rotation group , the invariant is constructed through orbit integration: .

[0037] where is the rotation matrix with rotation angle , is the feature extraction function, and the integration is performed on the rotation group. In actual calculation, a discrete approximation is used, selecting uniformly distributed rotation angles , where , and calculating: .

[0038] For the translation group , the invariants are obtained by computing the Fourier transform amplitude spectrum of the features. The amplitude spectrum of the Fourier transform is invariant to translation because translation only changes the phase and not the amplitude. The translation invariants are defined as .

[0039] where denotes the Fourier transform, and denotes taking the amplitude. For the scaling group , the invariants are constructed by a logarithmic polar coordinate transformation. In the logarithmic polar coordinate system, scaling corresponds to translation, so the same approach as for translation invariants can be adopted.

[0040] Combining the rotation, translation, and scaling three transformations, the joint invariants are constructed as .

[0041] where is the scaling invariant, and the superscript T denotes the transpose. This joint invariant remains invariant to rotation, translation, and scaling, and can achieve robust damage pattern recognition.

[0042] The damage pattern classification unit classifies the damage types based on the Lie group invariants. The classification process adopts the orbit decomposition theory of the transformation group. Under the Lie group action, the feature space is divided into several orbits, and each orbit contains all feature vectors that can be transformed into each other by group transformation. The feature vectors belonging to the same orbit correspond to the same damage pattern observed at different angles, positions, and scales. By classifying the feature vectors into different orbits, the automatic classification of damage types is achieved.

[0043] Let denote the orbit of the feature vector , which is defined as .

[0044] The distance between two orbits and is measured by the Hausdorff distance:

[0045] .

[0046] where denotes the distance from a point to an orbit , which is defined as . In practical calculations, it is difficult to directly calculate the Hausdorff distance because the orbit is a continuous set. A simplified method based on Lie group invariants is adopted to directly calculate the distance between invariants as an approximation of the orbit distance: .

[0047] Based on the distance of orbits, the damage modes are classified by kernel k-means clustering algorithm. The kernel function is Gaussian radial basis function: .

[0048] where, is the kernel width parameter. The number of clusters is set as K=5, corresponding to five types of damage: crack, pit, settlement, spalling and net crack. For a new feature vector , its class label is determined by calculating the kernel distance to each cluster center: .

[0049] where, is the center of the k-th cluster, which is determined by training data. For crack damage, further classification is performed by calculating the principal curvature and geodesic curvature of the damage profile. The principal curvature and are obtained by differential geometry analysis of the damage profile. The geodesic curvature is determined by the rate of change of the tangent vector of the profile. When and , it is determined as transverse or longitudinal crack, which is further distinguished by the crack strike angle. When and , it is determined as a crack or net crack.

[0050] The damage quantification unit measures the geometric parameters of the classified damage, including length, width, depth, area and volume. For crack damage, the length is calculated by integrating along the crack centerline, the width is determined by measuring the maximum transverse distance perpendicular to the centerline, and the depth is obtained by the three-dimensional profile data of the laser scanning device. For pit damage, the area is determined by counting the number of pixels in the damage area and converting it to the actual size, and the volume is calculated by integrating the three-dimensional profile data. Based on the geometric parameters, the damage severity is evaluated and the damage grade label is generated. The damage grade is divided into four levels: mild, moderate, severe and serious. For cracks, the grade judgment criteria are: mild crack width less than 3mm, length less than 1m; moderate crack width 3mm to 5mm, length 1m to 3m; severe crack width 5mm to 10mm, length 3m to 10m; serious crack width greater than 10mm or length greater than 10m. For pits, the grade judgment criteria are: mild pit depth less than 20mm, area less than 0.1m 2； ; moderate pit depth 20mm to 40mm, area 0.1m 2 to 0.5m 2 ; severe pit depth 40mm to 60mm, area 0.5m 2 to 1m 2； ; serious pit depth greater than 60mm or area greater than 1m2 .

[0051] The output of the damage type intelligent identification module is a damage feature descriptor, which includes damage location , type , geometric parameters and severity level , wherein is length, w is width, h is depth, A is area, and V is volume. The descriptor serves as an input of the time-series damage evolution prediction module and also provides current damage state information for the adaptive maintenance decision module, realizing state-level deep coupling between modules.

[0052] Referring to Figure 4 , the time-series damage evolution prediction module 3 includes a phase space reconstruction unit, a dynamics modeling unit and a prediction calculation unit. The core innovation of the module lies in establishing a differential equation model of damage propagation based on the theory of dynamic systems, revealing the dynamics mechanism of damage evolution through phase space reconstruction, and realizing accurate prediction of future damage states.

[0053] The phase space reconstruction unit reconstructs the phase space of the damage feature time series in the historical image sequence, revealing the dynamics trajectory of damage evolution. Phase space reconstruction is based on the Takens delay embedding theorem, which states that for a deterministic dynamic system, the delay coordinate vector of a single variable time series can reconstruct the attractor of the original system, thereby recovering the dynamics characteristics of the system.

[0054] Let the time series of a certain geometric parameter (such as crack width) of the damage be , wherein is the i-th sampling time. The delay embedding vector is defined as:

[0055] .

[0056] wherein is the time delay, is the embedding dimension. The delay embedding vector constitutes a point in the m-dimensional phase space, and the set of all delay vectors describes the dynamics trajectory of the system. The time delay is determined by the mutual information method. The mutual information measures the information correlation of the time series after delay : .

[0057] wherein is the joint probability distribution, and are the marginal probability distributions. The delay corresponding to the first local minimum is selected as the embedding delay The delay ensures sufficient time correlation and avoids information loss. In embodiments of the present application, the time delay is usually taken as 1 day to 3 days.

[0058] The embedding dimension m is determined by the false nearest neighbor method. The false nearest neighbor method is based on the idea that if the embedding dimension is not enough, some points that seem to be neighbors in the phase space are actually far apart on the real attractor, which is caused by the false nearest neighbors due to the dimension projection. By gradually increasing the embedding dimension, the proportion of false nearest neighbors is calculated, and when the proportion of false nearest neighbors is below a threshold, it is considered that the embedding dimension is sufficient. The false nearest neighbor criterion is: .

[0059] wherein, represents a delay vector in the m-dimensional space, is the nearest neighbor of , and is a threshold. If , it is considered that is the false nearest neighbor of . When the proportion of false nearest neighbors is less than 1%, the increase in the dimension is stopped, and the dimension at this time is the optimal embedding dimension m. In embodiments of the present application, the embedding dimension is usually to .

[0060] After the time delay and the embedding dimension are determined by the above method, the phase space reconstruction is performed on a plurality of geometric parameters (crack width, length, area, etc.) of the damage, and the dynamic trajectory of the damage evolution is recovered in the m-dimensional phase space. The phase space trajectory reveals the law of damage evolution and provides a basis for subsequent dynamic modeling.

[0061] The dynamic modeling unit establishes a differential equation model of damage propagation based on the phase space trajectory. Damage evolution is affected by a variety of factors, including traffic load, environmental temperature and humidity, and material aging degree. Based on the material fatigue cumulative damage theory, a differential equation is established to describe the damage growth.

[0062] For crack damage, its propagation process follows the Paris-Erdogan law, which describes the growth law of fatigue cracks under cyclic loading. The crack length changes with the number of cycles satisfies: .

[0063] wherein, is the stress intensity factor amplitude, and are material constants. The stress intensity factor amplitude is related to the stress amplitude and the crack length Correlation: .

[0064] where is a geometry correction factor, which is related to the crack shape and loading mode. For surface cracks, the geometry correction factor is . Substituting the stress intensity factor expression into Paris law, we obtain .

[0065] This is a differential equation for crack length . In practical applications, the number of cycles is related to time through traffic flow. Let the average daily vehicle traffic be , then the number of cycles per unit time is . Using the chain rule, the derivative of crack length with respect to the number of cycles is converted to the derivative with respect to time: .

[0066] In the embodiments of the present invention, based on the fatigue test data of asphalt pavement materials, the material constants are , . The stress amplitude is determined through pavement structure analysis, which is related to vehicle axle load and pavement modulus. For asphalt pavement under standard axle load (100 kN), the stress amplitude is about MPa. The average daily traffic is obtained from traffic survey data, and the typical value for expressways is to vehicles per day.

[0067] In addition to cyclic loading, environmental factors also significantly affect damage evolution. Temperature changes cause thermal expansion and contraction of materials, generating temperature stresses that accelerate crack propagation. Changes in humidity affect the viscoelastic properties of materials, reducing fatigue life. To consider the influence of environmental factors, an environmental correction factor is introduced, where is temperature, is humidity. The environmental correction factor is determined through regression analysis:

[0068] .

[0069] where ℃ and % are reference temperature and humidity, / ℃ and / % are temperature and humidity influence coefficients. Considering both traffic loading and environmental factors, the differential equation model for damage evolution is: This equation describes the variation of crack length over time, revealing the influence of traffic load, environmental conditions, and the current state of the crack on the evolution rate. Similar differential equation models are established for other types of damage, such as potholes and spalling, taking into account the corresponding physical mechanisms and influencing factors.

[0070] Furthermore, the Miner linear cumulative damage criterion is introduced to quantify the contribution of cyclic loading to damage growth. This criterion assumes that the fatigue life of the material accumulates linearly under different stress levels, and material failure occurs when the cumulative damage reaches 1. (Cumulative damage) The calculation formula is: .

[0071] in, Stress level The number of loops below, Stress level The fatigue life is below, Number of stress level types. Fatigue life. Determined through the SN curve. For asphalt concrete, the empirical formula for the SN curve is: .

[0072] in, and These are material parameters. The cumulative damage criterion can be used to assess the remaining life of the pavement, providing a basis for maintenance decisions.

[0073] The predictive computation unit calculates the damage state at future time steps by solving a differential equation model. Numerical integration is performed using the fourth-order Runge-Kutta method. For the differential equation... The iterative formula for the Runge-Kutta method is: .

[0074] in, For time step, , , , Time step The prediction timeframe is 180 days from now. Iterative calculations are used to obtain the predicted crack length at various future moments. .

[0075] To quantify the uncertainty of the prediction, the Monte Carlo simulation method is employed. This takes into account model parameters (such as material constants). and Traffic flow Environmental conditions and There is randomness, and a probability distribution is assigned to each parameter. Material constants follow a lognormal distribution, traffic flow follows a Poisson distribution, and environmental conditions follow a normal distribution. A second Monte Carlo simulation is performed, in which a prediction trajectory is obtained by solving the differential equation for each parameter sampled randomly from the parameter distribution. Based on the prediction trajectory, the prediction mean and the prediction standard deviation are calculated.

[0076] .

[0077] .

[0078] where is the prediction value of the jth simulation at time . The prediction uncertainty interval is defined as , which contains the true value with 95% probability.

[0079] The output of the time series damage evolution prediction module is the damage state distribution at future time, including the prediction mean and uncertainty interval of damage location, extent, and severity. This output serves as the input of the adaptive maintenance decision module and is fed back to the multi-scale feature hierarchy extraction module to adjust the sampling strategy for high-risk areas, realizing the logical level deep coupling and closed-loop feedback mechanism between modules.

[0080] Referring to Figure 5 , the adaptive maintenance decision module 4 includes a multi-objective modeling unit, a game solving unit, and a feedback control unit. The core innovation of this module is to build a multi-objective maintenance decision optimization model based on the game theory framework, to achieve a scientific trade-off between multiple mutually restrictive objectives such as maintenance cost, road performance, and traffic impact, and to realize the adaptive optimization of the detection system through a feedback control mechanism.

[0081] The multi-objective modeling unit builds a multi-objective optimization model that includes minimizing maintenance cost, maximizing road performance index, and minimizing traffic interruption time. Maintenance decision variables include maintenance time , maintenance method , and maintenance range . Maintenance methods include preventive maintenance (such as joint sealing, thin overlay), corrective maintenance (such as milling and repaving, pothole repair), and reconstruction. The cost, effect, and construction time of different maintenance methods are different.

[0082] The first objective is to minimize the total maintenance cost , including direct maintenance cost and indirect social cost : .

[0083] The direct maintenance cost is related to the maintenance method and the extent of the maintenance: .

[0084] where, is the unit area maintenance cost, for joint sealing $ / m 2 , for thin overlay $ / m 2 , and for mill and overlay $ / m 2 The indirect social cost mainly includes the cost of traffic delay, which is related to the construction time and the traffic volume: .

[0085] where, is the cost of delay per vehicle, is the average daily traffic volume, and is the construction time. The construction time is related to the maintenance method and the extent of the maintenance, which is days for joint sealing, days for thin overlay, and days for mill and overlay.

[0086] The second objective is to maximize the post-maintenance pavement condition index (PCI): (Pavement Condition Index): .

[0087] The PCI is determined by a comprehensive evaluation of the road roughness, rut depth, and crack rate, etc. The value of PCI ranges from 0 to 100, and the larger the value, the better the performance. The post-maintenance PCI is predicted by an empirical model: .

[0088] where, is the pre-maintenance PCI, is the maintenance method corresponding to the performance improvement, is the total area of the road segment. The performance improvement for joint sealing is , for thin overlay is , and for mill and overlay is .

[0089] The third objective is to minimize the traffic interruption time: : .

[0090] There is a conflict among the three objectives. Reducing maintenance cost usually means adopting low-cost preventive maintenance measures, but the performance improvement of such measures is limited. Improving road performance requires high-standard repair measures, which increases cost and construction time. Shortening traffic interruption time requires fast construction, but may affect maintenance quality. Therefore, trade-offs among multiple objectives are needed to find the Pareto optimal solution.

[0091] The constraints include that maintenance time must be before the predicted damage reaches the failure threshold, i.e. ; the maintenance range cannot exceed the total area of the road segment, i.e. ; and the performance index after maintenance must reach the minimum standard, i.e. .

[0092] The game solving unit converts the multi-objective optimization problem into a multi-party game model. In the game model, each objective corresponds to a game party (participant), and each objective function serves as the payoff function of the game party. The strategy space of the game party is the set of feasible maintenance schemes. The goal of the game is to find the Nash equilibrium point, at which any game party cannot improve its own payoff by changing its strategy unilaterally. The Nash equilibrium corresponds to the Pareto optimal solution, i.e., there is no other scheme that can improve other objectives without reducing a certain objective.

[0093] Let the three game parties be the cost minimization party P1, the performance maximization party P2, and the interruption time minimization party P3. The payoff functions of the game parties are , and , respectively. The negative sign indicates the conversion of the minimization problem into a maximization problem. The strategy space is all maintenance schemes that satisfy the constraints .

[0094] The Nash equilibrium point satisfies:

[0095] .

[0096] .

[0097] .

[0098] That is, under the condition that the strategies of other game parties are fixed, any game party cannot improve its own payoff by changing its strategy unilaterally.

[0099] The iterative optimal response algorithm is used to solve the Nash equilibrium. The basic idea of this algorithm is that in each iteration, each game party solves its optimal strategy under the condition that the strategies of other parties are fixed, until the strategies of all game parties converge to an equilibrium state. The specific steps are as follows:

[0100] Initialization: Randomly select an initial strategy ;

[0101] Iteration: For the k-th iteration, player P1 solves the optimization problem under the fixed conditions of : .

[0102] Player P2 solves the optimization problem under the fixed conditions of :

[0103] .

[0104] Player P3 solves the optimization problem under the fixed conditions of :

[0105] .

[0106] Convergence judgment: If and and , where is the convergence threshold, the algorithm terminates and outputs as the Nash equilibrium point; otherwise, let and continue iteration.

[0107] In typical scenarios, 10 to 20 iterations can converge to the Nash equilibrium. The Pareto optimal maintenance scheme obtained by the final solution achieves the best trade-off among the three objectives, controlling the cost, ensuring the performance, and minimizing the traffic impact.

[0108] The feedback control unit generates feedback control signals according to the Pareto optimal maintenance scheme, adjusting the image acquisition strategy and feature extraction weight of the multi-scale feature level extraction module. The core idea of feedback control is to increase the image acquisition frequency and feature extraction weight for road segments with rapidly growing predicted damage, achieving intensive monitoring of high-risk areas; for road segments with stable damage, reduce the sampling frequency to save computing resources and acquisition costs.

[0109] Specifically, according to the damage growth rate output by the time-series damage evolution prediction module, the road segments are divided into three risk levels. High-risk road segments are defined as mm / month, medium-risk road segments are defined as mm / month , and low-risk road segments are defined as For high-risk sections, the sampling frequency is set to once a week, and the feature extraction weight is increased to 1.5 times the standard value. For medium-risk sections, the sampling frequency is once a month, and the weight remains the standard value. For low-risk sections, the sampling frequency is once a quarter, and the weight is reduced to 0.5 times the standard value.

[0110] The feedback control signal is represented in matrix form:

[0111] .

[0112] wherein, is the total number of sections, is the sampling frequency of the i-th section, , and are the feature extraction weights of macro, meso and micro scales respectively. The feedback control signal is transmitted to the multi-scale feature level extraction module, which dynamically adjusts the image acquisition and feature extraction strategy, realizing the adaptive optimization of resources.

[0113] The four modules of the system form a closed-loop collaborative system through deep coupling, realizing mutual promotion, superposition of efficiency, and collaborative optimization of overall performance.

[0114] First, the multi-scale feature level extraction module 1 and the damage type intelligent identification module 2 form a parameter-level coupling. The geometric invariance feature vectors of each scale output by the multi-scale feature level extraction module 1 are directly used as the input of the damage type intelligent identification module 2, and the feature quality directly affects the recognition accuracy. The features extracted by topological manifold learning preserve the intrinsic geometric structure of the data, providing a high-quality basis representation for Lie group invariance coding. On the other hand, the accuracy of the recognition result of the damage type intelligent identification module 2 verifies the effectiveness of the feature extraction of the multi-scale feature level extraction module 1, and the two promote each other to form a positive feedback loop.

[0115] Second, the damage type intelligent identification module 2 and the time sequence damage evolution prediction module 3 form a state-level coupling. The damage feature descriptor output by the damage type intelligent identification module 2 provides the current state of the damage, including position, type, geometric parameters and severity, which are used as the initial conditions and boundary conditions for the dynamic modeling of module 3. The time sequence damage evolution prediction module 3 predicts the future damage distribution based on the current damage state and historical evolution trend. The combination of the two realizes superposition of efficiency, current state recognition + historical trend analysis → future prediction, and the prediction accuracy is significantly higher than that of a single method based on only the current state or historical trend.

[0116] Thirdly, the time sequence damage evolution prediction module 3 and the multi-scale feature level extraction module 1 form a logical level coupling and closed loop feedback. The high-risk area information predicted by the time sequence damage evolution prediction module 3 generates a feedback control signal through the adaptive maintenance decision module 4, which is transmitted to the multi-scale feature level extraction module 1 to adjust the sampling density and feature extraction weight of the high-risk area. This feedback mechanism realizes adaptive adjustment, and the system can dynamically optimize resource allocation according to the damage evolution trend, concentrating limited computing and acquisition resources in the areas most in need of attention, and improving overall monitoring efficiency.

[0117] Fourthly, the adaptive maintenance decision module 4 comprehensively utilizes the current damage state of the damage type intelligent identification module 2 and the future damage prediction of the time sequence damage evolution prediction module 3 to build a multi-objective optimization model and solve the optimal maintenance scheme. The scientificity of maintenance decision depends on accurate damage identification and reliable evolution prediction, and the three form a synergistic relationship to jointly support the development of preventive maintenance strategy. At the same time, the adaptive maintenance decision module 4 feeds back the decision result to the multi-scale feature level extraction module 1 through the feedback control unit, realizing decision-driven monitoring optimization and forming a complete closed loop of "monitoring→identification→prediction→decision→optimized monitoring".

[0118] Fifthly, the synergistic effect of the four modules is reflected in the nonlinear growth of the overall performance. The single feature extraction module can only provide image features, the single identification module can only judge the current damage, the single prediction module can only estimate the future trend, and the single decision module can only develop a scheme based on limited information. However, the deep coupling of the four modules makes the feature quality improve→the identification accuracy improve→the prediction reliability enhance→the decision scientificity improve→the monitoring efficiency optimize, forming a positive cycle and realizing 1+1>2 synergistic effect. The overall performance of the system far exceeds the sum of each module running independently, which reflects the advantages of the deep coupling closed loop synergistic system.

[0119] In one specific application example, a 6-month monitoring of a highway section is conducted. The initial monitoring finds multiple cracks on the section, and the multi-scale feature hierarchy extraction module extracts features at macro, meso and micro scales. The damage type intelligent identification module identifies 15 transverse cracks, 8 longitudinal cracks and 3 alligatoring cracks based on the Lie group invariance, and measures the length, width and depth of each crack. The time-series damage evolution prediction module establishes a differential equation model of damage evolution based on the historical data of the previous 3 months, and predicts the development trend of each crack in the next 3 months. The prediction result shows that the growth rate of 5 cracks exceeds 2 mm / month, which belongs to high-risk damage. The adaptive maintenance decision module solves a multi-objective optimization model based on the current damage state and evolution prediction to determine the optimal maintenance scheme: milling and re-paving for the 5 high-risk cracks, and sealing treatment for the remaining cracks, with a total maintenance cost of 180,000 yuan, an expected road performance index from 65 to 85 after maintenance, and a construction time of 7 days. At the same time, the feedback control unit increases the sampling frequency of the 5 high-risk areas from once a month to once a week, and the feature extraction weight to 1.5 times. The actual monitoring result after 3 months shows that the predicted damage development trend is consistent with the actual situation with a degree of 92%, and the road performance after maintenance meets the expectation, verifying the effectiveness of the system.

[0120] The system has significant technical advantages and application value. Compared with the prior art, the creativity lies in: using topological manifold learning and Lie group transformation theory to extract multi-scale features with geometric invariance, overcoming the defects of traditional methods sensitive to geometric transformation; based on the dynamic system theory, a damage evolution prediction model is established to accurately predict the future damage state, filling the gap in predictive analysis of the prior art; a multi-objective maintenance decision optimization framework based on game theory is constructed to achieve a scientific trade-off between cost, performance and traffic impact; a closed-loop adaptive optimization system is formed through a feedback control mechanism, significantly improving the intelligent level and operation efficiency. The deep coupling of the four modules produces synergistic effect, and the overall performance far exceeds that of the existing single-function system.

[0121] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modifications, equivalent replacements and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-level image intelligent damage assessment and prediction system for highway infrastructure, characterized in that, The method comprises the following steps: a multi-scale feature hierarchy extraction module is used to obtain a multi-resolution image sequence of the highway infrastructure, construct a macro, meso and micro scale feature representation space based on a topological manifold learning method, and map high-dimensional image features to a low-dimensional topological manifold through a manifold embedding algorithm to extract geometric invariance feature vectors at each scale; an intelligent damage type identification module is connected to the multi-scale feature hierarchy extraction module, used to receive the geometric invariance feature vectors at each scale, construct a rotation translation scale invariance representation of the damage mode based on Lie group transformation theory, identify five types of damage, i.e. cracks, pits, subsidence, spalling and net cracking, through orbit decomposition of the transformation group, and generate a damage feature descriptor containing damage location, type, geometric parameters and severity; a time series damage evolution prediction module is connected to the intelligent damage type identification module, used to receive the damage feature descriptor and historical image sequence, establish a differential equation model of damage propagation based on dynamic system theory, extract the dynamics of damage evolution through phase space reconstruction, calculate the damage growth rate, expansion direction and stability index, and predict the damage state distribution at a future time; an adaptive maintenance decision module is connected to the intelligent damage type identification module and the time series damage evolution prediction module, used to receive the damage feature descriptor and the damage state distribution, construct a multi-objective optimization model containing maintenance cost, road performance and traffic impact based on game theory framework, solve the Nash equilibrium strategy to determine the optimal maintenance scheme, and generate a feedback control signal to adjust the image acquisition strategy and feature extraction weight of the multi-scale feature hierarchy extraction module according to the optimal maintenance scheme, forming a closed-loop adaptive optimization system.

2. The system of claim 1, wherein, The multi-scale feature hierarchy extraction module comprises: an image acquisition unit is used to obtain multi-source image data of the highway infrastructure through a visible light camera, an infrared camera and a laser scanning device, and construct an image pyramid of macro scale pixel level images, meso scale millimeter level images and micro scale micron level images; a topological feature extraction unit is used to perform topological manifold learning on each level image of the image pyramid, establish a manifold neighborhood graph by calculating a geodesic distance matrix, project the image data to a low-dimensional manifold space using a Laplace eigenmap algorithm, and extract topological features that maintain local geometric structure; a scale fusion unit is used to weight and fuse topological features at each scale, adaptively adjust the fusion weight of each scale feature according to the feature response intensity of the damage type, and generate a multi-scale fusion feature vector.

3. The system of claim 1, wherein, The intelligent damage type identification module comprises: a Lie group invariance encoding unit is used to perform Lie group transformation on the geometric invariance feature vector, calculate the invariants of the feature under the action of the rotation group, translation group and scale transformation group, and construct a damage mode representation with geometric invariance; a damage mode classification unit is used to classify similar damage modes into the same orbit through orbit decomposition of the transformation group based on the damage mode representation, measure the distance between different orbits using a kernel function, and realize automatic classification of the damage type. The damage quantification evaluation unit is configured to perform geometric parameter measurement on the classified damage, calculate length, width, depth, area and volume geometric features of the damage, evaluate damage severity and generate a damage grade label.

4. The system of claim 1, wherein, The time sequence damage evolution prediction module comprises: The phase space reconstruction unit is configured to perform phase space reconstruction on the time sequence of damage features in the historical image sequence, determine embedding dimension and time delay parameters through the delay embedding theorem, and restore a dynamic trajectory of damage evolution in the phase space. The dynamic modeling unit is configured to establish a differential equation model of damage propagation based on the dynamic trajectory, identify driving factors affecting damage evolution, including traffic load, environmental temperature and humidity, and material aging degree, and quantify contribution weights of each factor to damage growth rate. The prediction calculation unit is configured to solve the differential equation model, calculate damage state at a future time through a numerical integration method, generate a spatiotemporal distribution prediction result containing damage position, range and severity, and calculate a prediction uncertainty interval.

5. The system of claim 1, wherein, The adaptive maintenance decision module comprises: The multi-objective modeling unit is configured to construct a multi-objective optimization model containing minimized maintenance cost, maximized road performance index and minimized traffic interruption time, set constraint conditions and weight coefficients of each objective. The game solving unit is configured to convert the multi-objective optimization model into a multi-party game model, where each maintenance strategy is a game party and each objective function is a payoff function, and determine a Pareto optimal maintenance scheme by solving a Nash equilibrium point through an iterative algorithm. The feedback control unit is configured to generate a feedback control signal according to the Pareto optimal maintenance scheme, adjust sampling density and feature extraction weight of the multi-scale feature level extraction module on a high-risk area, and realize adaptive optimization configuration of resources.

6. The system of claim 1, wherein, When the multi-scale feature level extraction module maps high-dimensional image features to a low-dimensional topological manifold through a manifold embedding algorithm, a geodesic distance-based isometric mapping method is adopted to keep intrinsic geometric distance between data points in a high-dimensional space unchanged in a low-dimensional space, ensuring geometric fidelity of feature representation.

7. The system of claim 1, wherein, When the damage type intelligent identification module identifies crack type damage, principal curvatures and geodesic curvatures of a damage contour are calculated to distinguish transverse cracks, longitudinal cracks, diagonal cracks, crazing and net-shaped cracks.

8. The system of claim 1, wherein, When the time sequence damage evolution prediction module establishes a differential equation model of damage propagation, a material fatigue cumulative damage theory is introduced to quantify contribution of cyclic load to damage growth through a Miner linear cumulative damage criterion.

9. The system of claim 1, wherein, When the adaptive maintenance decision module solves a Nash equilibrium point, an iterative optimal response algorithm is adopted to solve an optimal strategy of each game party under a fixed strategy of other parties in each iteration until strategies of all game parties converge to an equilibrium state.

10. The system of claim 1, wherein, The closed-loop adaptive optimization system adjusts image acquisition strategy in real time through the feedback control signal, increases sampling frequency on a road section where a prediction result shows rapid damage growth in the future, and reduces sampling frequency on a road section where damage is stable, thereby realizing dynamic optimization configuration of computing resources and acquisition resources.

Images