Multi-level image intelligent damage assessment and prediction system for road infrastructure
By combining multi-scale feature extraction, Lie group invariance identification, and dynamic system prediction with game theory optimization, the single-scale problem of damage detection in highway infrastructure is solved, enabling comprehensive damage assessment and prediction, optimizing maintenance decisions, and forming an intelligent adaptive system.
Patent Information
- Application Number
- CN202511726918.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-24
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2045-11-24
Smart Images

Figure CN121191099A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing and artificial intelligence, and particularly relates to a multi-level image intelligent damage assessment and prediction system for highway infrastructure, which is suitable for health monitoring and preventive maintenance management of highway pavement, bridges, tunnels and other infrastructure. BACKGROUND
[0002] As an important part of the transportation system, the operation state of highway infrastructure directly affects traffic safety and transportation efficiency. With the continuous expansion of the highway network scale and the growth of the service life, the infrastructure damage problem is increasingly prominent. Traditional manual inspection methods have defects such as low efficiency, high cost, and strong subjectivity, which are difficult to meet the needs of large-scale road network management. In recent years, automatic damage detection technology based on image recognition has been widely studied, but the existing technology still has many shortcomings.
[0003] The existing image recognition method mainly adopts single-scale feature extraction, which cannot comprehensively capture damage features at different scales. Macroscopic images can reflect the overall condition of the pavement, but have limited ability to identify fine cracks; microscopic images can clearly show the material surface texture, but lack overall spatial information. This single-scale analysis method leads to insufficient recognition accuracy and is prone to missed detection and false detection.
[0004] A Chinese invention patent with the authorized publication number CN115641334B discloses a road anti-collision facility damage identification method, which identifies the anti-collision barrel through an object recognition model and judges the damage by using the reflective area and the reflective intensity. This method mainly targets reflective film facilities and judges whether there is damage by comparing the reflective characteristics with a preset threshold. Although this method has certain effect in specific scenarios, it has obvious limitations. First, this method only judges damage based on reflective characteristics, which is difficult to accurately evaluate facilities with non-reflective materials or degraded reflective performance. Second, this method only considers the damage state at the current moment and lacks the ability to predict the damage evolution trend, which cannot support preventive maintenance decisions. Third, this method uses fixed threshold judgment and fails to adaptively adjust according to environmental conditions and facility types, which lacks robustness in complex scenarios. Fourth, this method only focuses on single facility damage identification and does not consider the damage correlation and systematic degradation rules of multiple facilities in the region.
[0005] Most of the existing damage detection methods focus on the static identification of damage, lacking dynamic analysis of the damage evolution process and prediction of the future state. Pavement damage is a gradual development process, and early minor cracks will gradually expand under the action of traffic load and environmental factors, eventually evolving into serious structural damage. If maintenance is only carried out when the damage is obvious, not only the maintenance cost is high, but also it may affect traffic safety. Therefore, it is of great significance to establish a damage evolution prediction model to accurately predict the damage development trend for formulating preventive maintenance strategies.
[0006] The existing technology mainly adopts traditional convolutional neural network in feature extraction. Although this method has achieved good results in target detection tasks, it is often difficult to extract essential features with geometric invariance when dealing with damage images with complex geometric structures. Damage presents different image features under different viewing angles, lighting conditions and collection distances. Traditional methods lack robustness to such geometric transformations, leading to performance degradation in practical applications.
[0007] Most of the existing maintenance decision methods are based on single objective optimization, such as only considering minimizing maintenance cost or maximizing road performance, without considering multiple mutually restrictive objectives such as cost, performance, and traffic impact. In practical applications, maintenance decision is a multi-objective optimization problem that needs to be balanced between different objectives. In addition, existing methods lack adaptive adjustment mechanism for detection system, and cannot dynamically optimize resource allocation according to damage distribution and evolution trend.
[0008] In view of the above problems, it is urgent to develop a multi-level, intelligent and predictive highway infrastructure damage assessment and prediction system, which can comprehensively analyze damage features from multiple scales, accurately predict damage evolution trend, and realize adaptive maintenance decision optimization, providing technical support for scientific management of highway infrastructure. SUMMARY
[0009] The purpose of the present application is to overcome the shortcomings of the prior art and provide a multi-level image intelligent damage assessment and prediction system for highway infrastructure. Through multi-scale feature level extraction, damage identification based on Lie group invariance, damage prediction driven by dynamic system theory, and maintenance decision optimization under the framework of game theory, the system realizes comprehensive assessment, accurate prediction and intelligent decision of highway infrastructure damage, forming a deep coupling closed-loop adaptive optimization system.
[0010] To achieve the above object, the application adopts the following technical solutions. The application provides a highway infrastructure multi-level image intelligent damage assessment and prediction system, which comprises a multi-scale feature level extraction module, an intelligent damage type identification module, a time sequence damage evolution prediction module and a self-adaptive maintenance decision module. The multi-scale feature level extraction module acquires a multi-resolution image sequence of the highway infrastructure, constructs a feature representation space of three scales of macro, meso and micro based on a topological manifold learning method, maps high-dimensional image features to low-dimensional topological manifolds through a manifold embedding algorithm, and extracts geometric invariance feature vectors of each scale. The intelligent damage type identification module receives the geometric invariance feature vectors of each scale, constructs a rotation translation scale invariance representation of a damage mode based on a Lie group transformation theory, identifies a damage type through orbit decomposition of a transformation group, and generates a damage feature descriptor. The time sequence damage evolution prediction module receives the damage feature descriptor and a historical image sequence, establishes a differential equation model of damage propagation based on a dynamic system theory, and predicts a damage state distribution at a future time. The self-adaptive maintenance decision module receives the damage feature descriptor and the damage state distribution, constructs a multi-objective optimization model based on a game theory framework, solves an optimal maintenance scheme, generates a feedback control signal to adjust an image acquisition strategy and a feature extraction weight of the multi-scale feature level extraction module, and forms a closed-loop self-adaptive optimization system.
[0011] The beneficial effects of the present application are as follows: first, by the multi-scale feature level extraction module, the image information of three scales of macro, meso and micro is fused, the damage characteristics of different levels are comprehensively captured, the limitations of single scale analysis are overcome, and the comprehensiveness and accuracy of damage identification are significantly improved. Second, the topological manifold learning method is used to extract image features, the intrinsic geometric structure of the data is maintained, the low-dimensional feature representation with geometric fidelity is obtained, and compared with the traditional Euclidean space feature extraction method, the expression ability and robustness are stronger when dealing with complex geometric structure of damage image. Third, based on the Lie group transformation theory, the invariance representation of damage mode is constructed, the robust recognition of damage under different angles, illumination and scales is realized, the defects of sensitivity to geometric transformation of existing methods are overcome, and the adaptability of the system in complex environment is improved. Fourth, the dynamic system theory is introduced to establish the damage evolution prediction model, the dynamics mechanism of damage propagation is revealed through phase space reconstruction, the accurate prediction of future damage state is realized, and a scientific basis is provided for preventive maintenance. Fifth, a multi-objective maintenance decision optimization framework based on game theory is constructed, scientific trade-off is realized among maintenance cost, road performance and traffic influence and other multiple objectives, the Pareto optimal maintenance scheme is obtained, and the rationality and economy of decision-making are improved. Sixth, the adaptive optimization of the detection system is realized through the feedback control mechanism, the sampling strategy and resource allocation are dynamically adjusted according to the damage distribution and evolution prediction result, the closed-loop optimization system is formed, and the intelligent level and operation efficiency of the system are significantly improved. Seventh, the deep coupling and collaborative relationship is formed among the modules, the multi-scale feature extraction provides high-quality input for damage identification, the results of damage identification and evolution prediction jointly drive the maintenance decision, and the feedback of maintenance decision optimizes the feature extraction process, the synergistic effect of 1+1>2 is realized, and the overall system performance is much higher than the sum of the independent operation of each module. BRIEF DESCRIPTION OF DRAWINGS
[0012] Figure 1 Fig. 1 is a schematic diagram of the overall structure of the present application; Figure 2 Fig. 2 is a detailed structure diagram of the multi-scale feature level extraction module of the present application; Figure 3 Fig. 3 is a detailed structure diagram of the intelligent damage type identification module of the present application; Figure 4 Fig. 4 is a detailed structure diagram of the time sequence damage evolution prediction module of the present application; Figure 5 Fig. 5 is a detailed structure diagram of the adaptive maintenance decision module of the present application. DETAILED DESCRIPTION
[0013] The present application will be described in detail below in combination with the drawings and specific embodiments.
[0014] Referring to Figure 1 The highway infrastructure multi-level image intelligent damage assessment and prediction system comprises a multi-scale feature level extraction module 1, a damage type intelligent recognition module 2, a time sequence damage evolution prediction module 3 and a self-adaptive maintenance decision module 4. The four modules form a deep-coupled closed-loop collaborative system, and through multi-level coupling of the parameter level, the state level and the logic level, the system performance of mutual promotion, superposition of efficiency, and collaborative optimization is realized.
[0015] Referring to Figure 2 The multi-scale feature level extraction module 1 comprises an image acquisition unit, a topological feature extraction unit and a scale fusion unit. The core innovation of this module is to use topological manifold learning method to extract damage features with geometric invariance from multiple scales, providing high-quality feature representation for subsequent damage recognition and prediction.
[0016] The image acquisition unit obtains multi-source image data of highway infrastructure through visible light cameras, infrared cameras and laser scanning devices. The visible light camera obtains natural light images of the road surface, with a resolution of 4096x2160 pixels and a frame rate of 60fps, which can clearly capture the overall condition and obvious damage of the road surface. The infrared camera works in the 8-14 μm wave band and can identify hidden damages such as internal cavities and delamination by detecting the temperature distribution of the road surface. The laser scanning device uses a line laser method with a scanning frequency of 2 kHz, which can obtain three-dimensional topographic data with an accuracy of 0.1 mm for precise measurement of crack width and depth. Through the fusion of multi-source images, an image pyramid containing macro-scale pixel-level images, meso-scale millimeter-level images and micro-scale micron-level images is constructed. The field of view of the macro image is 120°, covering an area of 4m wide, which is used to evaluate the overall condition of the road section. The field of view of the meso image is 30°, covering an area of 1m wide, which is used to identify typical damage types. The field of view of the micro image is 5°, covering an area of 10cm wide, which is used to analyze the material surface texture and micro cracks.
[0017] The topological feature extraction unit performs topological manifold learning on the images at each level of the image pyramid to extract topological features that preserve local geometric structure. This unit first pre-processes the images at each scale, including denoising, contrast enhancement and edge detection. Non-local mean filtering is used for denoising, which can remove noise while preserving edge information. Adaptive histogram equalization is used for contrast enhancement, which automatically adjusts the image contrast for different lighting conditions. Canny operator is used for edge detection, which extracts damage edges through double-threshold detection. The pre-processed images are converted into high-dimensional feature vectors, where each pixel point's features include grayscale value, gradient amplitude, gradient direction, local binary pattern and Gabor filter response, forming a feature space with dimension D=128.
[0018] In a high-dimensional feature space, the topological feature extraction unit applies a topological manifold learning method to map the data to a low-dimensional topological manifold. This process is based on the manifold assumption, that is, high-dimensional image data actually lies on a low-dimensional manifold. First, it computes any two points in the feature space... and The geodesic distance between points is defined by the shortest path length on a manifold. Unlike the straight-line distance in Euclidean space, geodesic distance more accurately reflects the intrinsic geometric structure of the data. The geodesic distance is calculated using Dijkstra's shortest path algorithm on a k-nearest neighbor graph. For each data point, k=15 nearest neighbors are selected to construct a neighborhood graph, where the edge weights between adjacent points are their Euclidean distances. By calculating the shortest path on the neighborhood graph, the geodesic distance matrix between any two points is obtained. ,in Point and The geodesic distance between them.
[0019] Based on the geodesic distance matrix, the Laplacian eigenmap algorithm is used to project the data into a low-dimensional manifold space. This algorithm achieves dimensionality reduction by minimizing the following objective function: .
[0020] in, For low-dimensional embedding coordinate matrices, Let N be the coordinates of the i-th data point in the d-dimensional manifold space, where d = 32 is the dimension of the manifold, and N is the total number of data points. The adjacency weight matrix is defined as follows: .
[0021] in, express The set of k nearest neighbors, The Gaussian kernel width parameter is adaptively determined based on the local neighborhood density, and the calculation formula is as follows: The physical meaning of this objective function is to maintain the local neighborhood relationships in the high-dimensional space in the low-dimensional space, so that originally adjacent data points are still close in the low-dimensional space.
[0022] The optimal solution to the objective function is obtained by solving the generalized eigenvalue problem: .
[0023] in, For the graph Laplace matrix, Let be a degree matrix, satisfying , Let d be the eigenvalues. Select the eigenvectors corresponding to the d smallest eigenvalues (excluding the smallest eigenvalue, which is usually 0) to construct a low-dimensional embedding coordinate matrix. These eigenvectors correspond to a global coordinate system on the manifold and can reveal the intrinsic topological structure of the data.
[0024] Through the manifold learning process described above, the high-dimensional features of images at each scale are mapped to a low-dimensional topological manifold space, resulting in macroscopic features. Mesoscopic characteristics and microscopic features These features preserve the geometric structure of the original image while achieving dimensionality reduction, thus lowering the computational complexity of subsequent processing.
[0025] The scale fusion unit weights and fuses the topological features at each scale to generate a multi-scale fused feature vector. The fusion process is based on an attention mechanism, adaptively adjusting the fusion weights according to the feature response intensity of different damage types at each scale. The formula for calculating the multi-scale fused feature vector is: .
[0026] in, , and The fusion weights for features at each scale satisfy the following conditions: and The fusion weights are calculated using the softmax function: .
[0027] Where s represents the scale type. The characteristic response intensity at scale *s* is determined by calculating the L2 norm of the eigenvector, i.e. , The temperature coefficient is used to control the smoothness of the weight distribution; preferably, it is taken as... , , For crack-type damage, the characteristic response at the mesoscale is the strongest, therefore Typically, the response is largest at the macroscopic scale. For pit-type damage, the characteristic response at the macroscopic scale is strongest, therefore... Typically the largest. For material exfoliation, the characteristic response at the microscale is the strongest, therefore... Typically, the weights are the largest. Through this adaptive fusion mechanism, the system can automatically adjust the weights of features at each scale according to the damage type, thereby improving the discriminative power of the feature representation.
[0028] The output of the multi-scale feature extraction module is the geometrically invariant feature vector at each scale. , , and multi-scale fused feature vectors These feature vectors serve as input to the damage type intelligent identification module, achieving deep parameter-level coupling between modules.
[0029] Reference Figure 3The damage type intelligent identification module 2 includes a Lie group invariant encoding unit, a damage pattern classification unit, and a damage quantification assessment unit. The core innovation of this module lies in constructing a geometrically invariant representation of damage patterns based on Lie group transformation theory, enabling robust identification of damage under different viewpoints, illumination, and scales.
[0030] The Lie group invariant coding unit performs a Lie group transformation on the geometrically invariant feature vectors output by the multi-scale feature hierarchical extraction module 1, calculating the invariants of the features under the influence of rotation, translation, and scale transformation groups. Lie groups are mathematical objects that possess both group and manifold structures, capable of describing geometric transformations of images. In damage recognition tasks, changes in camera viewpoint correspond to rotation transformations, changes in acquisition position correspond to translation transformations, and changes in acquisition distance correspond to scale transformations. By constructing feature representations invariant to these transformations, the influence of geometric transformations on the recognition results can be eliminated.
[0031] Let the eigenvectors be... In Li Qun Transformed into under the action of ,in For group elements. Lie group invariants. satisfy That is, it remains invariant under group transformations. For rotation groups... The invariants are constructed using orbital integrals: .
[0032] in, The rotation angle is The rotation matrix, The feature extraction function is integrated over a rotation group. In practical calculations, a discretization approximation is used, and [the appropriate function is selected]. A uniformly distributed rotation angle ,in ,calculate: .
[0033] For translation groups The invariants are obtained by calculating the Fourier transform amplitude spectrum of the characteristic. The Fourier transform amplitude spectrum is invariant to translation because translation only changes the phase and not the amplitude. The translation invariant is defined as: .
[0034] in, Indicates Fourier transform, This indicates the magnitude. For the scaling transformation group... The invariants are constructed through logarithmic polar coordinate transformation. In logarithmic polar coordinates, scaling transformation corresponds to translation, and therefore can be handled in the same way as translation invariants.
[0035] By combining rotation, translation, and scaling transformations, a joint invariant is constructed: .
[0036] in, This is a scale invariant, with the superscript T indicating transpose. This joint invariant remains unchanged under rotation, translation, and scaling transformations, enabling robust damage pattern recognition.
[0037] The damage pattern classification unit classifies damage types based on Lie group invariants. The classification process employs the orbital decomposition theory of transformation groups. Under the action of the Lie group, the feature space is divided into several orbits, each containing all feature vectors that can be interconverted through group transformations. Feature vectors belonging to the same orbit correspond to observations of the same damage pattern at different viewpoints, locations, and scales. By classifying feature vectors into different orbits, automatic damage type classification is achieved.
[0038] set up Representing the eigenvector The orbit is defined as: .
[0039] Two orbits and The distance between them is measured using the Hausdorff distance metric: .
[0040] in, Point to orbit The distance is defined as In practical calculations, since the orbits are a continuous set, directly calculating the Hausdorff distance is difficult. A simplified method based on Lie group invariants is adopted, directly calculating the distance between the invariants as an approximation of the orbital distance: .
[0041] Based on orbital distance, a kernel k-means clustering algorithm is used to classify damage patterns. The kernel function chosen is the Gaussian radial basis function. .
[0042] in, The kernel width parameter is used. The number of clusters is set to K=5, corresponding to the five damage types: cracks, pits, subsidence, spalling, and network cracks. For the new feature vector... The category label is determined by calculating the kernel distance to each cluster center: .
[0043] in, The center of the k-th cluster is determined using training data. For crack-type damage, the principal curvature and geodesic curvature of the damage profile are further calculated to distinguish transverse cracks, longitudinal cracks, oblique cracks, crazing, and network cracks. Principal curvature and The geodesic curvature was obtained through differential geometric analysis of the damage profile. It is determined by calculating the rate of change of the contour tangent vector. and At that time, it was determined to be a transverse or longitudinal crack, further distinguished by the crack's direction and angle. and At that time, it was determined to be a crack or a network of cracks.
[0044] The damage quantification assessment unit measures the geometric parameters of the classified damage, calculating its length, width, depth, area, and volume. For crack-type damage, the length is calculated by integrating along the crack centerline, the width is determined by measuring the maximum lateral distance perpendicular to the centerline, and the depth is obtained from the 3D topographic data of the laser scanning equipment. For pit-type damage, the area is determined by counting the number of pixels in the damaged area and converting it based on pixel size, and the volume is calculated by integrating the 3D topographic data. Based on the geometric parameters, the severity of the damage is assessed and a damage level label is generated. The damage level is divided into four levels: mild, moderate, severe, and critical. For cracks, the level criteria are: mild cracks with a width less than 3mm and a length less than 1m; moderate cracks with a width of 3mm to 5mm and a length of 1m to 3m; severe cracks with a width of 5mm to 10mm and a length of 3m to 10m; and critical cracks with a width greater than 10mm or a length greater than 10m. For pits, the level criteria are: mild pits with a depth less than 20mm and an area less than 0.1m². 2; Moderate potholes have a depth of 20mm to 40mm and an area of 0.1m². 2 up to 0.5m 2 Severe potholes have a depth of 40mm to 60mm and an area of 0.5m². 2 Up to 1m 2; Severe potholes with a depth greater than 60mm or an area greater than 1m² 2 .
[0045] The output of the damage type intelligent identification module is a damage feature descriptor, including the damage location. ,type Geometric parameters and severity level ,in Let be the length, w be the width, h be the depth, A be the area, and V be the volume. This descriptor serves as the input to the temporal damage evolution prediction module and also provides the current damage state information to the adaptive maintenance decision module, achieving deep state-level coupling between modules.
[0046] Reference Figure 4 The temporal damage evolution prediction module 3 includes a phase space reconstruction unit, a dynamic modeling unit, and a prediction calculation unit. The core innovation of this module lies in establishing a differential equation model of damage propagation based on dynamical system theory, revealing the dynamic mechanism of damage evolution through phase space reconstruction, and achieving accurate prediction of future damage states.
[0047] The phase space reconstruction unit performs phase space reconstruction on the time series of damage features in historical image sequences, revealing the dynamic trajectory of damage evolution. Phase space reconstruction is based on Takens' delayed embedding theorem, which states that for a deterministic dynamical system, the delayed coordinate vector of its univariate time series can reconstruct the attractor of the original system, thereby recovering the system's dynamic characteristics.
[0048] Let the time series of a certain geometric parameter of the damage (such as crack width) be... ,in Let be the i-th sampling time. The delayed embedding vector is defined as: .
[0049] in, For time delay, The embedding dimension. Delayed embedding vector. The time delay constitutes a point in the m-dimensional phase space, and the set of all delay vectors describes the dynamic trajectory of the system. Determined through mutual information. Mutual information. Measures the time series in delay Information relevance afterward: .
[0050] in, For joint probability distribution, and The marginal probability distribution is used. The delay corresponding to the first local minimum is selected as the embedding delay. This delay ensures sufficient time for correlation while avoiding information loss. In embodiments of the present invention, based on the evolution characteristics of road surface damage, the time delay is typically taken as... Heavenly sky.
[0051] The embedding dimension *m* is determined using the false nearest neighbor method. The false nearest neighbor method is based on the idea that if the embedding dimension is insufficient, some seemingly adjacent points in phase space are actually far apart on the real attractor; these are false nearest neighbors caused by dimension projection. By gradually increasing the embedding dimension, the proportion of false nearest neighbors is calculated. When the proportion of false nearest neighbors drops below a threshold, the embedding dimension is considered sufficient. The criterion for judging false nearest neighbors is: .
[0052] in, This represents the delay vector in m-dimensional space. for The nearest neighbor, This is the threshold. If... Then it is believed yes The number of false nearest neighbors is determined. When the proportion of false nearest neighbors is less than 1%, the dimension is no longer increased, and the dimension at this point is the optimal embedding dimension m. In embodiments of this invention, the embedding dimension is typically [value missing]. to .
[0053] After determining the time delay and embedding dimension using the methods described above, phase space reconstruction is performed on multiple geometric parameters of the damage (crack width, length, area, etc.) to recover the dynamic trajectory of damage evolution in an m-dimensional phase space. The phase space trajectory reveals the law of damage evolution, providing a foundation for subsequent dynamic modeling.
[0054] The dynamic modeling unit establishes a differential equation model for damage propagation based on phase space trajectories. Damage evolution is influenced by various factors, including traffic load, environmental temperature and humidity, and material aging. Based on the theory of cumulative damage from material fatigue, a differential equation describing damage growth is established.
[0055] For crack-type damage, its propagation process follows the Paris-Erdogan law, which describes the growth of fatigue cracks under cyclic loading. Crack length. With the number of loops The changes satisfy: .
[0056] in, This represents the stress intensity factor amplitude. and This is a material constant. The stress intensity factor amplitude and stress amplitude are related. and crack length Related: .
[0057] in, This is a geometric correction factor, related to the crack shape and loading method. For surface cracks, the geometric correction factor is... Substituting the stress intensity factor expression into Paris's law, we get: .
[0058] This is about the crack length. The differential equation. In practical applications, the number of iterations... With time The relationship is determined through traffic flow. Let the average daily vehicle traffic volume be... The number of cycles per unit time is Using the chain rule, the derivative of the crack length with respect to the number of cycles is converted to its derivative with respect to time: .
[0059] In an embodiment of the present invention, based on fatigue test data of asphalt pavement materials, the material constant is... , Stress amplitude Pavement structure analysis determined that the stress is related to vehicle axle load and pavement modulus. For asphalt pavement under a standard axle load (100kN), the stress amplitude is approximately... MPa. Average daily traffic flow Based on traffic survey data, the typical value for highways is... to Vehicles / day.
[0060] Besides cyclic loading, environmental factors also significantly influence damage evolution. Temperature changes cause thermal expansion and contraction of materials, generating thermal stress that accelerates crack propagation. Humidity changes affect the viscoelastic properties of materials, reducing fatigue life. To account for the impact of environmental factors, an environmental correction factor is introduced. ,in For temperature, For humidity. The environmental correction factor was determined through regression analysis: .
[0061] in, ℃ and % represents reference temperature and humidity. / ℃ and / % represents the influence coefficient of temperature and humidity. Taking into account both traffic load and environmental factors, the differential equation model for damage evolution is as follows: This equation describes the variation of crack length over time, revealing the influence of traffic load, environmental conditions, and the current state of the crack on the evolution rate. Similar differential equation models are established for other types of damage, such as potholes and spalling, taking into account the corresponding physical mechanisms and influencing factors.
[0062] Furthermore, the Miner linear cumulative damage criterion is introduced to quantify the contribution of cyclic loading to damage growth. This criterion assumes that the fatigue life of the material accumulates linearly under different stress levels, and material failure occurs when the cumulative damage reaches 1. (Cumulative damage) The calculation formula is: .
[0063] in, Stress level The number of loops below, Stress level The fatigue life is below, Number of stress level types. Fatigue life. Determined through the SN curve. For asphalt concrete, the empirical formula for the SN curve is: .
[0064] in, and These are material parameters. The cumulative damage criterion can be used to assess the remaining life of the pavement, providing a basis for maintenance decisions.
[0065] The predictive computation unit calculates the damage state at future time steps by solving a differential equation model. Numerical integration is performed using the fourth-order Runge-Kutta method. For the differential equation... The iterative formula for the Runge-Kutta method is: .
[0066] in, For time step, , , , Time step The prediction timeframe is 180 days from now. Iterative calculations are used to obtain the predicted crack length at various future moments. .
[0067] To quantify the uncertainty of the prediction, the Monte Carlo simulation method is employed. This takes into account model parameters (such as material constants). and Traffic flow Environmental conditions and The problem involves randomness, and a probability distribution is assigned to each parameter. Material constants follow a log-normal distribution, traffic flow follows a Poisson distribution, and environmental conditions follow a normal distribution. Each Monte Carlo simulation randomly samples from the parameter distribution and solves the differential equation to obtain a predicted trajectory. Based on Calculate the predicted mean of the trajectory. and the predicted standard deviation : .
[0068] .
[0069] in, For the j-th simulation at time... The predicted value. The prediction uncertainty interval is defined as... This interval contains the true value with a 95% probability.
[0070] The output of the temporal damage evolution prediction module is the damage state distribution at future time points, including the predicted mean and uncertainty interval of damage location, extent, and severity. This output serves as the input to the adaptive maintenance decision module and is also fed back to the multi-scale feature extraction module to adjust the sampling strategy for high-risk areas, thus achieving deep logical coupling and a closed-loop feedback mechanism between modules.
[0071] Reference Figure 5 The adaptive maintenance decision module 4 includes a multi-objective modeling unit, a game theory solving unit, and a feedback control unit. The core innovation of this module lies in constructing a multi-objective maintenance decision optimization model based on a game theory framework. This model achieves a scientific trade-off among multiple mutually constraining objectives such as maintenance costs, road performance, and traffic impact, and realizes adaptive optimization of the detection system through a feedback control mechanism.
[0072] The multi-objective modeling unit constructs a multi-objective optimization model that minimizes maintenance costs, maximizes the road performance index, and minimizes traffic downtime. Maintenance decision variables include maintenance time. Maintenance methods and maintenance scope Maintenance methods include preventative maintenance (such as sealing cracks and applying thin overlays), corrective repairs (such as milling and repaving, and pothole repair), and reconstruction. The costs, effectiveness, and construction time vary depending on the maintenance method.
[0073] The first goal is to minimize total maintenance costs. Including direct maintenance costs and indirect social costs : .
[0074] Direct maintenance costs are related to the maintenance methods and scope: .
[0075] in, For maintenance cost per unit area, for sealing Yuan / m 2 For thin-layer coatings Yuan / m 2 For milling redigradation Yuan / m 2 Indirect social costs mainly include traffic delay costs, which are related to construction time and traffic flow: .
[0076] in, The cost per vehicle is the delay cost per vehicle. Daily traffic flow The construction time is related to the maintenance method and scope; the sealing construction time is... The thin-layer overlay construction time is [day]. The milling and repaving construction time is [number] days. sky.
[0077] The second objective is to maximize the road performance index after maintenance. (Pavement Condition Index): .
[0078] The road performance index is determined by comprehensively evaluating indicators such as pavement smoothness, rut depth, and crack rate. The value ranges from 0 to 100, with higher values indicating better performance. The post-maintenance performance index is predicted using an empirical model. .
[0079] in, Performance index before maintenance. For maintenance method The corresponding performance improvement This represents the total area of the road section. The performance improvement of the joint sealing is... The performance of the thin-layer coating is improved to The performance of milling and resurfacing is improved to .
[0080] The third objective is to minimize traffic disruption time. : .
[0081] There are conflicts among the three objectives. Reducing maintenance costs often means adopting low-cost preventative maintenance measures, but these measures offer limited performance improvements. Improving road performance requires high-standard maintenance measures, but this increases costs and construction time. Reducing traffic downtime requires rapid construction, but this may compromise maintenance quality. Therefore, trade-offs need to be struck among these objectives to find a Pareto optimal solution.
[0082] The constraints include: maintenance must be completed before the predicted damage reaches the failure threshold, i.e. The maintenance scope cannot exceed the total area of the road section, i.e. The performance index after maintenance must meet the minimum standard, that is... .
[0083] The game theory solving unit transforms the multi-objective optimization problem into a multi-party game model. In this model, each objective corresponds to a player, and each objective function serves as that player's payoff function. The player's strategy space is a set of feasible maintenance strategies. The goal of the game is to find a Nash equilibrium, where no player can improve their payoff by unilaterally changing their strategy. Nash equilibrium corresponds to Pareto optimality, meaning there is no other solution that can improve other objectives without decreasing one.
[0084] Let the three players be P1, who minimizes cost; P2, who maximizes performance; and P3, who minimizes downtime. The payoff functions for each player are as follows: , and The negative sign indicates that the minimization problem is transformed into a maximization problem. The policy space represents all maintenance schemes that satisfy the constraints. .
[0085] Nash equilibrium point satisfy: .
[0086] .
[0087] .
[0088] That is, given that the strategies of other players are fixed, no player can increase their own payoff by unilaterally changing their strategy.
[0089] An iterative optimal response algorithm is used to solve for the Nash equilibrium. The basic idea of this algorithm is that in each iteration, each player sequentially solves for their own optimal strategy while keeping the strategies of the other players fixed, until all players' strategies converge to an equilibrium state. The specific steps are as follows: Initialization: Randomly select an initial strategy ; Iteration: For the k-th iteration, player P1 in Solving for the optimal solution under fixed conditions : .
[0090] Player P2 in Solving for the optimal solution under fixed conditions : .
[0091] Player P3 in Solving for the optimal solution under fixed conditions : .
[0092] Convergence criterion: If and and ,in If the threshold for convergence is reached, the algorithm terminates and outputs the result. Let it be the Nash equilibrium point; otherwise, let Continue iterating.
[0093] In typical scenarios, convergence to Nash equilibrium is achieved within 10 to 20 iterations. The final Pareto-optimal maintenance scheme achieves the best trade-off among the three objectives, controlling costs, ensuring performance, and minimizing traffic impact.
[0094] The feedback control unit generates feedback control signals based on the Pareto optimal maintenance scheme, adjusting the image acquisition strategy and feature extraction weights of the multi-scale feature extraction module. The core idea of feedback control is to increase the image acquisition frequency and feature extraction weights for road sections where predictions indicate rapid future damage growth, achieving intensive monitoring of high-risk areas; and to reduce the sampling frequency for road sections with stable damage, saving computational resources and acquisition costs.
[0095] Specifically, based on the damage growth rate output by the temporal damage evolution prediction module. The road sections are divided into three risk levels. High-risk road sections are defined as follows: mm / month, medium-risk road sections are defined as mm / month Low-risk road sections are defined as For high-risk road sections, the sampling frequency is set to once a week, and the feature extraction weight is increased to 1.5 times the standard value. For medium-risk road sections, the sampling frequency is once a month, and the weight remains at the standard value. For low-risk road sections, the sampling frequency is once a quarter, and the weight is reduced to 0.5 times the standard value.
[0096] Feedback control signals are represented in matrix form: .
[0097] in, The total number of road segments. Let be the sampling frequency of the i-th road segment. , and These represent the feature extraction weights at the macro, meso, and micro scales, respectively. This feedback control signal is passed to the multi-scale feature extraction module, dynamically adjusting the image acquisition and feature extraction strategies to achieve adaptive optimization of resource allocation.
[0098] The four modules of the system of this invention form a closed-loop collaborative system through deep coupling, which realizes the overall performance of mutual promotion, superimposed efficiency, and collaborative optimization.
[0099] First, a parameter-level coupling is formed between the multi-scale feature extraction module 1 and the damage type intelligent identification module 2. The geometrically invariant feature vectors output by the multi-scale feature extraction module 1 are directly used as inputs to the damage type intelligent identification module 2, and the quality of the features directly affects the identification accuracy. The features extracted through topological manifold learning preserve the intrinsic geometric structure of the data, providing a high-quality basic representation for Lie group invariant encoding. Conversely, the accuracy of the identification results of the damage type intelligent identification module 2 verifies the effectiveness of the feature extraction by the multi-scale feature extraction module 1, and the two promote each other, forming a positive feedback loop.
[0100] Second, a state-level coupling is formed between the damage type intelligent identification module 2 and the temporal damage evolution prediction module 3. The damage feature descriptor output by the damage type intelligent identification module 2 provides the current state of the damage, including location, type, geometric parameters, and severity. This information serves as the initial and boundary conditions for the dynamic modeling of module 3. The temporal damage evolution prediction module 3 predicts the future damage distribution based on the current damage state and historical evolution trends. The combination of the two achieves synergistic effects: current state identification + historical trend analysis → future prediction, resulting in significantly higher prediction accuracy than a single method based solely on the current state or historical trends.
[0101] Third, a logical coupling and closed-loop feedback are formed between the temporal damage evolution prediction module 3 and the multi-scale feature extraction module 1. The high-risk area information predicted by the temporal damage evolution prediction module 3 generates feedback control signals through the adaptive maintenance decision module 4, which are then transmitted to the multi-scale feature extraction module 1 to adjust the sampling density and feature extraction weights for high-risk areas. This feedback mechanism achieves adaptive adjustment, enabling the system to dynamically optimize resource allocation based on damage evolution trends, concentrating limited computational and acquisition resources on the areas of greatest concern, and improving overall monitoring efficiency.
[0102] Fourth, the adaptive maintenance decision module 4 comprehensively utilizes the current damage state of the damage type intelligent identification module 2 and the future damage prediction of the time-series damage evolution prediction module 3 to construct a multi-objective optimization model and solve for the optimal maintenance scheme. The scientific nature of maintenance decisions depends on accurate damage identification and reliable evolution prediction; these three elements form a synergistic relationship, jointly supporting the formulation of preventive maintenance strategies. Simultaneously, the adaptive maintenance decision module 4 feeds back the decision results to the multi-scale feature hierarchical extraction module 1 through the feedback control unit, realizing decision-driven monitoring optimization and forming a complete closed loop of "monitoring → identification → prediction → decision → optimized monitoring".
[0103] Fifth, the synergistic effect of the four modules is reflected in the non-linear growth of overall performance. A single feature extraction module can only provide image features, a single recognition module can only determine current damage, a single prediction module can only estimate future trends, and a single decision-making module can only formulate a solution based on limited information. However, the deep coupling of the four modules creates a positive cycle: improved feature quality → increased recognition accuracy → enhanced prediction reliability → improved decision-making scientific rigor → optimized monitoring efficiency, achieving a synergistic effect where 1+1>2. The overall performance of the system far exceeds the sum of the individual module operations, demonstrating the advantages of a deeply coupled closed-loop collaborative system.
[0104] In a specific application example, a section of a highway was monitored for six months. Initial monitoring revealed multiple cracks in the section, and a multi-scale feature extraction module extracted features at macro, meso, and micro scales. A damage type intelligent identification module, based on Lie group invariance, identified 15 transverse cracks, 8 longitudinal cracks, and 3 alligator cracks, measuring the length, width, and depth of each crack. A temporal damage evolution prediction module, based on historical data from the previous three months, established a differential equation model for damage evolution, predicting the development trend of each crack over the next three months. The prediction results showed that five cracks had a growth rate exceeding 2 mm / month, classifying them as high-risk damage. An adaptive maintenance decision module, based on the current damage state and evolution prediction, solved a multi-objective optimization model to determine the optimal maintenance plan: milling and repaving for the five high-risk cracks, and sealing for the remaining cracks, with a total maintenance cost of 180,000 yuan. The projected road performance index after maintenance was to increase from 65 to 85, with a construction time of 7 days. Simultaneously, the feedback control unit increased the sampling frequency of the five high-risk areas from once a month to once a week, and increased the feature extraction weight by 1.5 times. The actual monitoring results three months later showed that the predicted damage development trend matched the actual situation 92% well, and the road performance after maintenance met expectations, verifying the effectiveness of the system.
[0105] This invention possesses significant technical advantages and application value. Compared with existing technologies, its innovation lies in: employing topological manifold learning and Lie group transformation theory to extract geometrically invariant multi-scale features, overcoming the sensitivity of traditional methods to geometric transformations; establishing a damage evolution prediction model based on dynamical system theory, achieving accurate prediction of future damage states, filling the gap in predictive analysis in existing technologies; constructing a multi-objective maintenance decision optimization framework based on game theory, realizing a scientific trade-off between cost, performance, and traffic impact; and forming a closed-loop adaptive optimization system through a feedback control mechanism, significantly improving the level of intelligence and operational efficiency. The deep coupling of the four modules generates synergistic effects, with overall performance far exceeding that of existing single-function systems.
[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A multi-level image-based intelligent damage assessment and prediction system for highway infrastructure, characterized in that, include: The multi-scale feature extraction module is used to acquire multi-resolution image sequences of highway infrastructure. It constructs feature representation spaces at three scales (macro, meso, and micro) based on the topological manifold learning method, and maps high-dimensional image features to low-dimensional topological manifolds through manifold embedding algorithm to extract geometrically invariant feature vectors at each scale. The damage type intelligent identification module is connected to the multi-scale feature hierarchical extraction module. It is used to receive the geometric invariant feature vectors of each scale, construct the rotation and translation scale invariant representation of the damage mode based on the Lie group transformation theory, identify five types of damage, namely cracks, pits, subsidence, spalling and network cracks, through the orbit decomposition of the transformation group, and generate a damage feature descriptor containing damage location, type, geometric parameters and severity. The temporal damage evolution prediction module is connected to the damage type intelligent identification module. It is used to receive the damage feature descriptor and historical image sequence, establish a differential equation model of damage propagation based on dynamic system theory, extract the dynamic features of damage evolution through phase space reconstruction method, calculate the damage growth rate, expansion direction and stability index, and predict the damage state distribution at future time. An adaptive maintenance decision module, connected to the damage type intelligent identification module and the temporal damage evolution prediction module, is used to receive the damage feature descriptor and the damage state distribution, construct a multi-objective optimization model based on a game theory framework that includes maintenance cost, road performance and traffic impact, solve the Nash equilibrium strategy to determine the optimal maintenance scheme, and generate feedback control signals according to the optimal maintenance scheme to adjust the image acquisition strategy and feature extraction weights of the multi-scale feature hierarchical extraction module, forming a closed-loop adaptive optimization system.
2. The system according to claim 1, characterized in that, The multi-scale feature hierarchical extraction module includes: The image acquisition unit is used to acquire multi-source image data of highway infrastructure through visible light cameras, infrared cameras and laser scanning equipment, and to construct an image pyramid of macro-scale pixel-level images, meso-scale millimeter-level images and micro-scale micrometer-level images; The topological feature extraction unit is used to perform topological manifold learning on each level of the image pyramid, establish a manifold neighborhood graph by calculating the geodesic distance matrix, and project the image data to a low-dimensional manifold space using the Laplacian feature mapping algorithm to extract topological features that preserve local geometric structure. The scale fusion unit is used to perform weighted fusion of topological features at each scale. It adaptively adjusts the fusion weights of features at each scale according to the feature response intensity of the damage type, and generates a multi-scale fusion feature vector.
3. The system according to claim 1, characterized in that, The intelligent damage type identification module includes: The Lie group invariant coding unit is used to perform Lie group transformation on the geometrically invariant feature vector, calculate the invariants of the feature under the action of rotation group, translation group and scaling transformation group, and construct a damage pattern representation with geometric invariance. The damage pattern classification unit is used to classify similar damage patterns into the same orbit based on the damage pattern representation by orbit decomposition of the transformation group, and to use a kernel function to measure the distance between different orbits to achieve automatic classification of damage types. The damage quantification assessment unit is used to measure the geometric parameters of classified damage, calculate the length, width, depth, area and volume geometric features of the damage, assess the severity of the damage and generate damage level labels.
4. The system according to claim 1, characterized in that, The temporal damage evolution prediction module includes: The phase space reconstruction unit is used to reconstruct the time series of damage features in the historical image sequence in phase space, and to determine the embedding dimension and time delay parameters by means of the delay embedding theorem, so as to recover the dynamic trajectory of damage evolution in phase space. The dynamic modeling unit is used to establish a differential equation model of damage propagation based on the dynamic trajectory, identify the driving factors affecting damage evolution, including traffic load, environmental temperature and humidity and material aging degree, and quantify the contribution weight of each factor to the damage growth rate. The prediction calculation unit is used to solve the differential equation model, calculate the damage state at future times through numerical integration, generate spatiotemporal distribution prediction results including damage location, range and severity, and calculate the prediction uncertainty interval.
5. The system according to claim 1, characterized in that, The adaptive maintenance decision module includes: The multi-objective modeling unit is used to construct a multi-objective optimization model that includes minimizing maintenance costs, maximizing road performance index, and minimizing traffic interruption time, and sets the constraints and weight coefficients for each objective; The game-solving unit is used to transform the multi-objective optimization problem into a multi-party game model, in which each maintenance strategy is a player, each objective function is a payoff function, and the Pareto optimal maintenance scheme is determined by solving the Nash equilibrium point through an iterative algorithm. The feedback control unit is used to generate feedback control signals according to the Pareto optimal maintenance scheme, adjust the sampling density and feature extraction weight of the multi-scale feature hierarchy extraction module for high-risk areas, and realize adaptive optimization of resource allocation.
6. The system according to claim 1, characterized in that, When the multi-scale feature extraction module maps high-dimensional image features to low-dimensional topological manifolds using the manifold embedding algorithm, it employs an isometric mapping method based on geodesic distance to maintain the intrinsic geometric distance between data points in the high-dimensional space in the low-dimensional space, thus ensuring the geometric fidelity of the feature representation.
7. The system according to claim 1, characterized in that, When identifying crack types, the intelligent damage type identification module distinguishes between transverse cracks, longitudinal cracks, oblique cracks, alligator cracks, and network cracks by calculating the principal curvature and geodesic curvature of the damage profile.
8. The system according to claim 1, characterized in that, The time-series damage evolution prediction module introduces the material fatigue cumulative damage theory when establishing the differential equation model of damage propagation, and quantifies the contribution of cyclic loading to damage growth through the Miner linear cumulative damage criterion.
9. The system according to claim 1, characterized in that, When solving for the Nash equilibrium point, the adaptive maintenance decision module adopts an iterative optimal response algorithm. In each iteration, each player solves for its own optimal strategy under the condition that the strategies of the other players are fixed, until all players' strategies converge to the equilibrium state.
10. The system according to claim 1, characterized in that, The closed-loop adaptive optimization system adjusts the image acquisition strategy in real time through the feedback control signal, increasing the sampling frequency for road sections where the prediction results indicate rapid future damage growth, and decreasing the sampling frequency for road sections with stable damage, thereby achieving dynamic optimization of computing and acquisition resources.
Citation Information
Patent Citations
Methods and devices for identifying damage to road crash barriers
CN115641334B
Bridge damage detection method and system
CN119540183A
Highway pavement crack image intelligent detection system and method thereof
CN120852421A
Automated assessment of cracks using lidar and camera data
US20230080178A1
