An airport pavement health monitoring method, system, electronic device and storage medium

Abstract

An airport pavement health monitoring method, system, electronic device and storage medium, which relate to the field of airport pavement monitoring. In this method, multi-source physical data is obtained from target sensors and radars, and inspection images are obtained from inspection drones; the multi-source physical data and inspection images are spatio-temporally aligned, and the vibration signals in the spatio-temporally fused multi-source physical data are fused with the visual features in the inspection images to obtain disease feature vectors; based on the pavement digital twin model, the failure probability and remaining life are predicted according to the disease feature vectors and environmental real-time data, and the dynamic risk map is determined according to the failure probability and remaining life; a hierarchical maintenance strategy is generated according to the dynamic risk map and airport operation data, and the airport operation data includes flight schedules and maintenance resource inventories. Implementing the technical solution provided by this application realizes the accurate assessment of the health status of airport pavements and the intelligent maintenance plan, and improves the monitoring efficiency and decision-making science.

Description

Technical Field

[0001] This application relates to the technical field of airport pavement monitoring, and particularly relates to a method, system, electronic device, and storage medium for airport pavement health monitoring. Background Art

[0002] With the rapid development of the civil aviation industry, as a key infrastructure, the health status of airport pavements is directly related to flight safety and airport operation efficiency. In recent years, with the development of sensor technology and unmanned aerial vehicle (UAV) inspection technology, the health monitoring of airport pavements has gradually shifted from traditional visual inspections to intelligent monitoring with multi-source data fusion, greatly improving the monitoring accuracy and efficiency.

[0003] In the prior art, to solve the problem of airport pavement health monitoring, the following means are usually adopted: one is to conduct regular manual inspections, combined with visual observation and simple measurement tools to record pavement disease conditions; the other is to take images by UAVs and use image processing technology to identify surface diseases such as cracks. The main defects of the above prior art are low efficiency, inability to accurately locate and quantitatively evaluate pavement diseases, and difficulty in dynamically predicting the risk of pavement failure in combination with environmental factors, resulting in a lack of scientific basis for maintenance decisions. Summary of the Invention

[0004] This application provides a method, system, electronic device, and storage medium for airport pavement health monitoring, which realizes the accurate assessment of the health status of airport pavements and intelligent maintenance planning, improves the monitoring efficiency and decision-making scientificity, effectively extends the service life of the pavement, and ensures the safety of airport operations.

[0005] In the first aspect of this application, a method for airport pavement health monitoring is provided, which is applied to an airport pavement monitoring platform. The method includes:

[0006] Obtain multi-source physical data from target sensors and radars, and obtain inspection images from inspection UAVs. The multi-source physical data includes sensor data and radar data;

[0007] Perform spatio-temporal alignment on the multi-source physical data and the inspection images, and fuse the vibration signals in the spatio-temporally fused multi-source physical data with the visual features in the inspection images to obtain disease feature vectors. The vibration signals include the spectral features of pavement load responses, and the visual features include crack morphology and void area texture;

[0008] Based on the pavement digital twin model, predict the failure probability and remaining life according to the disease feature vectors and real-time environmental data, and determine a dynamic risk map according to the failure probability and the remaining life. The real-time environmental data includes temperature, humidity, and flight load frequency;

[0009] Generate a hierarchical maintenance strategy based on the dynamic risk map and airport operation data, where the airport operation data includes flight schedules and maintenance resource inventories.

[0010] Optionally, the spatio-temporal alignment of the multi-source physical data and the inspection images, and the fusion of the vibration signals in the multi-source physical data after spatio-temporal fusion with the visual features in the inspection images to obtain the disease feature vector includes:

[0011] Through a spatio-temporal alignment algorithm, match the acquisition timestamps and GPS coordinates of the sensor data and radar data with the shooting time and spatial position of the inspection images to establish a mapping relationship of multi-source data under the same spatio-temporal reference;

[0012] Extract the spectral features of the vibration signal and extract the geometric shape features of the cracks in the inspection images;

[0013] Use a multi-modal Transformer model to perform cross-modal fusion on the spectral features and the geometric shape features to obtain a fusion feature vector;

[0014] Perform dynamic noise filtering on the fusion feature vector to obtain a disease feature vector, where the disease feature vector includes the category probability, position coordinates, and damage degree quantization value of the disease.

[0015] Optionally, the extraction of the spectral features of the vibration signal and the extraction of the geometric shape features of the cracks in the inspection images includes:

[0016] Perform short-time Fourier transform on the vibration signal to obtain a spectrum, map the spectrum to the Mel scale to calculate the Mel filter bank energy, take the logarithm of the Mel filter bank energy and perform discrete cosine transform to obtain Mel frequency cepstral coefficients features;

[0017] Perform three-layer wavelet packet decomposition on the vibration signal to obtain basic spectral features, and splice the basic spectral features with the Mel frequency cepstral coefficients features to form the spectral features;

[0018] Extract the geometric parameters of the cracks through edge detection and contour fitting, where the geometric parameters include length, width, and curvature, and calculate the texture entropy value of the void area based on the gray-level co-occurrence matrix;

[0019] Obtain the geometric shape features based on the geometric parameters and the texture entropy value.

[0020] Optionally, the use of a multi-modal Transformer model to perform cross-modal fusion on the spectral features and the geometric shape features to obtain a fusion feature vector includes:

[0021] Project the spectral features into a shared semantic space to generate query vectors, and project the geometric morphological features into the shared semantic space to generate key-value pairs;

[0022] Generate the first attention of the spectral features to the geometric morphological features and the second attention of the geometric morphological features to the spectral features according to the query vectors and the key-value pairs;

[0023] Concatenate the first attention and the second attention to obtain a fused feature vector.

[0024] Optionally, the predicting the failure probability and remaining life based on the pavement digital twin model according to the disease feature vector and real-time environmental data includes:

[0025] Input the disease feature vector into the pavement digital twin model, and dynamically update the model boundary conditions in combination with the real-time environmental data;

[0026] Use a physics-informed neural network to solve the multi-physics field coupling equation, output the damage accumulation amount of the pavement structure within a preset future time step, and determine the failure probability according to the damage accumulation amount;

[0027] Calculate the remaining life according to the failure probability.

[0028] Optionally, the determining the dynamic risk map according to the failure probability and the remaining life includes:

[0029] Construct a two-dimensional risk matrix based on the failure probability and the remaining life;

[0030] Generate a full-pavement dynamic risk heat map based on the two-dimensional risk matrix through a spatial interpolation algorithm;

[0031] Combine Monte Carlo simulation to evaluate the risk fluctuation range of the full pavement under different environmental scenarios to obtain a risk confidence interval;

[0032] Generate a dynamic risk map according to the dynamic risk heat map and the risk confidence interval.

[0033] Optionally, the generating a hierarchical maintenance strategy according to the dynamic risk map and airport operation data includes:

[0034] Generate a mapping table of risk areas and maintenance actions according to the dynamic risk map;

[0035] Generate a maintenance plan with time constraints according to the mapping table and the airport operation data;

[0036] Use a genetic algorithm to select a target solution with the lowest cost and shortest downtime from the maintenance plan, and output the maintenance time, maintenance location, maintenance actions, and resource allocation.

[0037] In a second aspect of the present application, an airport pavement health monitoring system is provided, including an acquisition module, a fusion module, a prediction module, and a maintenance module, wherein:

[0038] The acquisition module is configured to obtain multi-source physical data from target sensors and radars, and obtain inspection images from inspection drones. The multi-source physical data includes sensor data and radar data.

[0039] The fusion module is configured to perform spatio-temporal alignment on the multi-source physical data and the inspection images, and fuse the vibration signals in the spatio-temporally fused multi-source physical data with the visual features in the inspection images to obtain disease feature vectors. The vibration signals include spectral features of pavement load responses, and the visual features include crack morphology and void area texture.

[0040] The prediction module is configured to predict the failure probability and remaining life based on the pavement digital twin model according to the disease feature vectors and environmental real-time data, and determine a dynamic risk map according to the failure probability and the remaining life. The environmental real-time data includes temperature, humidity, and flight load frequency.

[0041] The maintenance module is configured to generate a hierarchical maintenance strategy according to the dynamic risk map and airport operation data. The airport operation data includes flight schedules and maintenance resource inventories.

[0042] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to enable the electronic device to execute the method described in any one of the above.

[0043] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0044] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0045] 1. By acquiring multi-source physical data and inspection images from target sensors, radars, and inspection drones, the fusion of multi-source data is achieved. This fusion not only enriches the types of monitoring data but also improves the accuracy and reliability of the data, enabling a more comprehensive reflection of the health status of the airport pavement; the vibration signal is fused with the visual features in the inspection image to obtain a disease feature vector containing information such as the spectral characteristics of the pavement load response, crack morphology, and the texture of the void area. This fusion method enhances the robustness and expressive ability of the feature vector, contributing to a more accurate identification of pavement diseases;

[0046] 2. The spatio-temporal alignment of multi-source physical data and inspection images ensures the consistency of data from different sources in terms of time and space, facilitating subsequent analysis and processing; the fusion of vibration signals and visual features further simplifies the data analysis process and improves the analysis efficiency, making the airport pavement health monitoring more efficient and real-time;

[0047] 3. Using the pavement digital twin model, the failure probability and remaining life of the pavement are predicted based on the disease feature vector and real-time environmental data (such as temperature, humidity, and flight load frequency). This prediction method is based on big data and machine learning technologies, enabling a more accurate assessment of the pavement health status and potential risks; by predicting the failure probability and remaining life, the airport can formulate a more scientific and reasonable maintenance plan to avoid over-maintenance or under-maintenance;

[0048] 4. A dynamic risk map is determined based on the failure probability and remaining life, providing an intuitive risk assessment tool for airport operations. The dynamic risk map can reflect the risk status of the pavement in real-time, helping airport managers make decisions in a timely manner; combined with airport operation data (such as flight schedules and maintenance resource inventories), a hierarchical maintenance strategy is generated. This strategy can reasonably arrange maintenance resources and time according to different risk levels and operation requirements, improving maintenance efficiency and economy;

[0049] 5. By real-time monitoring and predicting the pavement health status, potential safety hazards can be detected and addressed in a timely manner to ensure the safety of airport operations; at the same time, the formulation and implementation of the hierarchical maintenance strategy can reduce unnecessary maintenance work and downtime, improving the operation efficiency and economic benefits of the airport. Brief Description of the Drawings

[0050] Figure 1 is a schematic flow chart of an airport pavement health monitoring method disclosed in an embodiment of the present application;

[0051] Figure 2 is a schematic module diagram of an airport pavement health monitoring system disclosed in an embodiment of the present application;

[0052] Figure 3It is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

[0053] Explanation of reference numerals in the drawings: 201, acquisition module; 202, fusion module; 203, prediction module; 204, maintenance module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Specific implementation manners

[0054] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0055] In the description of the embodiments of the present application, words such as "for example" or "for illustration" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0056] In the description of the embodiments of the present application, the meaning of the term "a plurality" refers to two or more. For example, a plurality of systems refers to two or more systems, and a plurality of screen terminals refers to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0057] This embodiment discloses an airport pavement health monitoring method, which is applied to an airport pavement monitoring platform. Figure 1 It is a schematic flowchart of an airport pavement health monitoring method disclosed in an embodiment of the present application. As Figure 1 shown, the method includes the following steps:

[0058] S101. Obtain multi-source physical data from target sensors and radars, and obtain inspection images from inspection drones. The multi-source physical data includes sensor data and radar data.

[0059] S102. Spatially and temporally align the multi-source physical data and the inspection images, and fuse the vibration signals in the multi-source physical data after spatio-temporal fusion with the visual features in the inspection images to obtain a disease feature vector. The vibration signals include the spectral features of the pavement load response, and the visual features include crack morphology and void area texture;

[0060] S103. Based on the pavement digital twin model, predict the failure probability and remaining life according to the disease feature vector and environmental real-time data, and determine a dynamic risk map according to the failure probability and the remaining life. The environmental real-time data includes temperature, humidity, and flight load frequency;

[0061] S104. Generate a hierarchical maintenance strategy according to the dynamic risk map and airport operation data. The airport operation data includes flight schedules and maintenance resource inventories.

[0062] Collect multi-source physical data from target sensors and radars. Among them, sensor data may come from various types of sensors, such as sensors for monitoring parameters such as pavement stress, strain, temperature, etc.; radar data may be used to detect structural deformations, foreign objects, etc. on the pavement. These data provide rich information about the physical state of the pavement for subsequent analysis. Obtain inspection images using inspection drones. The drones can take pictures of the pavement at different heights and angles to obtain appearance images of the pavement, which can visually display the surface conditions of the pavement, such as the appearance characteristics of diseases such as cracks and voids. Align the multi-source physical data and inspection images in time and space. Since the multi-source data come from different devices and acquisition methods, there may be differences in time and space. The purpose of time-space alignment is to ensure that these data are correlated in the same time and space dimensions for subsequent effective fusion. For example, match the sensor data at a certain moment with the inspection image at the corresponding location and moment. Fuse the vibration signals in the multi-source physical data after time-space fusion with the visual features in the inspection images. The vibration signals contain the spectral characteristics of the pavement load response. By analyzing these spectral characteristics, the dynamic response of the pavement under load can be understood; the visual features include crack morphology and void area texture, which are directly extracted from the inspection images and reflect the surface disease conditions of the pavement. Fuse these two different types of features to obtain a comprehensive disease feature vector, which can more comprehensively describe the disease state of the pavement. Use the pavement digital twin model to predict the failure probability and remaining life in combination with the disease feature vector and real-time environmental data. The pavement digital twin model is a virtual mapping of the actual pavement. It can simulate the performance evolution process of the pavement according to the input disease characteristics and environmental factors. Real-time environmental data include temperature, humidity, and flight load frequency, etc. These factors have an important impact on the performance and life of the pavement. Through model calculation, the probability of the pavement failing in the future period of time and the remaining service life can be obtained. Determine the dynamic risk map based on the predicted failure probability and remaining life. The dynamic risk map can visually display the risk status of different areas of the pavement at different times, providing a basis for subsequent maintenance decisions. For example, different colors may be used in the risk map to represent areas with different risk levels, and red indicates high-risk areas that need to be maintained first. Combine the dynamic risk map and airport operation data: Generate a hierarchical maintenance strategy based on the dynamic risk map and airport operation data. Airport operation data includes information such as flight schedules and maintenance resource inventories. The flight schedule determines the usage frequency and time distribution of the pavement, and the maintenance resource inventory limits the available manpower, material resources, and financial resources for maintenance. Considering these factors comprehensively, different levels of maintenance strategies can be formulated. The hierarchical maintenance strategy means dividing the maintenance work into different priorities and types according to the risk level of the pavement and the actual situation of the airport.For example, for high-risk areas, emergency repairs may need to be arranged during flight gaps; for low-risk areas, regular inspections and preventive maintenance can be adopted. This can reasonably allocate maintenance resources, improve maintenance efficiency, and ensure the safe operation of airport pavements.

[0063] Optionally, the spatio-temporal alignment of the multi-source physical data and the inspection images, and the fusion of the vibration signals in the multi-source physical data after spatio-temporal fusion with the visual features in the inspection images to obtain the disease feature vector includes:

[0064] Through a spatio-temporal alignment algorithm, match the acquisition timestamps and GPS coordinates of the sensor data and radar data with the shooting time and spatial position of the inspection images, and establish a mapping relationship of multi-source data under the same spatio-temporal reference;

[0065] Extract the spectral features of the vibration signal, and extract the geometric shape features of the cracks in the inspection images;

[0066] Adopt a multi-modal Transformer model to perform cross-modal fusion on the spectral features and the geometric shape features to obtain a fusion feature vector;

[0067] Perform dynamic noise filtering on the fusion feature vector to obtain a disease feature vector, where the disease feature vector includes the category probability, position coordinates, and damage degree quantization value of the disease.

[0068] Using the spatio-temporal alignment algorithm, match the acquisition timestamps of sensor data and radar data with the shooting time of inspection images. For example, if sensor data is collected every 1 second and the shooting time interval of inspection images is not fixed, the algorithm will find the closest time point to associate data from different data sources at the same or similar times. Match the acquisition locations of sensor data and radar data with the shooting spatial locations of inspection images through GPS coordinates. For sensors and radars, their data usually comes with GPS information of the acquisition location; for inspection images, the spatial location corresponding to each image can be determined through the flight trajectory and shooting parameters of the drone. The algorithm will calculate the spatial distance between different data sources and associate data with close positions. After spatio-temporal matching, establish a multi-source data mapping relationship under the same spatio-temporal reference. For example, associate sensor data, radar data at a certain moment and position with the corresponding inspection image data to form a unified dataset, providing a basis for subsequent feature extraction and fusion. The vibration signal contains dynamic response information of the pavement under load. Through spectral analysis, it can be decomposed into components of different frequencies, thereby extracting spectral features reflecting the pavement structure characteristics. Adopt signal processing techniques such as Fourier transform and wavelet transform to convert the vibration signal from the time domain to the frequency domain to obtain the spectrogram of the signal. Extract key features from the spectrogram, such as frequency peaks and band energies, which can reflect the mechanical properties of the pavement such as stiffness and damping. The crack morphology in the inspection image is an important visual feature of pavement diseases. Through image processing techniques, information such as the geometric shape, length, and width of the cracks can be extracted, providing a basis for disease identification and assessment. First, preprocess the inspection image, such as grayscale conversion and denoising, to enhance the edge information of the cracks. Then, use an edge detection algorithm (such as the Canny algorithm) to extract the edges of the cracks, and then obtain the contours of the cracks through a contour tracking algorithm. Finally, calculate geometric parameters such as the length, width, and area of the cracks as the geometric morphology features of the cracks. The multi-modal Transformer model is a deep learning model that can process different modal data (such as text, images, audio, etc.). In this scenario, it is used to perform cross-modal fusion of the spectral features of vibration signals and the geometric morphology features of inspection images. The model can learn the association and complementary information between different modal features, thereby generating a more representative fusion feature vector. Input the extracted spectral features and geometric morphology features into different branches of the multi-modal Transformer model respectively. The model uses the self-attention mechanism to interact and fuse different modal features, capturing the semantic relationships between them. After being processed by multiple Transformer layers, a fusion feature vector is output, which synthesizes the information of vibration signals and inspection images. During the feature extraction and fusion process, some noise may be introduced, which will affect the accuracy and reliability of the disease feature vector.The purpose of dynamic noise filtering is to remove this noise and improve the quality of the disease feature vector. The dynamic noise filtering algorithm is adopted to adaptively adjust the filtering parameters according to the statistical characteristics of the feature vector and the distribution of the noise. For example, algorithms such as the Kalman filter and the particle filter can be used to perform real-time filtering on the fused feature vector. The filtered feature vector is the disease feature vector. The category probability of the disease represents the probability that the disease corresponding to the feature vector belongs to different types (such as cracks, voids, etc.). Through learning a large number of known disease samples, the model can predict the most likely type of the disease represented by the current feature vector. The position coordinates indicate the specific position of the disease on the pavement, usually represented in the form of GPS coordinates or other spatial coordinates. This helps maintenance personnel accurately locate the disease position for targeted maintenance. The damage degree quantification value quantitatively evaluates the damage degree of the disease, such as the width and depth of cracks, the area and volume of voids, etc. The damage degree quantification value can provide more detailed information for maintenance decisions, helping to determine the maintenance priority and method.

[0069] Data from different sources often exhibit differences in time and space. This spatio-temporal alignment operation can eliminate these differences, enabling multi-source data to be correlated and analyzed in the same spatio-temporal dimension. The spectral characteristics of vibration signals can reflect the dynamic response of the pavement under load. For example, the amplitude and phase information of different frequency components can reveal internal defects and damage levels of the pavement structure. By analyzing the spectral characteristics, the vibration modes and energy distribution of the pavement can be understood, thereby judging the health status of the pavement. The geometric morphological characteristics of cracks in inspection images, such as the length, width, and shape of the cracks, are important indicators that intuitively reflect surface diseases of the pavement. Extracting these characteristics can help determine the type, development degree, and potential hazards of the cracks, providing a basis for the assessment and repair of diseases. The spectral characteristics of vibration signals and the geometric morphological characteristics of inspection images belong to different modalities, and they contain different aspects of information about pavement diseases. Through cross-modal fusion, these two different types of information can be organically combined to achieve information complementarity. For example, the spectral characteristics may focus more on the internal structure and dynamic performance of the pavement, while the geometric morphological characteristics more intuitively reflect the surface condition of the pavement. The fused feature vector can more comprehensively describe the disease situation of the pavement. The multi-modal Transformer model has powerful feature learning and fusion capabilities. It can automatically learn the correlation and mapping relationships between different modal features, thereby generating a more expressive fused feature vector. This fused feature vector can capture more complex disease patterns, improving the accuracy and reliability of disease recognition. During the actual data acquisition process, various noises will inevitably be introduced, and these noises will affect the quality of the feature vector. Dynamic noise filtering can adaptively remove noise according to the statistical characteristics of the feature vector and the distribution of the noise, improving the signal-to-noise ratio of the feature vector. The final disease feature vector contains key information such as the category probability of the disease, location coordinates, and quantification value of the damage degree. The disease category probability can help determine the type of disease, such as cracks, voids, etc.; the location coordinates can accurately locate the position of the disease, facilitating subsequent maintenance and repair work; the quantification value of the damage degree can evaluate the severity of the disease, providing an important basis for formulating maintenance strategies. The comprehensive provision of this information makes the assessment of pavement diseases more comprehensive and accurate.

[0070] Optionally, extracting the spectral characteristics of the vibration signal and extracting the geometric morphological characteristics of the cracks in the inspection image includes:

[0071] Performing short-time Fourier transform on the vibration signal to obtain a spectrum, mapping the spectrum to the Mel scale to calculate the Mel filter bank energy, taking the logarithm of the Mel filter bank energy and performing discrete cosine transform to obtain Mel frequency cepstral coefficient features;

[0072] Perform three-layer wavelet packet decomposition on the vibration signal to obtain the basic spectral features, and splice the basic spectral features and the Mel frequency cepstral coefficient features to form the spectral features;

[0073] Extract the geometric parameters of the crack through edge detection and contour fitting. The geometric parameters include length, width, and curvature, and calculate the texture entropy value of the debonded area based on the gray-level co-occurrence matrix;

[0074] Obtain the geometric shape features according to the geometric parameters and the texture entropy value.

[0075] Perform short-time Fourier transform (STFT) on the vibration signal to convert the time-domain signal into a frequency-domain signal and obtain the spectrum. STFT segments the signal in time and performs Fourier transform on each segment, thereby retaining both the time-domain and frequency-domain information of the signal. This allows for the analysis of the frequency components of the signal at different time intervals, facilitating an understanding of the spectral characteristics of the vibration signal over time. Map the obtained spectrum onto the Mel scale. The Mel scale is a frequency scale that conforms to the auditory characteristics of the human ear, having a higher resolution for low-frequency signals and a lower resolution for high-frequency signals. Then calculate the Mel filter bank energy. The Mel filter bank is a set of band-pass filters evenly distributed on the Mel scale. By calculating the energy of the signal passing through each filter, the energy distribution of the signal in different Mel frequency bands can be obtained. Take the logarithm of the Mel filter bank energy, which can compress the dynamic range of the energy and make the features more stable. Then perform discrete cosine transform (DCT) to convert the Mel filter bank energy into Mel-frequency cepstral coefficients (MFCC). MFCC is a parameter that can effectively represent the characteristics of speech and vibration signals. It removes the correlation in the signal and retains the main characteristic information of the signal. Perform three-layer wavelet packet decomposition on the vibration signal to obtain the basic spectral features. Wavelet packet decomposition is a more refined signal decomposition method that can decompose the signal into different frequency bands and the number of decomposition layers can be selected as needed. Three-layer wavelet packet decomposition can decompose the signal into multiple sub-bands, each corresponding to a different frequency range, thereby obtaining more detailed spectral information. Concatenate the basic spectral features with the MFCC features to form the final spectral features. The basic spectral features provide information on the energy distribution of the signal in different frequency bands, while the MFCC features focus more on the speech and vibration characteristics of the signal. By concatenating these two types of features, the advantages of both can be comprehensively utilized to obtain more comprehensive and accurate spectral features. Detect the edges of the cracks in the inspection image through an edge detection algorithm, and then use a contour fitting algorithm (such as least squares fitting) to fit the edges to obtain the contours of the cracks. Edge detection can highlight the regions with drastic gray-level changes in the image, i.e., the edges of the cracks; contour fitting can connect the discrete edge points into a smooth curve to more accurately describe the shape of the cracks. According to the fitted crack contours, calculate the geometric parameters of the cracks, including length, width, and curvature. The length can be obtained by calculating the distances between points on the contour; the width can be measured in the vertical direction of the crack; and the curvature can reflect the degree of bending of the crack. These geometric parameters can intuitively describe the size and shape characteristics of the cracks. The gray-level co-occurrence matrix is a statistical method for analyzing the texture features of an image, which describes the spatial distribution of gray-level pairs in the image. By calculating the gray-level co-occurrence matrix of the void area in the inspection image, the texture information of this area can be obtained. Calculate the texture entropy value of the void area based on the gray-level co-occurrence matrix.The texture entropy value reflects the complexity of the image texture. The larger the entropy value, the more complex the texture; the smaller the entropy value, the simpler the texture. For the delamination area, the texture entropy value can be used as a feature to describe the surface roughness and irregularity. Based on the calculated crack geometric parameters and the texture entropy value of the delamination area, the geometric shape features are obtained. These features integrate the size and shape of the crack and the texture information of the delamination area, and can more comprehensively describe the geometric shape features of the diseases in the inspection image, providing an important basis for subsequent disease identification and analysis.

[0076] The Mel scale is designed based on the auditory perception characteristics of the human ear. Mapping the spectrum to the Mel scale can better simulate the human ear's perception of sounds at different frequencies. This makes the extracted features more in line with human auditory habits and can more effectively capture the frequency components related to pavement diseases in the vibration signal. By calculating the Mel filter bank energy and performing discrete cosine transform, the original spectrum data is reduced in dimension and feature concentrated. The redundancy of the data is reduced, while the key feature information of the vibration signal is retained, the complexity of subsequent processing is reduced, and the calculation efficiency is improved. The operation of taking the logarithm can compress the dynamic range of the data, making the features less sensitive to changes in the signal amplitude and enhancing the stability of the features. This is very useful for dealing with the noise and signal fluctuations that may exist in the actual environment and can improve the robustness of disease identification. Wavelet packet decomposition can perform multi-resolution analysis on the vibration signal and can capture both the low-frequency and high-frequency components of the signal simultaneously. The decomposition coefficients at different levels reflect the features of the signal in different frequency bands, which helps to more comprehensively understand the spectrum distribution of the pavement vibration signal and discover disease information at different scales. The basic spectrum features obtained by wavelet packet decomposition are spliced with the Mel frequency cepstrum coefficient features to achieve the complementarity between different features. The Mel frequency cepstrum coefficient features focus on simulating the human ear's auditory perception, while the wavelet packet decomposition features provide richer frequency band information. The combination of the two can more accurately describe the spectrum characteristics of the vibration signal and improve the accuracy of disease identification. Combining the geometric parameters of the crack and the texture entropy value of the delamination area to form geometric shape features can more comprehensively reflect the situation of pavement surface diseases. The geometric parameters describe the shape of the crack, and the texture entropy value reflects the texture characteristics of the delamination area. The two complement each other, providing richer information for subsequent disease analysis and maintenance decision-making. Different types of diseases may show different characteristics in geometric parameters and texture entropy values. By integrating these features, the discrimination degree of disease features can be improved, which helps to more accurately identify and classify pavement diseases.

[0077] Optionally, the cross-modal fusion of the spectrum features and the geometric shape features by using the multi-modal Transformer model to obtain a fusion feature vector includes:

[0078] Project the spectral features into the shared semantic space to generate query vectors, and project the geometric morphological features into the shared semantic space to generate key-value pairs;

[0079] Generate the first attention of the spectral features to the geometric morphological features and the second attention of the geometric morphological features to the spectral features according to the query vectors and the key-value pairs;

[0080] Concatenate the first attention and the second attention to obtain a fused feature vector.

[0081] Different modalities (spectral features and geometric morphological features) originally reside in different feature spaces, with different data distributions and representation forms. The shared semantic space is an abstract feature space that can map the features of different modalities to a unified semantic level, enabling effective interaction and comparison of features from different modalities. Through specific projection operations (usually linear transformations), the spectral features and geometric morphological features are respectively converted into query vectors and key-value pairs. The query vector can be regarded as a "question" about the target information, while the key-value pair contains the "answer" for matching and providing information. For example, the query vector after projecting the spectral features may ask "which geometric features are related to the current spectral pattern", and the key-value pair of geometric features provides the corresponding geometric information to answer this question. Based on the query vector and key-value pair, the first attention of spectral features to geometric morphological features and the second attention of geometric morphological features to spectral features are generated. The attention mechanism is a computational method that simulates the allocation of human attention. It can dynamically allocate weights according to the correlation between different parts of the input. In multimodal fusion, the attention mechanism is used to measure the degree of association between features of different modalities. Guided by the query vector of spectral features, matching and calculation are performed in the key-value pairs of geometric features. By calculating the similarity between the query vector and the key vector (such as dot product similarity) and normalizing it using the softmax function, the attention weights of geometric features to spectral features are obtained. These weights represent the degree of correlation between each geometric feature element and the current spectral feature element. Then, the attention weights are multiplied by the value vector and summed to obtain the first attention representation of spectral features to geometric features. This can be understood as spectral features "focusing" on the information most relevant to them from geometric features. Similarly, guided by the query vector of geometric features, matching and calculation are performed in the key-value pairs of spectral features to obtain the second attention of geometric features to spectral features. This indicates that geometric features "focus" on the information most relevant to them from spectral features. The first attention and the second attention are concatenated to obtain a fused feature vector. Concatenation is a simple and effective feature fusion method that combines attention representations from different sources to form a more comprehensive feature vector. The first attention and the second attention respectively reflect the interaction relationship between spectral features and geometric features from different perspectives. Concatenating them together can preserve this information, enabling the fused feature vector to contain the features of both modalities and the association information between them. The obtained fused feature vector synthesizes the information of spectral features and geometric morphological features and takes into account their mutual relationship. This fused feature vector has stronger expressive power and discriminability, can better describe the characteristics of pavement diseases, and provides more effective input for subsequent tasks such as disease recognition, classification, and evaluation.

[0082] Spectral features and geometric morphological features come from different modalities (vibration signals and images), and there are significant differences in data form, distribution, and semantic expression between them. By projecting into a shared semantic space, the features of these two different modalities can be mapped into a unified semantic framework, eliminating the differences between modalities and enabling them to be compared and fused at the same semantic level. In the shared semantic space, it is easier to discover the correlations between features of different modalities. For example, certain patterns in the spectral features that reflect the internal structure of the pavement may correspond to the morphology of cracks in the geometric morphological features. By projecting into the shared semantic space, these potential correlations can be better captured, providing a more accurate basis for subsequent attention calculation. The first attention represents the degree of attention of the spectral features to the geometric features, and the second attention represents the degree of attention of the geometric features to the spectral features. By calculating the bidirectional attention, the interaction information between the features of the two modalities can be comprehensively captured. For example, when judging pavement diseases, certain frequency components in the spectral features may be closely related to specific morphologies of cracks in the geometric features, and the bidirectional attention mechanism can discover this correlation, thus better understanding the essence of the diseases. The attention mechanism can dynamically assign weights to different features according to the correlations between the features. For features that are more relevant to the current task, higher weights will be assigned, thereby highlighting the roles of these features in the fusion process. This dynamic weighting method can improve the pertinence and effectiveness of the fused features, making the fused feature vector better reflect the key information of pavement diseases. By concatenating the bidirectional attention, the fused feature vector retains the rich interaction information between the spectral features and the geometric features. This information not only includes the characteristics of the features of the two modalities themselves but also the relationships between them, providing a more comprehensive basis for subsequent disease identification and analysis. The concatenated fused feature vector has stronger expressive power and can better describe the complex situations of pavement diseases. For example, in practical applications, different types of diseases may exhibit different patterns in spectral features and geometric features, and the fused feature vector can integrate this information to more accurately distinguish different types of diseases, improving the accuracy and reliability of disease identification.

[0083] Optionally, the predicting the failure probability and remaining life based on the pavement digital twin model according to the disease feature vector and real-time environmental data includes:

[0084] Inputting the disease feature vector into the pavement digital twin model and dynamically updating the model boundary conditions in combination with the real-time environmental data;

[0085] Using a physics-informed neural network to solve the multi-physics field coupling equation, outputting the damage accumulation amount of the pavement structure within a preset future time step, and determining the failure probability according to the damage accumulation amount;

[0086] Calculating the remaining life according to the failure probability.

[0087] Input the disease feature vector into the pavement digital twin model. The disease feature vector contains the key information of pavement diseases, such as the category probability of diseases, location coordinates, and the quantification value of damage degree, etc. These information can reflect the current health status of the pavement and provide the basic data for the prediction of the model. The real-time environmental data includes temperature, humidity, and flight load frequency, etc. These factors will have a significant impact on the performance and lifespan of the pavement. For example, high temperature will cause the pavement material to expand and increase the internal stress; humidity will affect the strength and durability of the material; the flight load frequency is directly related to the fatigue damage borne by the pavement. The pavement digital twin model needs to dynamically update its boundary conditions according to these real-time environmental data. The boundary conditions refer to the external constraints and initial conditions set in the model calculation. By updating the boundary conditions, the model can more accurately simulate the working state of the pavement in the actual environment, thereby improving the accuracy of prediction. The physics-informed neural network is a new modeling method that combines physical knowledge and neural networks. It can not only utilize the powerful fitting ability of neural networks, but also integrate physical laws and equations into the network training process, thereby ensuring that the prediction results of the model conform to physical laws. The pavement structure will be affected by multiple physical fields during actual operation, such as the mechanical field, thermal field, etc. These physical fields are coupled with each other and jointly affect the performance of the pavement. The multi-physics field coupling equation describes the interaction relationship between these physical fields. Using the physics-informed neural network to solve the multi-physics field coupling equation, the damage accumulation amount of the pavement structure within the future preset time step can be obtained. During the solution process, the neural network will continuously adjust its own parameters according to the input disease feature vector and the updated boundary conditions, so that the prediction results meet the physical equations and boundary conditions. The damage accumulation amount reflects the damage degree borne by the pavement structure within a certain period of time in the future. It is a comprehensive index that considers the influence of various factors on the pavement performance. As time goes by, the damage accumulation amount will continue to increase. When it reaches a certain level, the pavement may fail. According to the damage accumulation amount, a certain probability model or statistical method can be used to determine the failure probability of the pavement. For example, a mapping relationship between the damage accumulation amount and the failure probability can be established, and the failure probability under the current damage accumulation amount can be obtained by looking up the table or calculation. The failure probability represents the likelihood of the pavement failing within a certain period of time in the future. The remaining life refers to the remaining time from the current moment until the pavement fails. It is an important index used to evaluate the service life of the pavement and formulate maintenance strategies. According to the failure probability, reliability theory or survival analysis methods can be used to calculate the remaining life. For example, it can be assumed that the failure time of the pavement follows a certain probability distribution, and then according to the current failure probability and the parameters of this probability distribution, the expected value or confidence interval of the remaining life can be calculated.

[0088] The disease feature vector contains detailed information about pavement diseases, such as disease categories, locations, damage degrees, etc. Inputting this information into the digital twin model can accurately simulate the impact of diseases on the pavement structural performance, making the model closer to the actual situation of the pavement. Real-time environmental data has an important impact on the performance and lifespan of the pavement. Dynamically updating the model boundary conditions can enable the model to reflect the changes in environmental factors in real time, improving the accuracy and reliability of the simulation. The damage process of the pavement involves the interaction of multiple physical fields, such as mechanics, thermotics, acoustics, etc. The physics-informed neural network can solve the coupled equations of multiple physical fields, comprehensively considering the mutual influence between these physical fields, so as to more accurately predict the damage accumulation of the pavement. For example, under the action of aircraft loads, the pavement will generate mechanical responses, and at the same time, the change in temperature will also affect the performance of the pavement material. The multi-physical field coupling model can comprehensively consider these factors. The physics-informed neural network combines physical knowledge and the learning ability of neural networks, and can use physical laws for constraints under the condition of limited data to improve the accuracy of prediction. By outputting the damage accumulation amount within a preset future time step, the development trend of pavement damage can be clearly understood, providing a reliable basis for subsequent failure probability and remaining lifespan prediction. The failure probability is an important indicator to measure the safety of the pavement. By determining the failure probability based on the damage accumulation amount, the failure risk of the pavement can be quantified, enabling decision-makers to more intuitively understand the safety status of the pavement. For example, when the failure probability is high, it indicates that there is a greater safety risk for the pavement and maintenance measures need to be taken in a timely manner. The determination of the failure probability is based on the coupled equations of multiple physical fields solved by the physics-informed neural network and the damage accumulation amount. This method has a scientific theoretical basis, avoids the uncertainty of subjective judgment, and improves the accuracy and reliability of the assessment. The calculation of the remaining lifespan can provide an important reference for the pavement maintenance plan. By understanding the remaining lifespan of the pavement, the maintenance time and maintenance resources can be reasonably arranged to avoid over-maintenance or under-maintenance. For example, when the remaining lifespan is short, major repairs or pavement replacement can be arranged in advance to ensure the safe operation of the pavement. Accurately calculating the remaining lifespan helps to optimize the operation resource allocation of the airport. The airport can reasonably arrange flight takeoff and landing plans according to the remaining lifespan of the pavement, avoid flight delays or cancellations caused by pavement damage, and improve the operation efficiency and service quality of the airport.

[0089] Optionally, the determining the dynamic risk map according to the failure probability and the remaining lifespan includes:

[0090] Constructing a two-dimensional risk matrix based on the failure probability and the remaining lifespan;

[0091] Generating a full-pavement dynamic risk heat map through a spatial interpolation algorithm based on the two-dimensional risk matrix;

[0092] Evaluate the risk fluctuation range of the full pavement under different environmental scenarios in combination with Monte Carlo simulation to obtain the risk confidence interval;

[0093] Generate a dynamic risk map according to the dynamic risk heat map and the risk confidence interval.

[0094] Based on two key indicators, namely the probability of failure and the remaining life, a two-dimensional risk matrix is constructed. Generally, the probability of failure is divided into different levels (such as low, medium, high), and the remaining life is also divided into corresponding intervals (such as short-term, medium-term, long-term). Each element in the matrix represents the risk level under a specific combination of the probability of failure and the remaining life. Through this matrix form, the risk levels faced by the pavement under different probabilities of failure and remaining lives can be visually displayed, providing a basic framework for subsequent risk assessment and visualization. For example, a combination of a high probability of failure and a short remaining life corresponds to a high risk level, while a combination of a low probability of failure and a long remaining life corresponds to a low risk level. Based on the two-dimensional risk matrix, a dynamic risk heat map of the entire pavement is generated through a spatial interpolation algorithm. The spatial interpolation algorithm estimates and interpolates the risks at other positions of the pavement according to the known risk matrix data points, so as to obtain the risk distribution of the entire pavement. The heat map represents the high and low risks through different colors or gray levels. The darker the color (or the larger the gray value), the higher the risk. The dynamic risk heat map can visually display the risk conditions in different areas of the pavement, enabling managers to quickly identify high-risk areas in order to take targeted maintenance measures. For example, in the heat map, the red area may indicate a high-risk area that requires priority maintenance and inspection. The risk fluctuation range of the entire pavement under different environmental scenarios is evaluated by combining Monte Carlo simulation. Monte Carlo simulation is a method of simulating the behavior of a system through random sampling. Here, various possible changes in environmental factors (such as temperature, humidity, load, etc.) are considered, and the probability of failure and the remaining life of the pavement under different environmental scenarios are calculated through multiple simulations, and then the risk fluctuation range is obtained. According to these fluctuation ranges, a risk confidence interval is determined, that is, the range within which the risk value may appear under a certain probability. Considering the uncertainty of environmental factors, the change of risk in different situations can be evaluated through Monte Carlo simulation, and the risk confidence interval can be obtained. This helps managers understand the reliability and uncertainty of risks and provides more comprehensive information for decision-making. For example, if the risk confidence interval is wide, it indicates that the uncertainty of the risk is large, and more cautious maintenance strategies need to be formulated. A dynamic risk atlas is generated based on the dynamic risk heat map and the risk confidence interval. The dynamic risk atlas combines the risk distribution shown in the heat map with the risk confidence interval, not only showing the current risk situation of the pavement, but also reflecting the uncertainty of the risk. The risk confidence interval can be represented in different ways in the atlas, such as using error bars, shaded areas, etc. The dynamic risk atlas provides comprehensive and intuitive risk information for pavement management. Managers can understand the risk levels and the uncertainty of risks in different areas of the pavement according to the atlas, so as to formulate more scientific and reasonable maintenance plans and decisions. For example, for areas with high risks and large risk confidence intervals, more conservative maintenance measures may be required to reduce potential risks.

[0095] Optionally, generating a hierarchical maintenance strategy according to the dynamic risk map and airport operation data includes:

[0096] Generating a mapping table of risk areas and maintenance actions according to the dynamic risk map;

[0097] generating a maintenance plan with time constraints according to the mapping table and the airport operation data;

[0098] A genetic algorithm is used to select a target solution with the lowest cost and the shortest downtime from the maintenance plan, and output maintenance time, maintenance location, maintenance action and resource allocation.

[0099] According to the dynamic risk map, the pavement is divided into different risk areas, such as high-risk areas, medium-risk areas and low-risk areas. For each risk area, the corresponding maintenance action is determined. For example, high-risk areas may require comprehensive structural inspection and repair, medium-risk areas may be partially repaired and monitored, and low-risk areas are routinely inspected. The correspondence between these risk areas and maintenance actions is organized into a mapping table. The mapping table provides clear guidance for the subsequent maintenance plan formulation, so that areas with different risk levels can receive matching maintenance treatments, ensure the rational allocation and effective use of maintenance resources, and improve the pertinence and effectiveness of maintenance work. The maintenance plan is generated by combining the mapping table with airport operation data. Airport operation data includes information such as flight take-off and landing times, passenger flow, and runway usage frequency. Based on these data, the time window in which each maintenance action can be executed is determined to avoid conflicts between maintenance work and normal airport operations. For example, during busy flight hours, large-scale maintenance operations should be avoided as much as possible to avoid affecting flight take-off and landing. At the same time, considering the continuity and timeliness of maintenance work, reasonable time constraints are set for each maintenance task to ensure that maintenance work can be completed on time. The maintenance plan with time constraints can balance the relationship between maintenance needs and airport operations. Under the premise of ensuring pavement safety and maintenance quality, it can minimize the impact on the normal operation of the airport and improve the airport's operating efficiency and service quality. Genetic algorithm is an optimization algorithm that simulates natural selection and genetic mechanisms. The generated maintenance plan is used as the initial population, with the lowest maintenance cost and the shortest downtime as the objective function. Through the selection, crossover and mutation operations of the genetic algorithm, the maintenance plan is continuously iterated and optimized, and the optimal target solution is selected from many possible solutions. Finally, the maintenance time, maintenance location, maintenance action and resource allocation of the target solution are output. Genetic algorithm can quickly search for the optimal solution or approximate optimal solution in a complex solution space. By taking cost and downtime as the optimization objectives, it can ensure that the selected maintenance solution can meet the pavement maintenance needs while minimizing the maintenance cost and the impact on airport operations, thereby improving the economic and social benefits of maintenance work.

[0100] The mapping table provides standardized operation guidelines for the maintenance of different risk areas, enabling maintenance personnel to clearly know the specific maintenance measures to be taken for different risk situations, avoiding blindness and randomness in maintenance work, and improving the standardization and consistency of maintenance work. When risks occur on the pavement, maintenance personnel can quickly determine the corresponding maintenance actions based on the mapping table, take timely measures to address the risks, prevent the risks from further expanding, and ensure the safe operation of the pavement. The maintenance plan with time constraints fully considers the operation needs of the airport, ensures that maintenance work is carried out without affecting the normal operation of the airport, reduces the impact of maintenance on flight takeoffs and landings, passenger travel, etc., and improves the operation efficiency and service quality of the airport. Reasonable arrangement of maintenance time can make full use of the idle resources of the airport, such as carrying out maintenance during the period with fewer night flights, avoiding the problem of resource shortage during the peak daytime period, and improving the utilization efficiency of resources. The genetic algorithm is optimized with the goal of minimizing costs. It can select the most economical maintenance plan on the premise of meeting maintenance requirements, reduce maintenance costs, and improve the economic benefits of the airport. By optimizing the algorithm to select the plan with the shortest downtime, the impact of pavement maintenance on airport operation can be minimized, and the economic losses and social impacts caused by downtime can be reduced. The output maintenance plan clarifies the maintenance time, location, actions, and resource allocation, enabling the reasonable and efficient use of maintenance resources, and avoiding waste and unreasonable allocation of resources.

[0101] This embodiment also discloses an airport pavement health monitoring system. Figure 2 It is a schematic diagram of the modules of an airport pavement health monitoring system disclosed in an embodiment of the present application. As Figure 2 shown, the system includes a collection module 201, a fusion module 202, a prediction module 203, and a maintenance module 204, where:

[0102] The collection module 201 is configured to obtain multi-source physical data from target sensors and radars, and obtain inspection images from inspection drones. The multi-source physical data includes sensor data and radar data;

[0103] The fusion module 202 is configured to perform spatio-temporal alignment on the multi-source physical data and the inspection images, and fuse the vibration signals in the spatio-temporally fused multi-source physical data with the visual features in the inspection images to obtain a disease feature vector. The vibration signals include the spectral features of pavement load responses, and the visual features include crack morphology and void area texture;

[0104] The prediction module 203 is configured to predict the failure probability and remaining life based on the pavement digital twin model according to the disease feature vector and real-time environmental data, and determine a dynamic risk map according to the failure probability and the remaining life. The real-time environmental data includes temperature, humidity, and flight load frequency;

[0105] A maintenance module 204, configured to generate a hierarchical maintenance strategy according to the dynamic risk map and airport operation data, where the airport operation data includes flight schedules and maintenance resource inventories.

[0106] Optionally, the fusion module 202 is configured to:

[0107] Match the acquisition timestamps and GPS coordinates of the sensor data and radar data with the shooting time and spatial position of the inspection images through a spatio-temporal alignment algorithm to establish a multi-source data mapping relationship under the same spatio-temporal reference;

[0108] Extract the spectral features of the vibration signal and extract the geometric morphological features of the cracks in the inspection images;

[0109] Use a multi-modal Transformer model to perform cross-modal fusion on the spectral features and the geometric morphological features to obtain a fused feature vector;

[0110] Perform dynamic noise filtering on the fused feature vector to obtain a disease feature vector, where the disease feature vector includes the category probability, position coordinates, and damage degree quantization value of the disease.

[0111] Optionally, the fusion module 202 is configured to:

[0112] Perform a short-time Fourier transform on the vibration signal to obtain a spectrum, map the spectrum to the Mel scale to calculate the Mel filter bank energy, take the logarithm of the Mel filter bank energy and perform a discrete cosine transform to obtain Mel frequency cepstral coefficient features;

[0113] Perform three-layer wavelet packet decomposition on the vibration signal to obtain basic spectral features, and splice the basic spectral features and the Mel frequency cepstral coefficient features to form the spectral features;

[0114] Extract the geometric parameters of the cracks through edge detection and contour fitting, where the geometric parameters include length, width, and curvature, and calculate the texture entropy value of the void area based on the gray-level co-occurrence matrix;

[0115] Obtain the geometric morphological features according to the geometric parameters and the texture entropy value.

[0116] Optionally, the fusion module 202 is configured to:

[0117] Project the spectral features into a shared semantic space to generate query vectors, and project the geometric morphological features into a shared semantic space to generate key-value pairs;

[0118] Generate the first attention of the spectral features to the geometric features and the second attention of the geometric features to the spectral features based on the query vector and the key-value pairs;

[0119] Concatenate the first attention and the second attention to obtain a fused feature vector.

[0120] Optionally, the prediction module 203 is configured to:

[0121] Input the disease feature vector into the pavement digital twin model, and dynamically update the model boundary conditions in combination with the real-time environmental data;

[0122] Solve the multi-physical field coupling equation using a physics-informed neural network, output the damage accumulation amount of the pavement structure within a future preset time step, and determine the failure probability based on the damage accumulation amount;

[0123] Calculate the remaining life based on the failure probability.

[0124] Optionally, the prediction module 203 is configured to:

[0125] Construct a two-dimensional risk matrix based on the failure probability and the remaining life;

[0126] Generate a full-pavement dynamic risk heat map through a spatial interpolation algorithm based on the two-dimensional risk matrix;

[0127] Evaluate the risk fluctuation range of the full pavement under different environmental scenarios in combination with Monte Carlo simulation to obtain a risk confidence interval;

[0128] Generate a dynamic risk atlas based on the dynamic risk heat map and the risk confidence interval.

[0129] Optionally, the maintenance module 204 is configured to:

[0130] Generate a mapping table of risk areas and maintenance actions based on the dynamic risk atlas;

[0131] Generate a time-constrained maintenance plan based on the mapping table and the airport operation data;

[0132] Use a genetic algorithm to select an optimal solution with the lowest cost and the shortest downtime from the maintenance plan, and output the maintenance time, maintenance location, maintenance actions, and resource allocation.

[0133] It should be noted that: when the device provided in the above embodiment realizes its functions, only the division of the above function modules is used for illustration. In actual applications, the above functions can be allocated to different function modules according to needs, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be elaborated here.

[0134] This embodiment also discloses an electronic device. Referring to Figure 3 , the electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, a network interface 304, and at least one memory 305.

[0135] Among them, the communication bus 302 is used to realize the connection and communication between these components.

[0136] Among them, the user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may further include a standard wired interface and a wireless interface.

[0137] Among them, the network interface 304 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0138] Among them, the processor 301 may include one or more processing cores. The processor 301 connects various parts within the entire server through various interfaces and lines, and by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, as well as calling data stored in the memory 305, it executes various functions of the server and processes data. Optionally, the processor 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 301 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 301 and may be implemented separately by a single chip.

[0139] Among them, the memory 305 may include a Random Access Memory (RAM), or may also include a Read-Only Memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area can store the data involved in the above-mentioned method embodiments. Optionally, the memory 305 may also be at least one storage device located far from the aforementioned processor 301. As Figure 3 shown, in the memory 305 as a computer storage medium, there may be included an operating system, a network communication module, a user interface module, and an application program of a method for monitoring the health of airport pavements.

[0140] In Figure 3 the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user to obtain the data input by the user; and the processor 301 can be used to call the application program of a method for monitoring the health of airport pavements stored in the memory 305. When executed by one or more processors 301, the electronic device is caused to execute the method of one or more of the above-mentioned embodiments.

[0141] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0142] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0143] In several embodiments provided in this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection between each other can be through some service interfaces. The indirect coupling or communication connection of devices or units can be in electrical or other forms.

[0144] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in various embodiments of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 305 and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of this application. And the aforementioned memory 305 includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0147] The above are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, all equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will easily think of other implementation schemes of the present disclosure after considering the disclosure of the specification. This application aims to cover any variations, uses, or adaptive changes of the present disclosure, and these variations, uses, or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An airport pavement health monitoring method, characterized in that, Applied to the airport pavement monitoring platform, the method includes: Obtain multi-source physical data from target sensors and radar, and obtain inspection images from inspection drones. The multi-source physical data includes sensor data and radar data. Perform spatio-temporal alignment on the multi-source physical data and the inspection images, and fuse the vibration signals in the spatio-temporally fused multi-source physical data with the visual features in the inspection images to obtain disease feature vectors. The vibration signals include spectral features of pavement load responses, and the visual features include crack morphology and void area texture. Based on the pavement digital twin model, predict the failure probability and remaining life according to the disease feature vectors and environmental real-time data, and determine the dynamic risk map according to the failure probability and the remaining life. The environmental real-time data includes temperature, humidity, and flight load frequency. Generate a hierarchical maintenance strategy according to the dynamic risk map and airport operation data. The airport operation data includes flight schedules and maintenance resource inventories. The performing spatio-temporal alignment on the multi-source physical data and the inspection images, and fusing the vibration signals in the spatio-temporally fused multi-source physical data with the visual features in the inspection images to obtain disease feature vectors includes: Through a spatio-temporal alignment algorithm, match the acquisition timestamps and GPS coordinates of the sensor data and radar data with the shooting time and spatial position of the inspection images, and establish a mapping relationship of multi-source data under the same spatio-temporal reference. Extract the spectral features of the vibration signals, and extract the geometric morphological features of cracks in the inspection images. Use a multi-modal Transformer model to perform cross-modal fusion on the spectral features and the geometric morphological features to obtain a fused feature vector. Perform dynamic noise filtering on the fused feature vector to obtain disease feature vectors. The disease feature vectors include the category probability, position coordinates, and damage degree quantization values of diseases. The predicting the failure probability and remaining life based on the pavement digital twin model according to the disease feature vectors and environmental real-time data includes: Input the disease feature vectors into the pavement digital twin model, and dynamically update the model boundary conditions in combination with the environmental real-time data. Use a physics-informed neural network to solve the multi-physics field coupling equation, output the damage accumulation amount of the pavement structure within a preset future time step, and determine the failure probability according to the damage accumulation amount. Calculate the remaining life according to the failure probability.

2. The airport pavement health monitoring method according to claim 1, wherein The extracting the spectral features of the vibration signals, and extracting the geometric morphological features of cracks in the inspection images includes: Perform short-time Fourier transform on the vibration signals to obtain a spectrum, map the spectrum to the Mel scale to calculate the Mel filter bank energy, take the logarithm of the Mel filter bank energy and perform discrete cosine transform to obtain Mel frequency cepstral coefficient features. Perform three-layer wavelet packet decomposition on the vibration signals to obtain basic spectral features, and splice the basic spectral features and the Mel frequency cepstral coefficient features to form the spectral features. Extract the geometric parameters of the crack through edge detection and contour fitting. The geometric parameters include length, width, and curvature, and calculate the texture entropy value of the debonding area based on the gray-level co-occurrence matrix; Obtain the geometric morphological features according to the geometric parameters and the texture entropy value.

3. The airport pavement health monitoring method according to claim 1, characterized in that, The cross-modal fusion of the spectral features and the geometric morphological features by using the multi-modal Transformer model to obtain the fused feature vector includes: Project the spectral features into the shared semantic space to generate query vectors, and project the geometric morphological features into the shared semantic space to generate key-value pairs; Generate the first attention of the spectral features to the geometric morphological features and the second attention of the geometric morphological features to the spectral features according to the query vectors and the key-value pairs; Concatenate the first attention and the second attention to obtain the fused feature vector.

4. The airport pavement health monitoring method according to claim 1, characterized in that The determination of the dynamic risk map according to the failure probability and the remaining life includes: Construct a two-dimensional risk matrix based on the failure probability and the remaining life; Generate a full-pavement dynamic risk heat map through a spatial interpolation algorithm based on the two-dimensional risk matrix; Evaluate the risk fluctuation range of the full-pavement under different environmental scenarios by combining Monte Carlo simulation to obtain the risk confidence interval; Generate a dynamic risk map according to the dynamic risk heat map and the risk confidence interval.

5. The airport pavement health monitoring method according to claim 1, wherein The generation of the hierarchical maintenance strategy according to the dynamic risk map and the airport operation data includes: Generate a mapping table of risk areas and maintenance actions according to the dynamic risk map; Generate a time-constrained maintenance plan according to the mapping table and the airport operation data; Use the genetic algorithm to select the target plan with the lowest cost and the shortest downtime from the maintenance plan, and output the maintenance time, maintenance location, maintenance actions, and resource allocation.

6. An airport pavement health monitoring system, characterized in that, Includes a collection module, a fusion module, a prediction module, and a maintenance module, where: The collection module is configured to obtain multi-source physical data from target sensors and radars, and obtain inspection images from inspection drones. The multi-source physical data includes sensor data and radar data; The fusion module is configured to perform spatio-temporal alignment on the multi-source physical data and the inspection images, and fuse the vibration signals in the spatio-temporally fused multi-source physical data with the visual features in the inspection images to obtain the disease feature vector. The vibration signals include the spectral features of the pavement load response, and the visual features include the crack morphology and the texture of the debonding area; The prediction module is configured to predict the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and the environmental real-time data, and determine the dynamic risk map according to the failure probability and the remaining life. The environmental real-time data includes temperature, humidity, and flight load frequency; The maintenance module is configured to generate a hierarchical maintenance strategy according to the dynamic risk map and the airport operation data. The airport operation data includes the flight schedule and the maintenance resource inventory, The fusion module is configured to: Through the spatio-temporal alignment algorithm, match the acquisition timestamps and GPS coordinates of the sensor data and radar data with the shooting time and spatial position of the inspection image, and establish a multi-source data mapping relationship under the same spatio-temporal reference; Extract the spectral features of the vibration signal, and extract the geometric morphological features of the cracks in the inspection image; Use a multi-modal Transformer model to perform cross-modal fusion on the spectral features and the geometric morphological features to obtain a fused feature vector; Perform dynamic noise filtering on the fused feature vector to obtain a disease feature vector, where the disease feature vector includes the category probability, position coordinates, and damage degree quantization value of the disease; The prediction module is configured to: Input the disease feature vector into the pavement digital twin model, and dynamically update the model boundary conditions in combination with the real-time environmental data; Use a physics-informed neural network to solve the multi-physics field coupling equation, output the damage accumulation amount of the pavement structure within a preset future time step, and determine the failure probability according to the damage accumulation amount; Calculate the remaining life according to the failure probability.

7. An electronic device, characterized in that, It includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1-5 is executed.

Images