Airfield pavement health monitoring method and system, electronic equipment and storage medium

Abstract

The invention discloses an airport pavement health monitoring method and system, electronic equipment and a storage medium, and relates to the field of airport pavement monitoring. The method comprises the following steps: acquiring multi-source physical data from a target sensor and a radar, and acquiring an inspection image from an inspection unmanned aerial vehicle; performing space-time alignment on the multi-source physical data and the inspection image, and fusing a vibration signal in the multi-source physical data after space-time fusion with a visual feature in the inspection image to obtain a disease feature vector; predicting the failure probability and the residual life according to the disease feature vector and the environment real-time data based on a pavement digital twinborn model, and determining a dynamic risk map according to the failure probability and the residual life; and generating a hierarchical maintenance strategy according to the dynamic risk map and airport operation data, wherein the airport operation data comprises a flight timetable and a maintenance resource inventory. By implementing the technical scheme provided by the invention, accurate evaluation and intelligent maintenance planning of the health state of the airport pavement are realized, and the monitoring efficiency and decision scientificity are improved.

Description

Technical Field

[0001] The present application relates to the technical field of airport pavement monitoring, and in particular to an airport pavement health monitoring method, system, electronic equipment and storage medium. Background Art

[0002] With the rapid development of the civil aviation industry, the health status of airport pavements, as key infrastructure, is directly related to flight safety and airport operation efficiency. In recent years, with the development of sensor technology and drone inspection technology, the health monitoring of airport pavements has gradually shifted from traditional visual inspection to intelligent monitoring with multi-source data fusion, greatly improving the monitoring accuracy and efficiency.

[0003] In the existing technology, in order to solve the problem of airport pavement health monitoring, the following methods are usually used: first, regular manual inspections, combined with visual observation and simple measuring tools to record pavement disease conditions; second, drones are used to take images and use image processing technology to identify surface diseases such as cracks. The main drawbacks of the above existing technologies are that they are inefficient and cannot accurately locate and quantitatively evaluate pavement diseases. At the same time, it is difficult to dynamically predict the risk of pavement failure in combination with environmental factors, resulting in a lack of scientific basis for maintenance decisions. Summary of the invention

[0004] The present application provides an airport pavement health monitoring method, system, electronic equipment and storage medium, which realizes accurate assessment of the health status of airport pavements and intelligent maintenance planning, improves monitoring efficiency and scientific decision-making, effectively extends the service life of the pavement and ensures the safety of airport operations.

[0005] In a first aspect of the present application, a method for monitoring the health of an airport pavement is provided, which is applied to an airport pavement monitoring platform. The method comprises: Acquire multi-source physical data from target sensors and radars, and acquire inspection images from inspection drones, wherein the multi-source physical data includes sensor data and radar data; The multi-source physical data and the inspection image are aligned in time and space, and the vibration signal in the multi-source physical data after time and space fusion is fused with the visual features in the inspection image to obtain a defect feature vector, wherein the vibration signal includes the frequency spectrum feature of the pavement load response, and the visual feature includes the crack morphology and the texture of the void area; Predicting the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and the real-time environmental data, and determining a dynamic risk map according to the failure probability and the remaining life, wherein the real-time environmental data includes temperature, humidity and flight load frequency; A hierarchical maintenance strategy is generated based on the dynamic risk map and airport operation data, wherein the airport operation data includes flight schedules and maintenance resource inventory.

[0006] Optionally, the step of performing spatiotemporal alignment of the multi-source physical data and the inspection image, and fusing the vibration signal in the spatiotemporal fused multi-source physical data with the visual features in the inspection image to obtain the disease feature vector includes: By using a spatiotemporal alignment algorithm, the acquisition timestamps and GPS coordinates of the sensor data and radar data are matched with the shooting time and spatial position of the inspection image to establish a multi-source data mapping relationship under the same spatiotemporal reference; Extracting frequency spectrum features of the vibration signal and extracting geometric features of cracks in the inspection image; A multimodal Transformer model is used to perform cross-modal fusion of the spectral features and the geometric features to obtain a fused feature vector; Dynamic noise filtering is performed on the fused feature vector to obtain a disease feature vector, wherein the disease feature vector includes a disease category probability, a location coordinate, and a quantized value of the damage degree.

[0007] Optionally, extracting the frequency spectrum features of the vibration signal and extracting the geometric features of the cracks in the inspection image includes: Performing short-time Fourier transform on the vibration signal to obtain a spectrum, mapping the spectrum to a Mel scale to calculate Mel filter bank energy, taking the logarithm of the Mel filter bank energy and performing discrete cosine transform to obtain Mel frequency cepstrum coefficient features; Performing three-layer wavelet packet decomposition on the vibration signal to obtain basic spectrum features, and concatenating the basic spectrum features with the Mel-frequency cepstral coefficient features to form the spectrum features; The geometric parameters of the cracks are extracted by edge detection and contour fitting, wherein the geometric parameters include length, width and curvature, and the texture entropy value of the void area is calculated based on the gray-level co-occurrence matrix; The geometric morphological feature is obtained according to the geometric parameter and the texture entropy value.

[0008] Optionally, the adopting a multimodal Transformer model to perform cross-modal fusion on the spectral feature and the geometric morphology feature to obtain a fused feature vector includes: Projecting the spectral features into a shared semantic space to generate a query vector, and projecting the geometric features into a shared semantic space to generate a key-value pair; Generate a first attention of the spectral feature to the geometric feature and a second attention of the geometric feature to the spectral feature according to the query vector and the key-value pair; The first attention and the second attention are concatenated to obtain a fused feature vector.

[0009] Optionally, the predicting of failure probability and remaining life based on the pavement digital twin model according to the disease feature vector and environmental real-time data includes: Inputting the disease feature vector into the pavement digital twin model, and dynamically updating the model boundary conditions in combination with the real-time environmental data; A physical information neural network is used to solve the multi-physics field coupling equation, output the damage accumulation of the pavement structure in a future preset time step, and determine the failure probability based on the damage accumulation; The remaining life is calculated based on the failure probability.

[0010] Optionally, determining a dynamic risk map according to the failure probability and the remaining life includes: Based on the failure probability and the remaining life, construct a two-dimensional risk matrix; Based on the two-dimensional risk matrix, a full road surface dynamic risk heat map is generated by a spatial interpolation algorithm; Combined with Monte Carlo simulation, the risk fluctuation range of the entire road surface in different environmental scenarios is evaluated to obtain the risk confidence interval; A dynamic risk map is generated according to the dynamic risk heat map and the risk confidence interval.

[0011] Optionally, generating a hierarchical maintenance strategy according to the dynamic risk map and airport operation data includes: Generating a mapping table of risk areas and maintenance actions according to the dynamic risk map; generating a maintenance plan with time constraints according to the mapping table and the airport operation data; A genetic algorithm is used to select a target solution with the lowest cost and the shortest downtime from the maintenance plan, and output maintenance time, maintenance location, maintenance action and resource allocation.

[0012] In a second aspect of the present application, an airport pavement health monitoring system is provided, including a collection module, a fusion module, a prediction module and a maintenance module, wherein: An acquisition module is configured to acquire multi-source physical data from target sensors and radars, and to acquire inspection images from inspection drones, wherein the multi-source physical data includes sensor data and radar data; A fusion module is configured to perform spatiotemporal alignment on the multi-source physical data and the inspection image, and to fuse the vibration signal in the spatiotemporal fused multi-source physical data with the visual features in the inspection image to obtain a defect feature vector, wherein the vibration signal includes a frequency spectrum feature of a pavement load response, and the visual features include a crack morphology and a texture of a void area; A prediction module is configured to predict the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and environmental real-time data, and determine a dynamic risk map according to the failure probability and the remaining life, wherein the environmental real-time data includes temperature, humidity and flight load frequency; A maintenance module is configured to generate a hierarchical maintenance strategy based on the dynamic risk map and airport operation data, wherein the airport operation data includes flight schedules and maintenance resource inventory.

[0013] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0014] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0015] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By acquiring multi-source physical data and inspection images from target sensors, radars and inspection drones, multi-source data fusion is achieved. This fusion not only enriches the types of monitoring data, but also improves the accuracy and reliability of the data, thereby being able to more comprehensively reflect the health of the airport pavement; the vibration signal is fused with the visual features in the inspection image to obtain a disease feature vector containing the pavement load response spectrum characteristics and crack morphology, void area texture and other information. This fusion method enhances the robustness and expression ability of the feature vector, which helps to more accurately identify pavement diseases; 2. Temporal and spatial alignment of multi-source physical data and inspection images ensures the temporal and spatial consistency of data from different sources, facilitating subsequent analysis and processing; the fusion of vibration signals and visual features further simplifies the data analysis process, improves analysis efficiency, and makes airport pavement health monitoring more efficient and real-time; 3. Use the pavement digital twin model to predict the failure probability and remaining life of the pavement based on the disease feature vector and real-time environmental data (such as temperature, humidity, and flight load frequency). This prediction method is based on big data and machine learning technology, and can more accurately assess the health status and potential risks of the pavement; by predicting the failure probability and remaining life, the airport can formulate a more scientific and reasonable maintenance plan to avoid over-maintenance or under-maintenance; 4. Determine the dynamic risk map based on the failure probability and remaining life, providing an intuitive risk assessment tool for airport operations. The dynamic risk map can reflect the risk status of the pavement in real time, helping airport managers make timely decisions; combined with airport operation data (such as flight schedules and maintenance resource inventory), a hierarchical maintenance strategy is generated. This strategy can reasonably arrange maintenance resources and time according to different risk levels and operational needs, and improve maintenance efficiency and economy; 5. Through real-time monitoring and prediction of pavement health, potential safety hazards can be discovered and dealt with in a timely manner to ensure the safety of airport operations; at the same time, the formulation and implementation of a hierarchical maintenance strategy can reduce unnecessary maintenance work and downtime, and improve the airport's operational efficiency and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of a method for monitoring the health of an airport pavement disclosed in an embodiment of the present application; Figure 2 It is a module schematic diagram of an airport pavement health monitoring system disclosed in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0017] Explanation of the reference numerals: 201, acquisition module; 202, fusion module; 203, prediction module; 204, maintenance module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0018] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0019] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0020] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0021] This embodiment discloses an airport pavement health monitoring method, which is applied to an airport pavement monitoring platform. Figure 1 is a flow chart of a method for monitoring the health of an airport pavement disclosed in an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps: S101, acquiring multi-source physical data from target sensors and radars, and acquiring inspection images from inspection drones, wherein the multi-source physical data includes sensor data and radar data; S102, aligning the multi-source physical data and the inspection image in time and space, and fusing the vibration signal in the multi-source physical data after time and space fusion with the visual features in the inspection image to obtain a defect feature vector, wherein the vibration signal includes the frequency spectrum feature of the pavement load response, and the visual feature includes the crack morphology and the texture of the void area; S103, predicting the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and the real-time environmental data, and determining a dynamic risk map according to the failure probability and the remaining life, wherein the real-time environmental data includes temperature, humidity and flight load frequency; S104: Generate a hierarchical maintenance strategy according to the dynamic risk map and airport operation data, wherein the airport operation data includes a flight schedule and a maintenance resource inventory.

[0022] Collect multi-source physical data from target sensors and radars. Sensor data may come from various types of sensors, such as sensors used to monitor pavement stress, strain, temperature and other parameters; radar data may be used to detect structural deformation, foreign objects and other conditions of the pavement. These data provide rich information about the physical state of the pavement for subsequent analysis. Use inspection drones to obtain inspection images. UAVs can take pictures of the pavement at different heights and angles to obtain pavement appearance images, which can intuitively show the surface condition of the pavement, such as the appearance characteristics of cracks, voids and other diseases. Align multi-source physical data and inspection images in time and space. Since multi-source data come from different devices and collection methods, they may differ in time and space. The purpose of time and space alignment is to ensure that these data are associated in the same time and space dimensions for subsequent effective fusion. For example, match sensor data at a certain moment with the inspection image at the corresponding position and time. Fuse the vibration signal in the multi-source physical data after time and space fusion with the visual features in the inspection image. The vibration signal contains the spectral characteristics of the pavement load response. By analyzing these spectral characteristics, we can understand the dynamic response of the pavement under load; the visual characteristics include crack morphology and texture of the void area. These characteristics are directly extracted from the inspection image and reflect the surface damage of the pavement. The two different types of features are fused to obtain a comprehensive damage feature vector, which can more comprehensively describe the damage state of the pavement. The pavement digital twin model is used to predict the failure probability and remaining life by combining the damage feature vector and environmental real-time data. The pavement digital twin model is a virtual mapping of the actual pavement. It can simulate the performance evolution process of the pavement according to the input damage characteristics and environmental factors. Environmental real-time data includes temperature, humidity, flight load frequency, etc. These factors have an important impact on the performance and life of the pavement. Through model calculation, the probability of failure of the pavement in the future and the remaining service life can be obtained. According to the predicted failure probability and remaining life, the dynamic risk map is determined. The dynamic risk map can intuitively display the risk status of different areas of the pavement at different times, providing a basis for subsequent maintenance decisions. For example, different colors may be used in the risk map to represent areas with different risk levels, with red representing high-risk areas that require priority maintenance. Combining dynamic risk maps with airport operation data: Generate a hierarchical maintenance strategy based on the dynamic risk map and airport operation data. Airport operation data includes information such as flight schedules and maintenance resource inventory. The flight schedule determines the frequency and time distribution of pavement use, while the maintenance resource inventory limits the manpower, material and financial resources available for maintenance. Taking these factors into consideration, different levels of maintenance strategies can be formulated. A hierarchical maintenance strategy means that maintenance work is divided into different priorities and types based on the risk level of the pavement and the actual situation of the airport.For example, for high-risk areas, emergency maintenance may need to be arranged between flights; for low-risk areas, regular inspections and preventive maintenance can be used. This can reasonably allocate maintenance resources, improve maintenance efficiency, and ensure the safe operation of airport pavements.

[0023] Optionally, the step of performing spatiotemporal alignment of the multi-source physical data and the inspection image, and fusing the vibration signal in the spatiotemporal fused multi-source physical data with the visual features in the inspection image to obtain the disease feature vector includes: By using a spatiotemporal alignment algorithm, the acquisition timestamps and GPS coordinates of the sensor data and radar data are matched with the shooting time and spatial position of the inspection image to establish a multi-source data mapping relationship under the same spatiotemporal reference; Extracting frequency spectrum features of the vibration signal and extracting geometric features of cracks in the inspection image; A multimodal Transformer model is used to perform cross-modal fusion of the spectral features and the geometric features to obtain a fused feature vector; Dynamic noise filtering is performed on the fused feature vector to obtain a disease feature vector, wherein the disease feature vector includes a disease category probability, a location coordinate, and a quantized value of the damage degree.

[0024] Using the spatiotemporal alignment algorithm, the acquisition timestamps of sensor data and radar data are matched with the shooting time of the inspection image. For example, if the sensor data is collected every 1 second, and the shooting time interval of the inspection image is not fixed, the algorithm will find the closest time point and associate the data of different data sources at the same or similar time. The acquisition location of sensor data and radar data is matched with the shooting spatial location of the inspection image through GPS coordinates. For sensors and radars, their data usually carry GPS information of the acquisition location; for inspection images, the spatial location corresponding to each image can be determined by the flight trajectory and shooting parameters of the drone. The algorithm calculates the distance between different data sources in space and associates data with similar locations. After matching time and space, a multi-source data mapping relationship under the same spatiotemporal benchmark is established. For example, the sensor data and radar data at a certain time and location are associated with the corresponding inspection image data to form a unified data set, which provides a basis for subsequent feature extraction and fusion. The vibration signal contains the dynamic response information of the pavement under the action of load. Through spectrum analysis, it can be decomposed into components of different frequencies, so as to extract the spectrum features reflecting the structural characteristics of the pavement. Signal processing techniques such as Fourier transform and wavelet transform are used to convert the vibration signal from the time domain to the frequency domain to obtain the spectrum of the signal. Key features such as frequency peak and frequency band energy are extracted from the spectrum. These features can reflect the mechanical properties of the pavement, such as stiffness and damping. The crack morphology in the inspection image is an important visual feature of pavement damage. The geometric shape, length, width and other information of the crack can be extracted through image processing technology, providing a basis for the identification and evaluation of the damage. First, the inspection image is preprocessed, such as grayscale and denoising, to enhance the edge information of the crack. Then, the edge detection algorithm (such as the Canny algorithm) is used to extract the edge of the crack, and then the contour of the crack is obtained through the contour tracking algorithm. Finally, the geometric parameters such as the length, width and area of ​​the crack are calculated as the geometric morphological features of the crack. The multimodal Transformer model is a deep learning model that can process data of different modalities (such as text, images, audio, etc.). In this scenario, it is used to cross-modally fuse the spectral features of the vibration signal and the geometric morphological features of the inspection image. The model can learn the association and complementary information between different modal features, thereby generating a more representative fused feature vector. The extracted spectral features and geometric morphological features are input into different branches of the multimodal Transformer model respectively. The model interacts and fuses different modal features through the self-attention mechanism to capture the semantic relationship between them. After being processed by multiple layers of Transformer layers, a fused feature vector is output, which integrates the information of the vibration signal and the inspection image. In the process of feature extraction and fusion, some noise may be introduced, which will affect the accuracy and reliability of the disease feature vector.The purpose of dynamic noise filtering is to remove these noises and improve the quality of the defect feature vector. The dynamic noise filtering algorithm is used to adaptively adjust the filtering parameters according to the statistical characteristics of the feature vector and the distribution of the noise. For example, the Kalman filter, particle filter and other algorithms can be used to filter the fused feature vector in real time. The filtered feature vector is the defect feature vector. The category probability of the defect indicates the probability that the defect corresponding to the feature vector belongs to different types (such as cracks, voids, etc.). By learning a large number of known defect samples, the model can predict the most likely type of the defect represented by the current feature vector. The location coordinates indicate the specific location of the defect on the pavement, usually expressed in the form of GPS coordinates or other spatial coordinates. This helps maintenance personnel accurately locate the defect location and carry out targeted maintenance. The damage degree quantification value quantitatively evaluates the damage degree of the defect, such as the width and depth of the crack, the area and volume of the void, etc. The damage degree quantification value can provide more detailed information for maintenance decisions and help determine the maintenance priority and method.

[0025] Data from different sources often differ in time and space. This time-space alignment operation can eliminate these differences, allowing multi-source data to be associated and analyzed in the same time-space dimension. The spectral characteristics of the vibration signal can reflect the dynamic response of the pavement under load. For example, the amplitude and phase information of different frequency components can reveal the internal defects and damage degree of the pavement structure. By analyzing the spectral characteristics, the vibration mode and energy distribution of the pavement can be understood, so as to judge the health of the pavement. The geometric features of cracks in the inspection image, such as the length, width, shape, etc. of the cracks, are important indicators that directly reflect the surface diseases of the pavement. Extracting these features can help determine the type, development degree and potential harm of the cracks, and provide a basis for the assessment and repair of the disease. The spectral characteristics of the vibration signal and the geometric features of the inspection image belong to different modes, and they contain different aspects of pavement disease information. Through cross-modal fusion, these two different types of information can be organically combined to achieve information complementarity. For example, spectral features may focus more on the internal structure and dynamic performance of the pavement, while geometric features more intuitively reflect the surface condition of the pavement. The fused feature vector can more comprehensively describe the pavement damage. The multimodal Transformer model has powerful feature learning and fusion capabilities. It can automatically learn the association and mapping relationship between different modal features to generate a more expressive fused feature vector. This fused feature vector can capture more complex damage patterns and improve the accuracy and reliability of damage identification. In the actual data collection process, various noises will inevitably be introduced, which will affect the quality of the feature vector. Dynamic noise filtering can adaptively remove noise and improve the signal-to-noise ratio of the feature vector based on the statistical characteristics of the feature vector and the distribution of noise. The final damage feature vector contains key information such as the category probability, location coordinates, and damage degree quantification value of the damage. The disease category probability can help determine the type of disease, such as cracks, voids, etc.; the location coordinates can accurately locate the location of the disease, which is convenient for subsequent maintenance and repair work; the damage degree quantification value can evaluate the severity of the disease and provide an important basis for formulating maintenance strategies. The comprehensive provision of this information makes the assessment of pavement damage more comprehensive and accurate.

[0026] Optionally, extracting the frequency spectrum features of the vibration signal and extracting the geometric features of the cracks in the inspection image includes: Performing short-time Fourier transform on the vibration signal to obtain a spectrum, mapping the spectrum to a Mel scale to calculate Mel filter bank energy, taking the logarithm of the Mel filter bank energy and performing discrete cosine transform to obtain Mel frequency cepstrum coefficient features; Performing three-layer wavelet packet decomposition on the vibration signal to obtain basic spectrum features, and concatenating the basic spectrum features with the Mel-frequency cepstral coefficient features to form the spectrum features; Extracting geometric parameters of the crack by edge detection and contour fitting, the geometric parameters include length, width and curvature, and calculating the texture entropy value of the void area based on the gray level co-occurrence matrix; The geometric morphological feature is obtained according to the geometric parameter and the texture entropy value.

[0027] Perform a short-time Fourier transform (STFT) on the vibration signal to convert the time domain signal into a frequency domain signal and obtain a spectrum. STFT segments the signal in time and performs a Fourier transform on each segment, thereby retaining both the time domain and frequency domain information of the signal. This allows the frequency components of the signal in different time periods to be analyzed, which helps to understand the spectral characteristics of the vibration signal over time. The obtained spectrum is mapped to the Mel scale. The Mel scale is a frequency scale that conforms to the auditory characteristics of the human ear. It has a higher resolution for low-frequency signals and a lower resolution for high-frequency signals. Then the energy of the Mel filter bank is calculated. The Mel filter bank is a group of bandpass filters evenly distributed on the Mel scale. By calculating the energy of the signal passing through each filter, the energy distribution of the signal in different Mel frequency bands can be obtained. The logarithm of the Mel filter bank energy is taken, which can compress the dynamic range of the energy and make the features more stable. Then a discrete cosine transform (DCT) is performed to convert the Mel filter bank energy into Mel frequency cepstral coefficients (MFCC). MFCC is a parameter that can effectively represent the characteristics of speech and vibration signals. It removes the correlation in the signal and retains the main characteristic information of the signal. The vibration signal is decomposed by three layers of wavelet packets to obtain the basic spectrum features. Wavelet packet decomposition is a more sophisticated signal decomposition method. It can decompose the signal into different frequency bands, and the number of decomposition layers can be selected as needed. Three-layer wavelet packet decomposition can decompose the signal into multiple sub-bands, each of which corresponds to a different frequency range, so as to obtain more detailed spectrum information. The basic spectrum features are spliced ​​with the MFCC features to form the final spectrum features. The basic spectrum features provide the energy distribution information of the signal in different frequency bands, while the MFCC features focus more on the voice and vibration characteristics of the signal. By splicing these two features, their advantages can be comprehensively utilized to obtain more comprehensive and accurate spectrum features. The edge of the crack in the inspection image is detected by the edge detection algorithm, and then the edge is fitted by the contour fitting algorithm (such as least squares fitting) to obtain the contour of the crack. Edge detection can highlight the area with drastic grayscale changes in the image, that is, the edge of the crack; contour fitting can connect discrete edge points into a smooth curve to more accurately describe the shape of the crack. According to the crack contour obtained by fitting, the geometric parameters of the crack are calculated, including length, width and curvature. The length can be obtained by calculating the distance between each point on the contour; the width can be measured in the vertical direction of the crack; and the curvature can reflect the degree of bending of the crack. These geometric parameters can intuitively describe the size and shape characteristics of the crack. The gray level co-occurrence matrix is ​​a statistical method for analyzing image texture features, which describes the spatial distribution of gray level pairs in the image. By calculating the gray level co-occurrence matrix of the void area in the inspection image, the texture information of the area can be obtained. The texture entropy value of the void area is calculated based on the gray level co-occurrence matrix.The texture entropy value reflects the complexity of the image texture. The larger the entropy value, the more complex the texture; the smaller the entropy value, the simpler the texture. For the hollow area, the texture entropy value can be used as a feature to describe the roughness and irregularity of its surface. The geometric features are obtained based on the calculated crack geometric parameters and the texture entropy value of the hollow area. These features combine the size, shape of the crack and the texture information of the hollow area, and can more comprehensively describe the geometric features of the defects in the inspection image, providing an important basis for subsequent defect identification and analysis.

[0028] The Mel scale is designed based on the human hearing perception characteristics. Mapping the spectrum to the Mel scale can better simulate the human ear's perception of sounds of different frequencies. This makes the extracted features more in line with human hearing habits and can more effectively capture the frequency components related to pavement defects in vibration signals. By calculating the energy of the Mel filter bank and performing discrete cosine transform, the original spectrum data is reduced in dimension and feature concentrated. The redundancy of the data is reduced, while the key feature information of the vibration signal is retained, the complexity of subsequent processing is reduced, and the computational efficiency is improved. The logarithm operation can compress the dynamic range of the data, making the features less sensitive to changes in the signal amplitude and enhancing the stability of the features. This is very useful for dealing with noise and signal fluctuations that may exist in the actual environment, and can improve the robustness of defect identification. Wavelet packet decomposition can perform multi-resolution analysis on vibration signals and can simultaneously capture the low-frequency and high-frequency components of the signal. The decomposition coefficients at different levels reflect the characteristics of the signal in different frequency bands, which helps to more comprehensively understand the spectrum distribution of pavement vibration signals and discover defect information at different scales. The basic spectrum features obtained by wavelet packet decomposition are spliced ​​with the Mel frequency cepstrum coefficient features to achieve complementarity between different features. The Mel frequency cepstral coefficient feature focuses on simulating human auditory perception, while the wavelet packet decomposition feature provides richer frequency band information. The combination of the two can more accurately describe the spectral characteristics of the vibration signal and improve the accuracy of disease identification. Combining the geometric parameters of the cracks and the texture entropy values ​​of the void area to form geometric morphological features can more comprehensively reflect the surface disease of the pavement. The geometric parameters describe the morphology of the cracks, and the texture entropy values ​​reflect the texture characteristics of the void area. The two complement each other and provide richer information for subsequent disease analysis and maintenance decisions. Different types of diseases may show different characteristics in geometric parameters and texture entropy values. By combining these features, the differentiation of disease characteristics can be improved, which helps to more accurately identify and classify pavement diseases.

[0029] Optionally, the adopting a multimodal Transformer model to perform cross-modal fusion on the spectral feature and the geometric morphology feature to obtain a fused feature vector includes: Projecting the spectral features into a shared semantic space to generate a query vector, and projecting the geometric features into a shared semantic space to generate a key-value pair; Generate a first attention of the spectral feature to the geometric feature and a second attention of the geometric feature to the spectral feature according to the query vector and the key-value pair; The first attention and the second attention are concatenated to obtain a fused feature vector.

[0030] Different modalities (spectral features and geometric features) are originally in different feature spaces, with different data distributions and representations. The shared semantic space is an abstract feature space that can map the features of different modalities to a unified semantic level, so that the features of different modalities can interact and compare effectively. Through specific projection operations (usually linear transformations), the spectral features and geometric features are converted into query vectors and key-value pairs respectively. The query vector can be regarded as a "question" to the target information, while the key-value pair contains the "answer" for matching and providing information. For example, the query vector after the spectral feature is projected may ask "which geometric features are related to the current spectral mode", and the key-value pair of the geometric feature provides the corresponding geometric information to answer this question. According to the query vector and the key-value pair, the first attention of the spectral feature to the geometric feature and the second attention of the geometric feature to the spectral feature are generated. The attention mechanism is a computational method that simulates human attention allocation. It can dynamically assign weights according to the correlation between different parts of the input. In multimodal fusion, the attention mechanism is used to measure the degree of association between features of different modalities. Guided by the query vector of the spectral feature, matching and calculation are performed in the key-value pairs of the geometric feature. By calculating the similarity between the query vector and the key vector (such as dot product similarity) and normalizing it with the softmax function, the attention weights of the geometric features for the spectral features are obtained. These weights represent the degree of relevance of each geometric feature element to the current spectral feature element. Then the attention weights are multiplied by the value vector and summed to obtain the first attention representation of the spectral features for the geometric features. This can be understood as the spectral features "paying attention" to the most relevant information from the geometric features. Similarly, guided by the query vector of the geometric features, matching and calculation are performed in the key-value pairs of the spectral features to obtain the second attention of the geometric features for the spectral features. This means that the geometric features "pay attention" to the most relevant information from the spectral features. The first attention and the second attention are concatenated to obtain a fused feature vector. Concatenation is a simple and effective feature fusion method that combines attention representations from different sources to form a more comprehensive feature vector. The first attention and the second attention reflect the interactive relationship between the spectral features and the geometric features from different perspectives. Concatenating them together can retain this information, so that the fused feature vector can simultaneously contain the features of the two modalities and the correlation information between them. The obtained fused feature vector integrates the information of spectral features and geometric features, and takes into account the relationship between them. This fused feature vector has stronger expression ability and discrimination, can better describe the characteristics of pavement diseases, and provide more effective input for subsequent tasks such as disease identification, classification and evaluation.

[0031] Spectral features and geometric features come from different modalities (vibration signals and images), and they have significant differences in data form, distribution, and semantic expression. By projecting them into a shared semantic space, the features of these two different modalities can be mapped into a unified semantic framework, eliminating the differences between the modalities, so that they can be compared and fused at the same semantic level. In a shared semantic space, features of different modalities can more easily find associations with each other. For example, some patterns in spectral features that reflect the internal structure of the pavement may correspond to the morphology of cracks in geometric features. By projecting them into a shared semantic space, these potential associations can be better captured, providing a more accurate basis for subsequent attention calculations. The first attention represents the degree of attention of spectral features to geometric features, and the second attention represents the degree of attention of geometric features to spectral features. By calculating bidirectional attention, the interactive information between the two modal features can be fully captured. For example, when judging pavement damage, some frequency components in spectral features may be closely related to the specific morphology of cracks in geometric features. The bidirectional attention mechanism can discover this association, thereby better understanding the nature of the damage. The attention mechanism can dynamically assign weights to different features based on the correlation between the features. For features that are more relevant to the current task, higher weights are assigned to highlight the role of these features in the fusion process. This dynamic weighting method can improve the pertinence and effectiveness of the fused features, so that the fused feature vector can better reflect the key information of pavement diseases. By splicing bidirectional attention, the fused feature vector retains the rich interactive information between spectral features and geometric features. This information not only contains the characteristics of the two modal features themselves, but also contains the relationship between them, providing a more comprehensive basis for subsequent disease identification and analysis. The spliced ​​fused feature vector has stronger expressive power and can better describe the complex situation of pavement diseases. For example, in practical applications, different types of diseases may show different patterns in spectral features and geometric features. The fused feature vector can integrate this information, more accurately distinguish different types of diseases, and improve the accuracy and reliability of disease identification.

[0032] Optionally, the predicting of failure probability and remaining life based on the pavement digital twin model according to the disease feature vector and environmental real-time data includes: Inputting the disease feature vector into the pavement digital twin model, and dynamically updating the model boundary conditions in combination with the real-time environmental data; A physical information neural network is used to solve the multi-physics field coupling equation, output the damage accumulation of the pavement structure in a future preset time step, and determine the failure probability based on the damage accumulation; The remaining life is calculated based on the failure probability.

[0033] The disease feature vector is input into the pavement digital twin model. The disease feature vector contains key information about pavement diseases, such as the probability of disease category, location coordinates, and quantitative values ​​of damage degree. This information can reflect the current health status of the pavement and provide basic data for the model's prediction. Real-time environmental data includes temperature, humidity, and flight load frequency. These factors have a significant impact on the performance and life of the pavement. For example, high temperature will cause the pavement material to expand and increase internal stress; humidity will affect the strength and durability of the material; and flight load frequency is directly related to the fatigue damage suffered by the pavement. The pavement digital twin model needs to dynamically update its boundary conditions based on these real-time environmental data. Boundary conditions refer to external constraints and initial conditions set in the model calculation. By updating the boundary conditions, the model can more accurately simulate the working state of the pavement in the actual environment, thereby improving the accuracy of the prediction. Physical information neural network is a new modeling method that combines physical knowledge and neural networks. It can not only utilize the powerful fitting ability of neural networks, but also integrate physical laws and equations into the network training process to ensure that the model's prediction results conform to physical laws. In actual work, the pavement structure will be affected by multiple physical fields, such as mechanical fields and thermal fields. These physical fields are coupled with each other and jointly affect the performance of the pavement. The multi-physics field coupling equation describes the interaction between these physical fields. By using the physical information neural network to solve the multi-physics field coupling equation, the damage accumulation of the pavement structure in the future preset time step can be obtained. During the solution process, the neural network will continuously adjust its own parameters according to the input disease feature vector and the updated boundary conditions, so that the prediction results meet the physical equations and boundary conditions. The damage accumulation reflects the degree of damage suffered by the pavement structure in the future. It is a comprehensive indicator that takes into account the impact of multiple factors on the pavement performance. As time goes by, the damage accumulation will continue to increase. When it reaches a certain level, the pavement may fail. According to the damage accumulation, a certain probability model or statistical method can be used to determine the failure probability of the pavement. For example, a mapping relationship between the damage accumulation and the failure probability can be established, and the failure probability under the current damage accumulation can be obtained by looking up a table or calculating. The failure probability indicates the possibility of failure of the pavement in the future. The remaining life refers to the remaining time from the current moment to the failure of the pavement. It is an important indicator for evaluating the service life of pavement and formulating maintenance strategies. According to the failure probability, the remaining life can be calculated by using reliability theory or survival analysis. For example, it can be assumed that the failure time of the pavement follows a certain probability distribution, and then the expected value or confidence interval of the remaining life can be calculated based on the current failure probability and the parameters of the probability distribution.

[0034] The disease feature vector contains detailed information about pavement diseases, such as disease type, location, and degree of damage. Inputting this information into the digital twin model can accurately simulate the impact of diseases on the pavement structure performance, making the model closer to the actual condition of the pavement. Real-time environmental data will have an important impact on the performance and life of the pavement. Dynamically updating the model boundary conditions can enable the model to reflect the changes in environmental factors in real time and improve the accuracy and reliability of the simulation. The pavement damage process involves the interaction of multiple physical fields, such as mechanics, thermal, and acoustics. The physical information neural network can solve the multi-physics field coupling equations and comprehensively consider the mutual influence between these physical fields, so as to more accurately predict the damage accumulation of the pavement. For example, under the action of aircraft loads, the pavement will produce a mechanical response, and the change in temperature will also affect the performance of the pavement material. The multi-physics field coupling model can fully consider these factors. The physical information neural network combines physical knowledge with the learning ability of neural networks. It can use physical laws to constrain and improve the accuracy of prediction when data is limited. By outputting the damage accumulation within the preset time step in the future, the development trend of pavement damage can be clearly understood, providing a reliable basis for the subsequent failure probability and remaining life prediction. Failure probability is an important indicator for measuring pavement safety. By determining the failure probability based on the damage accumulation, the failure risk of the pavement can be quantified, allowing decision makers to understand the safety status of the pavement more intuitively. For example, when the failure probability is high, it means that there is a greater safety risk on the pavement and maintenance measures need to be taken in time. The determination of failure probability is based on the multi-physics field coupling equation and damage accumulation solved by the physical information neural network. This method has a scientific theoretical basis, avoids the uncertainty of subjective judgment, and improves the accuracy and reliability of the evaluation. The calculation of remaining life can provide an important reference for the maintenance plan of the pavement. By understanding the remaining life of the pavement, maintenance time and maintenance resources can be reasonably arranged to avoid excessive or insufficient maintenance. For example, when the remaining life is short, overhaul or replacement of the pavement can be arranged in advance to ensure the safe operation of the pavement. Accurate calculation of remaining life helps to optimize the allocation of operational resources at the airport. Airports can reasonably arrange flight take-off and landing plans based on the remaining life of the pavement, avoid flight delays or cancellations due to pavement damage, and improve the airport's operational efficiency and service quality.

[0035] Optionally, determining a dynamic risk map according to the failure probability and the remaining life includes: Based on the failure probability and the remaining life, construct a two-dimensional risk matrix; Based on the two-dimensional risk matrix, a full road surface dynamic risk heat map is generated by a spatial interpolation algorithm; Combined with Monte Carlo simulation, the risk fluctuation range of the entire road surface in different environmental scenarios is evaluated to obtain the risk confidence interval; A dynamic risk map is generated according to the dynamic risk heat map and the risk confidence interval.

[0036] Based on the two key indicators of failure probability and remaining life, a two-dimensional risk matrix is ​​constructed. Usually, the failure probability is divided into different levels (such as low, medium, and high), and the remaining life is also divided into corresponding intervals (such as short, medium, and long). Each element in the matrix represents the risk level under a specific combination of failure probability and remaining life. Through this matrix form, the risk level faced by the pavement under different failure probabilities and remaining life can be intuitively displayed, providing a basic framework for subsequent risk assessment and visualization. For example, a combination of high failure probability and short remaining life corresponds to a high risk level, while a combination of low failure probability and long remaining life corresponds to a low risk level. Based on the two-dimensional risk matrix, a dynamic risk heat map of the entire pavement is generated through a spatial interpolation algorithm. The spatial interpolation algorithm estimates and interpolates the risks of other locations on the pavement based on the known risk matrix data points, thereby obtaining the risk distribution of the entire pavement. The heat map uses different colors or grayscale levels to indicate the level of risk. The darker the color (or the larger the grayscale value), the higher the risk. The dynamic risk heat map can intuitively display the risk status of different areas of the pavement, allowing managers to quickly identify high-risk areas so that targeted maintenance measures can be taken. For example, in the heat map, the red area may indicate a high-risk area, which needs to be maintained and inspected first. Combined with Monte Carlo simulation, the risk fluctuation range of the entire pavement in different environmental scenarios is evaluated. Monte Carlo simulation is a method of simulating system behavior through random sampling. Here, various possible changes in environmental factors (such as temperature, humidity, load, etc.) are considered, and the failure probability and remaining life of the pavement in different environmental scenarios are calculated through multiple simulations to obtain the risk fluctuation range. Based on these fluctuation ranges, the risk confidence interval is determined, that is, the range in which the risk value may appear under a certain probability. Taking into account the uncertainty of environmental factors, Monte Carlo simulation can be used to evaluate the changes in risk under different circumstances and obtain the risk confidence interval. This helps managers understand the reliability and uncertainty of risks and provide more comprehensive information for decision-making. For example, if the risk confidence interval is wide, it means that the uncertainty of the risk is large, and maintenance strategies need to be formulated more cautiously. Generate a dynamic risk map based on the dynamic risk heat map and the risk confidence interval. The dynamic risk map combines the risk distribution shown in the heat map with the risk confidence interval, which not only shows the current risk status of the pavement, but also reflects the uncertainty of the risk. The risk confidence interval can be represented in different ways in the map, such as error bars, shaded areas, etc. The dynamic risk map provides comprehensive and intuitive risk information for pavement management. Managers can understand the risk level and risk uncertainty of different areas of the pavement based on the map, so as to formulate more scientific and reasonable maintenance plans and decisions. For example, for areas with high risks and large risk confidence intervals, more conservative maintenance measures may be required to reduce potential risks.

[0037] Optionally, generating a hierarchical maintenance strategy according to the dynamic risk map and airport operation data includes: Generating a mapping table of risk areas and maintenance actions according to the dynamic risk map; generating a maintenance plan with time constraints according to the mapping table and the airport operation data; A genetic algorithm is used to select a target solution with the lowest cost and the shortest downtime from the maintenance plan, and output maintenance time, maintenance location, maintenance action and resource allocation.

[0038] According to the dynamic risk map, the pavement is divided into different risk areas, such as high-risk areas, medium-risk areas and low-risk areas. For each risk area, the corresponding maintenance action is determined. For example, high-risk areas may require comprehensive structural inspection and repair, medium-risk areas may be partially repaired and monitored, and low-risk areas are routinely inspected. The correspondence between these risk areas and maintenance actions is organized into a mapping table. The mapping table provides clear guidance for the subsequent maintenance plan formulation, so that areas with different risk levels can receive matching maintenance treatments, ensure the rational allocation and effective use of maintenance resources, and improve the pertinence and effectiveness of maintenance work. The maintenance plan is generated by combining the mapping table with airport operation data. Airport operation data includes information such as flight take-off and landing times, passenger flow, and runway usage frequency. Based on these data, the time window in which each maintenance action can be executed is determined to avoid conflicts between maintenance work and normal airport operations. For example, during busy flight hours, large-scale maintenance operations should be avoided as much as possible to avoid affecting flight take-off and landing. At the same time, considering the continuity and timeliness of maintenance work, reasonable time constraints are set for each maintenance task to ensure that maintenance work can be completed on time. The maintenance plan with time constraints can balance the relationship between maintenance needs and airport operations. Under the premise of ensuring pavement safety and maintenance quality, it can minimize the impact on the normal operation of the airport and improve the airport's operating efficiency and service quality. Genetic algorithm is an optimization algorithm that simulates natural selection and genetic mechanisms. The generated maintenance plan is used as the initial population, with the lowest maintenance cost and the shortest downtime as the objective function. Through the selection, crossover and mutation operations of the genetic algorithm, the maintenance plan is continuously iterated and optimized, and the optimal target solution is selected from many possible solutions. Finally, the maintenance time, maintenance location, maintenance action and resource allocation of the target solution are output. Genetic algorithm can quickly search for the optimal solution or approximate optimal solution in a complex solution space. By taking cost and downtime as the optimization objectives, it can ensure that the selected maintenance solution can meet the pavement maintenance needs while minimizing the maintenance cost and the impact on airport operations, thereby improving the economic and social benefits of maintenance work.

[0039] The mapping table provides a standardized operation guide for the maintenance of different risk areas, so that maintenance personnel can clearly know the specific maintenance measures to be taken for different risk situations, avoid the blindness and arbitrariness of maintenance work, and improve the standardization and consistency of maintenance work. When the pavement is at risk, maintenance personnel can quickly determine the corresponding maintenance actions based on the mapping table, take timely measures to deal with the risk, prevent the risk from further expanding, and ensure the safe operation of the pavement. The maintenance plan with time constraints fully considers the operational needs of the airport, ensures that the maintenance work is carried out without affecting the normal operation of the airport, reduces the impact of maintenance on flight takeoffs and landings, passenger travel, etc., and improves the airport's operational efficiency and service quality. Reasonable arrangement of maintenance time can make full use of the airport's idle resources, such as maintenance at night when there are fewer flights, avoiding the problem of resource shortage during peak hours during the day, and improving resource utilization efficiency. The genetic algorithm optimizes with the lowest cost as the goal, and can select the most economical maintenance plan on the premise of meeting maintenance needs, reduce maintenance costs, and improve the economic benefits of the airport. By selecting the plan with the shortest downtime through the optimization algorithm, the impact of pavement maintenance on airport operations can be minimized, and the economic losses and social impacts caused by downtime can be reduced. The output maintenance plan specifies the maintenance time, location, action and resource allocation, so that maintenance resources can be used reasonably and efficiently, avoiding waste and unreasonable allocation of resources.

[0040] This embodiment also discloses an airport pavement health monitoring system. Figure 2 is a module schematic diagram of an airport pavement health monitoring system disclosed in an embodiment of the present application, such as Figure 2 As shown, the system includes a collection module 201, a fusion module 202, a prediction module 203 and a maintenance module 204, wherein: The acquisition module 201 is configured to acquire multi-source physical data from target sensors and radars, and acquire inspection images from inspection drones, wherein the multi-source physical data includes sensor data and radar data; A fusion module 202 is configured to perform spatiotemporal alignment of the multi-source physical data and the inspection image, and to fuse the vibration signal in the spatiotemporal fused multi-source physical data with the visual features in the inspection image to obtain a defect feature vector, wherein the vibration signal includes a frequency spectrum feature of a pavement load response, and the visual features include a crack morphology and a texture of a void area; A prediction module 203 is configured to predict the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and the environmental real-time data, and determine a dynamic risk map according to the failure probability and the remaining life, wherein the environmental real-time data includes temperature, humidity and flight load frequency; The maintenance module 204 is configured to generate a hierarchical maintenance strategy based on the dynamic risk map and airport operation data, wherein the airport operation data includes flight schedules and maintenance resource inventory.

[0041] Optionally, the fusion module 202 is configured to: By using a spatiotemporal alignment algorithm, the acquisition timestamps and GPS coordinates of the sensor data and radar data are matched with the shooting time and spatial position of the inspection image to establish a multi-source data mapping relationship under the same spatiotemporal reference; Extracting frequency spectrum features of the vibration signal and extracting geometric features of cracks in the inspection image; A multimodal Transformer model is used to perform cross-modal fusion of the spectral features and the geometric features to obtain a fused feature vector; Dynamic noise filtering is performed on the fused feature vector to obtain a disease feature vector, wherein the disease feature vector includes a disease category probability, a location coordinate, and a quantized value of the damage degree.

[0042] Optionally, the fusion module 202 is configured to: Performing short-time Fourier transform on the vibration signal to obtain a spectrum, mapping the spectrum to a Mel scale to calculate Mel filter bank energy, taking the logarithm of the Mel filter bank energy and performing discrete cosine transform to obtain Mel frequency cepstrum coefficient features; Performing three-layer wavelet packet decomposition on the vibration signal to obtain basic spectrum features, and concatenating the basic spectrum features with the Mel-frequency cepstral coefficient features to form the spectrum features; Extracting geometric parameters of the crack by edge detection and contour fitting, the geometric parameters include length, width and curvature, and calculating the texture entropy value of the void area based on the gray level co-occurrence matrix; The geometric morphological feature is obtained according to the geometric parameter and the texture entropy value.

[0043] Optionally, the fusion module 202 is configured to: Projecting the spectral features into a shared semantic space to generate a query vector, and projecting the geometric features into a shared semantic space to generate a key-value pair; Generate a first attention of the spectral feature to the geometric feature and a second attention of the geometric feature to the spectral feature according to the query vector and the key-value pair; The first attention and the second attention are concatenated to obtain a fused feature vector.

[0044] Optionally, the prediction module 203 is configured to: Inputting the disease feature vector into the pavement digital twin model, and dynamically updating the model boundary conditions in combination with the real-time environmental data; A physical information neural network is used to solve the multi-physics field coupling equation, output the damage accumulation of the pavement structure in a future preset time step, and determine the failure probability based on the damage accumulation; The remaining life is calculated based on the failure probability.

[0045] Optionally, the prediction module 203 is configured to: Based on the failure probability and the remaining life, construct a two-dimensional risk matrix; Based on the two-dimensional risk matrix, a full road surface dynamic risk heat map is generated by a spatial interpolation algorithm; Combined with Monte Carlo simulation, the risk fluctuation range of the entire road surface in different environmental scenarios is evaluated to obtain the risk confidence interval; A dynamic risk map is generated according to the dynamic risk heat map and the risk confidence interval.

[0046] Optionally, the maintenance module 204 is configured to: Generating a mapping table of risk areas and maintenance actions according to the dynamic risk map; generating a maintenance plan with time constraints according to the mapping table and the airport operation data; A genetic algorithm is used to select a target solution with the lowest cost and the shortest downtime from the maintenance plan, and output maintenance time, maintenance location, maintenance action and resource allocation.

[0047] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0048] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0049] The communication bus 302 is used to realize the connection and communication between these components.

[0050] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0051] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0052] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0053] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be optionally at least one storage device located away from the aforementioned processor 301. As Figure 3As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of an airport pavement health monitoring method.

[0054] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call an application program storing a method for monitoring the health of an airport pavement in the memory 305. When executed by one or more processors 301, the electronic device executes one or more methods in the above-mentioned embodiments.

[0055] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0056] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0057] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0058] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0059] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0060] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory 305 and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory 305 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0061] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for monitoring the health of an airport pavement, characterized in that: Applied to an airport pavement monitoring platform, the method comprises: Acquire multi-source physical data from target sensors and radars, and acquire inspection images from inspection drones, wherein the multi-source physical data includes sensor data and radar data; The multi-source physical data and the inspection image are aligned in time and space, and the vibration signal in the multi-source physical data after time and space fusion is fused with the visual features in the inspection image to obtain a defect feature vector, wherein the vibration signal includes the frequency spectrum feature of the pavement load response, and the visual feature includes the crack morphology and the texture of the void area; Predicting the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and the real-time environmental data, and determining a dynamic risk map according to the failure probability and the remaining life, wherein the real-time environmental data includes temperature, humidity and flight load frequency; A hierarchical maintenance strategy is generated based on the dynamic risk map and airport operation data, wherein the airport operation data includes flight schedules and maintenance resource inventory.

2. The airport pavement health monitoring method according to claim 1, characterized in that: The step of performing spatiotemporal alignment of the multi-source physical data and the inspection image, and fusing the vibration signal in the spatiotemporal fused multi-source physical data with the visual features in the inspection image to obtain the disease feature vector includes: By using a spatiotemporal alignment algorithm, the acquisition timestamps and GPS coordinates of the sensor data and radar data are matched with the shooting time and spatial position of the inspection image to establish a multi-source data mapping relationship under the same spatiotemporal reference; Extracting frequency spectrum features of the vibration signal and extracting geometric features of cracks in the inspection image; A multimodal Transformer model is used to perform cross-modal fusion of the spectral features and the geometric features to obtain a fused feature vector; Dynamic noise filtering is performed on the fused feature vector to obtain a disease feature vector, wherein the disease feature vector includes a disease category probability, a location coordinate, and a quantized value of the damage degree.

3. The airport pavement health monitoring method according to claim 2, characterized in that: The step of extracting the frequency spectrum feature of the vibration signal and the geometric features of the cracks in the inspection image comprises: Performing short-time Fourier transform on the vibration signal to obtain a spectrum, mapping the spectrum to a Mel scale to calculate Mel filter bank energy, taking the logarithm of the Mel filter bank energy and performing discrete cosine transform to obtain Mel frequency cepstrum coefficient features; Performing three-layer wavelet packet decomposition on the vibration signal to obtain basic spectrum features, and concatenating the basic spectrum features with the Mel-frequency cepstral coefficient features to form the spectrum features; The geometric parameters of the cracks are extracted by edge detection and contour fitting, wherein the geometric parameters include length, width and curvature, and the texture entropy value of the void area is calculated based on the gray-level co-occurrence matrix; The geometric morphological feature is obtained according to the geometric parameter and the texture entropy value.

4. The airport pavement health monitoring method according to claim 2, characterized in that: The adopting a multimodal Transformer model to cross-modally fuse the spectral feature and the geometric morphology feature to obtain a fused feature vector includes: Projecting the spectral features into a shared semantic space to generate a query vector, and projecting the geometric features into a shared semantic space to generate a key-value pair; Generate a first attention of the spectral feature to the geometric feature and a second attention of the geometric feature to the spectral feature according to the query vector and the key-value pair; The first attention and the second attention are concatenated to obtain a fused feature vector.

5. The airport pavement health monitoring method according to claim 1, characterized in that: The method of predicting the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and the real-time environmental data includes: Inputting the disease feature vector into the pavement digital twin model, and dynamically updating the model boundary conditions in combination with the real-time environmental data; A physical information neural network is used to solve the multi-physics field coupling equation, output the damage accumulation of the pavement structure in a future preset time step, and determine the failure probability based on the damage accumulation; The remaining life is calculated based on the failure probability.

6. The airport pavement health monitoring method according to claim 5, characterized in that: Determining a dynamic risk map according to the failure probability and the remaining life includes: Based on the failure probability and the remaining life, construct a two-dimensional risk matrix; Based on the two-dimensional risk matrix, a full road surface dynamic risk heat map is generated by a spatial interpolation algorithm; Combined with Monte Carlo simulation, the risk fluctuation range of the entire road surface in different environmental scenarios is evaluated to obtain the risk confidence interval; A dynamic risk map is generated according to the dynamic risk heat map and the risk confidence interval.

7. The airport pavement health monitoring method according to claim 1, characterized in that: Generating a hierarchical maintenance strategy according to the dynamic risk map and airport operation data includes: Generating a mapping table of risk areas and maintenance actions according to the dynamic risk map; generating a maintenance plan with time constraints according to the mapping table and the airport operation data; A genetic algorithm is used to select a target solution with the lowest cost and the shortest downtime from the maintenance plan, and output maintenance time, maintenance location, maintenance action and resource allocation.

8. An airport pavement health monitoring system, characterized in that: It includes acquisition module, fusion module, prediction module and maintenance module, among which: An acquisition module is configured to acquire multi-source physical data from target sensors and radars, and to acquire inspection images from inspection drones, wherein the multi-source physical data includes sensor data and radar data; A fusion module is configured to perform spatiotemporal alignment on the multi-source physical data and the inspection image, and to fuse the vibration signal in the spatiotemporal fused multi-source physical data with the visual features in the inspection image to obtain a defect feature vector, wherein the vibration signal includes a frequency spectrum feature of a pavement load response, and the visual features include a crack morphology and a texture of a void area; A prediction module is configured to predict the failure probability and the remaining life based on the pavement digital twin model according to the disease feature vector and environmental real-time data, and determine a dynamic risk map according to the failure probability and the remaining life, wherein the environmental real-time data includes temperature, humidity and flight load frequency; A maintenance module is configured to generate a hierarchical maintenance strategy based on the dynamic risk map and airport operation data, wherein the airport operation data includes flight schedules and maintenance resource inventory.

9. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is executed.

Images