Self-adaptive airport runway underground structure disease automatic detection method
By using YOLOv5 model and unsupervised field adaptive technology in the detection of underground structures of airport runways, the problems of high cost of manual interpretation data and reduced migration performance are solved, and more efficient and accurate disease detection is achieved, suitable for different airport data sets .
Patent Information
- Application Number
- CN202410979173.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-22
AI Technical Summary
The existing disease detection method for underground structures of airport runways has high cost, time-consuming and heavy interference from clutter noise. Traditional machine learning algorithms require manual design of features and cannot effectively utilize unlabeled data, and the model is in different airport data sets. problem of degradation of migration performance.
Using the YOLOv5-based runway underground structure disease detection model, combining dual-stream backbone network, self-attention module and FPN neck network, the inter-domain distance is calculated through unsupervised field adaptive technology MMD, reducing dependence on labeled data, and using multi-channel exploration The ground radar robot collects data and preprocesses it, extracts the characteristics of the source domain and the target domain for multi-scale feature fusion, and performs underground disease location and classification.
It significantly reduces detection costs and time, improves the ability to generalize different data sets, can more accurately identify runway underground diseases, improves recognition accuracy and adapts to different airport data sets.
Smart Images

Figure CN119027716A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of civil aviation technology, and in particular relates to an adaptive automatic detection method for underground structure defects of airport runways. Background Art
[0002] The runway is an important part of the airport and is the direct operating area for aircraft takeoff and landing. Therefore, the structural integrity of the airport runway plays a vital role in ensuring the safety of aircraft during flight. However, due to factors such as the load caused by high-frequency operation of aircraft, the infiltration of groundwater, and the shrinkage and expansion effects caused by extreme temperature changes, the airport runway will inevitably have various underground structural diseases. These diseases will reduce the structural strength and integrity of the runway, seriously affect the service life of the runway, and even pose a threat to aviation safety and passenger safety. Therefore, it is necessary for us to detect these underground diseases to ensure the safety of daily aircraft operations.
[0003] Currently, the commonly used airport pavement inspection methods include manual inspection, core sampling, ultrasonic testing, and drop hammer deflectometer. Ground-penetrating radar (GPR) is a non-destructive technology that is fast, safe, and accurate. It is currently a commonly used detection device in airport runway inspections. However, there are still many challenges in detecting underground structural defects of airport runways based on GPR.
[0004] In the early days, ground penetrating radar data was mainly interpreted manually. Due to the complexity and heterogeneity of the underground environment, coupled with the noise generated by the ground penetrating radar equipment itself, the collected ground penetrating radar data is both complex and prone to defects and distortion. These interferences make manual interpretation challenging. Therefore, manual interpretation of radar data is expensive and time-consuming.
[0005] The raw data collected by GPR is signal data, so many works on GPR data interpretation are based on standard signal processing methods. For example, identifying defects by extracting the time-frequency characteristics of GPR signals, identifying voids by continuous wavelet transform, and detecting shallow pavement by shear wave characteristics. However, the biggest problem faced by these signal processing-based methods is that they are very susceptible to interference from clutter and noise.
[0006] In addition to signal processing-based methods, there are also many studies based on traditional machine learning. For example: using support vector machines (SVM) to detect hyperbolic features, designing a radial basis function-based classifier to identify GPR targets, and applying open scan clustering algorithms to extract reflection hyperbolic feature points. However, these traditional machine learning algorithms often require manual feature design, which is very time-consuming and laborious. Moreover, these algorithms often target a single target such as hyperbolic reflection, because this target has obvious hyperbolic performance characteristics that are different from other underground targets. In fact, this simplifies the actual application scenario.
[0007] Thanks to the powerful automatic feature extraction capability of deep learning networks, more and more scientific research works choose to use target detection algorithms to detect diseases based on B-SCAN images. Some of these studies are based on the more classic one-stage model and two-stage model. In addition to 2D target detection, some researchers also focus on 3D detection. They mine the three-dimensional feature information between multiple B-SCAN images and combine other latitude data of GPR, such as A-SCAN images, to detect underground targets. These algorithms have excellent performance and also show the effectiveness and trend of using deep learning to interpret GPR data.
[0008] Therefore, deep learning technology is gradually being widely used in the automatic identification of underground diseases.
[0009] Using deep learning models to detect underground runway defects can significantly improve detection efficiency, and there are many studies on defect detection based on GPR data. However, these studies have some shortcomings: (1) Most of these algorithms are supervised model training based on labeled data, and cannot utilize most of the unlabeled data in actual scenarios. (2) They cannot solve the problem of extremely low transfer generalization ability caused by the differences in the distribution of GPR data samples collected from different airports.
[0010] At present, in order to solve the performance degradation problem caused by the difference in data distribution, most of the existing technical solutions are to expand the size of the training data set. The purpose is to use a large number of labeled data samples and supervised training methods to allow the model to learn more disease characteristics, thereby improving the detection performance on unknown data sets.
[0011] However, it is very difficult to collect a large amount of underground data for airport runways, and as mentioned earlier, manual interpretation of GPR data is challenging and inefficient, and labeling datasets is time-consuming and costly. Therefore, it is costly and difficult to solve the problem of sample distribution differences by expanding the size of the training dataset. Summary of the invention
[0012] In order to solve the above problems, the purpose of the present invention is to provide an adaptive automatic detection method for underground structure defects of airport runways.
[0013] In order to achieve the above object, the adaptive airport runway underground structure disease automatic detection method provided by the present invention comprises the following steps performed in sequence:
[0014] Step 1: Use a robot equipped with a multi-channel ground penetrating radar to obtain B-scan images of the underground of different airport runways, and then divide them into source domain datasets and target domain datasets;
[0015] Step 2: Build a runway underground structure disease detection model based on YOLOv5, which includes a two-stream backbone network with shared weights for feature extraction and inter-domain distance calculation, a self-attention module for obtaining global information, an FPN neck network for multi-scale feature fusion, and three detection heads for different feature sizes;
[0016] Step 3: Input the source domain dataset and the target domain dataset obtained in step 1 into the runway underground structure disease detection model constructed in step 2, and use the dual-stream backbone network to extract the features p1 to p5 of the B-scan images in the source domain dataset and the target domain dataset respectively;
[0017] Step 4: Input the feature p5 obtained in step 3 into the self-attention module to obtain a new feature p*5;
[0018] Step 5: Input the features p3 and p4 obtained in step 3 and the new feature p*5 obtained in step 4 into the FPN neck network for multi-scale feature fusion, and then input the fused features of three different scales into the corresponding three detection heads to locate and classify underground diseases;
[0019] Step 6: Use the unsupervised domain adaptation technique (MMD) to calculate the inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3;
[0020] Step 7: Calculate the classification loss and regression loss of the positioning and classification results obtained in step 5 and the actual disease target label, and then calculate the total loss function of the runway underground structure disease detection model in combination with the inter-domain distance obtained in step 6. Then optimize the model through back propagation, and iterate the optimization until the total loss value converges, so as to determine the final weight of the runway underground structure disease detection model.
[0021] Step 8: After the preprocessing described in step 1, the B-scan image of the airport runway to be detected is input into the runway underground structure disease detection model whose final weight has been determined in the above step 7. The runway underground structure disease detection model outputs the final positioning and classification results of the disease target.
[0022] In step 1, the method of obtaining B-scan images of underground runways of different airports by using a robot equipped with a multi-channel ground penetrating radar and then dividing the images into a source domain data set and a target domain data set is as follows:
[0023] A robot equipped with a multi-channel ground penetrating radar was used to collect multiple original B-scan images of the underground of real airport runways according to the pre-planned routes on the airport runways. The original B-scan images were then preprocessed in sequence, including zero bias correction, zero point adjustment, background elimination, data filtering, gain adjustment and image visualization, to obtain B-scan images. All B-scan images were then divided into a source domain dataset and a target domain dataset. Both the source domain dataset and the target domain dataset included four typical underground diseases or objects of airport runways, namely: voids, cracks, steel bars and settlement.
[0024] In step 2, the backbone network includes a Focus module, an L-Dark2 module, an L-Dark3 module, an L-Dark4 module and an L-Dark5 module; wherein the L-Dark module includes a C3 module in YOLOv5 and a deep row attention module, which is used to obtain rich layer information and establish a correlation between underground diseases of the runway and the underground structure of the runway; the deep row attention module includes a learnable position encoding module and a row attention module.
[0025] In step 3, the source domain dataset and the target domain dataset obtained in step 1 are input into the runway underground structure disease detection model constructed in step 2, and the method of using the dual-stream backbone network to extract features p1 to p5 of the B-scan images in the source domain dataset and the target domain dataset respectively is:
[0026] First, the B-scan image sizes in the source domain dataset and the target domain dataset are unified into 448*448 pixels, and then respectively input into the respective backbone networks in the runway underground structure disease detection model. After that, they pass through the Focus module, L-Dark2 module, L-Dark3 module, L-Dark4 module and L-Dark5 module in sequence to obtain features with lengths and widths of 224*224 pixels, 112*112 pixels, 56*56 pixels, 28*28 pixels and 14*14 pixels, which are recorded as feature p1, feature p2, feature p3, feature p4 and feature p5.
[0027] The processing method of the learnable position encoding module in the L-Dark module is as follows: embed the one-dimensional position encoding into the algorithm model to represent only the depth position information; specifically, take the row feature vector in the feature as the unit, and use the learnable position encoding method to integrate the absolute depth position information into the extracted features; let the feature size be c*h*w, and the size of the learned depth position information be c*h*1; then, copy and expand the depth position information to a size that matches the feature size c*h*w; finally, use a simple addition operation to directly add it to the feature, thereby completing the fusion of the depth position information and the feature;
[0028] The processing method of the row attention module is as follows: when calculating, keep each row independent while mixing the information within the row, that is, take each row of features as a calculation unit, and then apply self-attention for calculation;
[0029] The calculation formula of self-attention is as follows:
[0030]
[0031] Where Q, K and V represent the query vector, key vector and value vector respectively, d k Represents the dimension of the key vector; the softmax function is used to normalize the correlation scores between different positions, and the final output is the vector weighted sum of the score values based on attention.
[0032] In step 5, the feature p3 and feature p4 obtained in step 3 and the new feature p*5 obtained in step 4 are input into the FPN neck network for multi-scale feature fusion, and then the fusion features of three different scales are input into the corresponding three detection heads. The method for locating and classifying underground diseases is:
[0033] The features p3 and p4 obtained in step 3 and the new feature p*5 obtained in step 4 are input into the FPN neck network. After calculation by the FPN neck network, the fused features P3 to P5 with lengths and widths of 56*56 pixels, 28*28 pixels and 14*14 pixels are obtained. Finally, the fused features P3 to P5 are sent to three detection heads for the detection of targets of large, medium and small sizes, respectively, thereby completing the positioning and classification of diseased targets.
[0034] In step 6, the method of calculating the inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3 by using the unsupervised domain adaptation technology is:
[0035] The inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3 is calculated using the following formula:
[0036]
[0037] Among them, X S represents the source domain data, X T represents the target domain data, x s and x t denote samples from the source domain and the target domain, respectively, and Φ(·) is a mapping function, usually a kernel function.
[0038] In step 7, the positioning and classification results obtained in step 5 are subjected to classification loss and regression loss calculation with the real disease target label, and then the total loss function of the runway underground structure disease detection model is calculated in combination with the inter-domain distance obtained in step 6. The model is then optimized by back propagation, and the method of determining the final weight of the runway underground structure disease detection model by continuously iterating the optimization until the total loss value converges is as follows:
[0039] The positioning and classification results obtained in step 5 are compared with the actual disease target label to calculate the classification loss and regression loss, and the classification loss values L are obtained respectively. cls And the regression loss value L box , and then combined with the inter-domain distance MMD obtained in step 6, the total loss value of the runway underground structure disease detection model is calculated using the following formula:
[0040] L=L cls +L box +λMMD(X S ,X T ) (3)
[0041] Among them, the hyperparameter λ represents the importance that the model assigns to the difference between the two domains. According to experimental verification, the best effect is achieved when the hyperparameter λ is set to 0.05;
[0042] After obtaining the total loss value, the model is optimized through back propagation. After continuous iterative optimization until the total loss value converges, the final weight of the runway underground structure disease detection model is determined.
[0043] The adaptive airport runway underground structure disease automatic detection method provided by the present invention has the following beneficial effects:
[0044] 1. Through the unsupervised domain adaptation technology MMD, the model's requirements for GPR data annotation can be reduced, reducing the time and financial costs in this regard. The model can be trained using unlabeled data sets, thereby solving the problem of low model generalization ability caused by sample distribution differences and better generalizing on different data sets.
[0045] 2. Mining domain knowledge in two areas of airport runway underground disease detection: Through deep row attention and self-attention mechanisms, the two domain knowledges of the correlation between diseases and runway underground structures and the size differences between different diseases are integrated into the deep learning algorithm model. The model's perception of runway layer information and the surrounding environment is established. On the one hand, more effective features can be extracted to improve the recognition accuracy of airport runway underground disease targets, allowing the system to more accurately and effectively identify target types. On the other hand, the extracted feature information is more common and can be adaptively applied to different airport data sets. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a schematic diagram of the structure of the runway underground structure disease detection model based on YOLOv5 in the present invention.
[0047] Figure 2 Schematic diagram of the structure of the deep row attention module in the present invention.
[0048] Figure 3 This is a visual comparison chart of the detection results of different algorithms in the present invention. DETAILED DESCRIPTION
[0049] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0050] The adaptive airport runway underground structure disease automatic detection method provided by the present invention comprises the following steps performed in sequence:
[0051] Step 1: Use a robot equipped with a multi-channel ground penetrating radar to obtain B-scan images of the underground of different airport runways, and then divide them into source domain datasets and target domain datasets;
[0052] A robot equipped with a multi-channel ground penetrating radar is used to collect multiple original B-scan images of the underground of real airport runways along the pre-planned routes on the airport runways. The original B-scan images are then preprocessed in sequence, including zero bias correction, zero point adjustment, background elimination, data filtering, gain adjustment, and image visualization, to obtain B-scan images. All B-scan images are then divided into source domain data sets and target domain data sets.
[0053] In the present invention, a total of 7 original B-scan images of the underground of real airport runways are collected; the ground penetrating radar operating frequencies are 200MHz, 300MHz and 500MHz. A 14-channel ground penetrating radar with equal spacing is used, and the distance between each channel is 0.7m. Both the source domain dataset and the target domain dataset include 4 typical underground diseases or objects of airport runways, namely: voids, cracks, steel bars and settlement.
[0054] Step 2: Build Figure 1 The YOLOv5-based runway underground structure disease detection model shown in the figure includes a shared weight, a two-stream backbone network for feature extraction and inter-domain distance calculation, a self-attention module for obtaining global information, an FPN neck network for multi-scale feature fusion, and three detection heads for different feature sizes;
[0055] The backbone network includes a Focus module, an L-Dark2 module, an L-Dark3 module, an L-Dark4 module and an L-Dark5 module; wherein the L-Dark module includes a C3 module in YOLOv5 and a deep row attention module, which is used to obtain rich layer information and establish the correlation between underground runway diseases and underground runway structures; Figure 2 As shown, the deep row attention module includes a learnable position encoding module and a row attention module.
[0056] Step 3: Input the source domain dataset and the target domain dataset obtained in step 1 into the runway underground structure disease detection model constructed in step 2, and use the dual-stream backbone network to extract the features p1 to p5 of the B-scan images in the source domain dataset and the target domain dataset respectively;
[0057] First, the B-scan image sizes in the source domain dataset and the target domain dataset are unified into 448*448 pixels, and then respectively input into the respective backbone networks in the runway underground structure disease detection model. After that, they pass through the Focus module, L-Dark2 module, L-Dark3 module, L-Dark4 module and L-Dark5 module in sequence to obtain features with lengths and widths of 224*224 pixels, 112*112 pixels, 56*56 pixels, 28*28 pixels and 14*14 pixels, which are recorded as feature p1, feature p2, feature p3, feature p4 and feature p5.
[0058] The processing method of the learnable position encoding module in the L-Dark module is as follows: embed the one-dimensional position encoding into the algorithm model to represent only the depth position information. Specifically, the row feature vector in the feature is used as the unit, and the absolute depth position information is integrated into the extracted feature using the learnable position encoding method; let the feature size be c*h*w, and the size of the learned depth position information be c*h*1; then, copy and expand the depth position information to a size that matches the feature size c*h*w; finally, use a simple addition operation to directly add it to the feature, thereby completing the fusion of the depth position information and the feature;
[0059] The processing method of the row attention module is as follows: row attention is an axial attention mechanism, which is essentially a self-attention mechanism. When calculating, each row is kept independent while mixing the information within the row, that is, each row of features is used as a calculation unit, and then self-attention is applied for calculation;
[0060] The calculation formula of self-attention is as follows:
[0061]
[0062] Where Q, K and V represent the query vector, key vector and value vector respectively, d k Represents the dimension of the key vector; the softmax function is used to normalize the correlation scores between different positions, and the final output is the vector weighted sum of the score values based on attention.
[0063] Step 4: Input the feature p5 obtained in step 3 into the self-attention module to obtain a new feature p*5;
[0064] The deepest feature p5 extracted by the dual-stream backbone network is processed using the self-attention module to obtain a new feature p*5, so that the runway underground structure disease detection model can obtain global information and strengthen the correlation between the disease and the surrounding environment, thereby solving the problem of weak continuity of large diseases in B-scan images.
[0065] Step 5: Input the features p3 and p4 obtained in step 3 and the new feature p*5 obtained in step 4 into the FPN neck network for multi-scale feature fusion, and then input the fused features of three different scales into the corresponding three detection heads to locate and classify underground diseases;
[0066] The features p3 and p4 obtained in step 3 and the new feature p*5 obtained in step 4 are input into the FPN neck network. After calculation by the FPN neck network, the fused features P3 to P5 with lengths and widths of 56*56 pixels, 28*28 pixels and 14*14 pixels are obtained. Finally, the fused features P3 to P5 are sent to three detection heads for the detection of targets of large, medium and small sizes, respectively, thereby completing the positioning and classification of diseased targets.
[0067] Step 6: Use the unsupervised domain adaptation technique (MMD) to calculate the inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3;
[0068] MMD is widely used to measure the distance between two distribution data, which is used to find a mapping function that maps both the source domain data and the target domain data to a reproducing kernel Hilbert space.
[0069] The inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3 is calculated using the following formula:
[0070]
[0071] Among them, X S represents the source domain data, X T represents the target domain data, x s and x t denote samples from the source domain and the target domain, respectively, and Φ(·) is a mapping function, usually a kernel function.
[0072] Step 7: Calculate the classification loss and regression loss of the positioning and classification results obtained in step 5 and the actual disease target label, and then calculate the total loss function of the runway underground structure disease detection model in combination with the inter-domain distance obtained in step 6. Then optimize the model through back propagation, and iterate the optimization until the total loss value converges, so as to determine the final weight of the runway underground structure disease detection model.
[0073] The positioning and classification results obtained in step 5 are compared with the actual disease target label to calculate the classification loss and regression loss, and the classification loss values L are obtained respectively. cls And the regression loss value L box , and then combined with the inter-domain distance MMD obtained in step 6, the total loss value of the runway underground structure disease detection model is calculated using the following formula:
[0074] L=L cls +L box +λMMD(X S ,X T ) (3)
[0075] Among them, the hyperparameter λ represents the importance that the model assigns to the difference between the two domains. According to experimental verification, the best effect is achieved when the hyperparameter λ is set to 0.05;
[0076] After obtaining the total loss value, the model is optimized through back propagation. After continuous iterative optimization until the total loss value converges, the final weight of the runway underground structure disease detection model is determined.
[0077] Step 8: After the preprocessing described in step 1, the B-scan image of the airport runway to be detected is input into the runway underground structure disease detection model whose final weight has been determined in the above step 7. The runway underground structure disease detection model outputs the final positioning and classification results of the disease target.
[0078] In order to verify the effect of the present invention, the inventors conducted the following experiments:
[0079] (1) Training parameter selection of runway underground structure disease detection model: The initial learning rate is set to 0.01 (it will change during training, but the minimum is not less than 0.0001). The momentum is set to 0.937 to optimize the gradient descent method. The batch size of the source domain and the target domain is set to 8. During training, the batch data of the two different domains pass through a two-stream backbone network with shared weights.
[0080] (2) Description of evaluation index: The present invention uses five evaluation indexes widely used in the field of object detection: precision, recall, F1-measure (F1), AP and mAP (mean average precision). The formulas are as follows:
[0081]
[0082] The present invention sets the intersection-over-union (IoU) ratio to 0.5. When the overlap ratio of the predicted box and the labeled box is greater than the IoU ratio, it is counted as a true positive (TP), otherwise it is a false positive (FP). Adding the false negative (FN), the specific values of the five evaluation indicators can be calculated.
[0083] (3) Description of comparison method: The performance of the method of the present invention will be compared with the following four algorithms.
[0084] ① The classic anchor-free algorithm Centernet. Reference: Duan K, Bai S, Xie L, et al. Centernet: Keypoint triplets for object detection [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2019: 6569-6578.
[0085] ②Swin transformer based on hierarchical design and shifted windows. References: Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2021: 10012-10022.
[0086] ③ The two latest models of the most popular one-stage YOLO series algorithm: YOLOv8 and YOLOv9. References: Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralytics YOLO, January 2023. and Wang CY, Yeh IH, Liao HY M. YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information[J].arXiv preprint arXiv:2402.13616,2024.
[0087] Table 1. Performance comparison of different algorithms
[0088]
[0089] From the data in Table 1, it can be concluded that the F1 of the proposed method for four underground diseases or objects, namely cracks, voids, settlements and steel bars, reached 24%, 30%, 38% and 72% respectively, and the mAP reached 38%, all of which surpassed the other four mainstream and latest algorithms. The visualization results of the detection results of each model are as follows Figure 3 As shown in the figure, the application of the method on actual real airport data proves that the method of the present invention can effectively detect the four diseases or objects.
Claims
1. An adaptive automatic detection method for underground structure defects of airport runways, characterized by: The method for automatically detecting defects in underground structures of an adaptive airport runway comprises the following steps performed in sequence: Step 1: Use a robot equipped with a multi-channel ground penetrating radar to obtain B-scan images of the underground of different airport runways, and then divide them into source domain datasets and target domain datasets; Step 2: Build a runway underground structure disease detection model based on YOLOv5, which includes a two-stream backbone network with shared weights for feature extraction and inter-domain distance calculation, a self-attention module for obtaining global information, an FPN neck network for multi-scale feature fusion, and three detection heads for different feature sizes; Step 3: Input the source domain dataset and the target domain dataset obtained in step 1 into the runway underground structure disease detection model constructed in step 2, and use the dual-stream backbone network to extract the features p1 to p5 of the B-scan images in the source domain dataset and the target domain dataset respectively; Step 4: Input the feature p5 obtained in step 3 into the self-attention module to obtain a new feature p*5; Step 5: Input the features p3 and p4 obtained in step 3 and the new feature p*5 obtained in step 4 into the FPN neck network for multi-scale feature fusion, and then input the fused features of three different scales into the corresponding three detection heads to locate and classify underground diseases; Step 6: Use unsupervised domain adaptation technology to calculate the inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3; Step 7: Calculate the classification loss and regression loss of the positioning and classification results obtained in step 5 and the actual disease target label, and then calculate the total loss function of the runway underground structure disease detection model in combination with the inter-domain distance obtained in step 6. Then optimize the model through back propagation, and iterate the optimization until the total loss value converges, so as to determine the final weight of the runway underground structure disease detection model. Step 8: After the preprocessing described in step 1, the B-scan image of the airport runway to be detected is input into the runway underground structure disease detection model whose final weight has been determined in the above step 7. The runway underground structure disease detection model outputs the final positioning and classification results of the disease target.
2. The adaptive automatic detection method for underground structure defects of airport runways according to claim 1 is characterized by: In step 1, the method of obtaining B-scan images of underground runways of different airports by using a robot equipped with a multi-channel ground penetrating radar and then dividing the images into a source domain data set and a target domain data set is as follows: A robot equipped with a multi-channel ground penetrating radar was used to collect multiple original B-scan images of the underground of real airport runways according to the pre-planned routes on the airport runways. The original B-scan images were then preprocessed in sequence, including zero bias correction, zero point adjustment, background elimination, data filtering, gain adjustment and image visualization, to obtain B-scan images. All B-scan images were then divided into a source domain dataset and a target domain dataset. Both the source domain dataset and the target domain dataset included four typical underground diseases or objects of airport runways, namely: voids, cracks, steel bars and settlement.
3. The adaptive airport runway underground structure disease automatic detection method according to claim 1 is characterized by: In step 2, the backbone network includes a Focus module, an L-Dark2 module, an L-Dark3 module, an L-Dark4 module and an L-Dark5 module; wherein the L-Dark module includes a C3 module in YOLOv5 and a deep row attention module, which is used to obtain rich layer information and establish a correlation between underground diseases of the runway and the underground structure of the runway; the deep row attention module includes a learnable position encoding module and a row attention module.
4. The adaptive airport runway underground structure disease automatic detection method according to claim 1 is characterized by: In step 3, the source domain dataset and the target domain dataset obtained in step 1 are input into the runway underground structure disease detection model constructed in step 2, and the method of using the dual-stream backbone network to extract features p1 to p5 of the B-scan images in the source domain dataset and the target domain dataset respectively is: First, the B-scan image sizes in the source domain dataset and the target domain dataset are unified into 448*448 pixels, and then respectively input into the respective backbone networks in the runway underground structure disease detection model. After that, they pass through the Focus module, L-Dark2 module, L-Dark3 module, L-Dark4 module and L-Dark5 module in sequence to obtain features with lengths and widths of 224*224 pixels, 112*112 pixels, 56*56 pixels, 28*28 pixels and 14*14 pixels, which are recorded as feature p1, feature p2, feature p3, feature p4 and feature p5. The processing method of the learnable position encoding module in the L-Dark module is as follows: embed the one-dimensional position encoding into the algorithm model to represent only the depth position information; specifically, take the row feature vector in the feature as the unit, and use the learnable position encoding method to integrate the absolute depth position information into the extracted features; let the feature size be c*h*w, and the size of the learned depth position information be c*h*1; then, copy and expand the depth position information to a size that matches the feature size c*h*w; finally, use a simple addition operation to directly add it to the feature, thereby completing the fusion of the depth position information and the feature; The processing method of the row attention module is as follows: when calculating, keep each row independent while mixing the information within the row, that is, take each row of features as a calculation unit, and then apply self-attention for calculation; The calculation formula of self-attention is as follows: Where Q, K and V represent the query vector, key vector and value vector respectively, d k Represents the dimension of the key vector; the softmax function is used to normalize the correlation scores between different positions, and the final output is the vector weighted sum of the score values based on attention.
5. The adaptive airport runway underground structure disease automatic detection method according to claim 1 is characterized by: In step 5, the feature p3 and feature p4 obtained in step 3 and the new feature p*5 obtained in step 4 are input into the FPN neck network for multi-scale feature fusion, and then the fusion features of three different scales are input into the corresponding three detection heads. The method for locating and classifying underground diseases is: The features p3 and p4 obtained in step 3 and the new feature p*5 obtained in step 4 are input into the FPN neck network. After calculation by the FPN neck network, the fused features P3 to P5 with lengths and widths of 56*56 pixels, 28*28 pixels and 14*14 pixels are obtained. Finally, the fused features P3 to P5 are sent to three detection heads for the detection of targets of large, medium and small sizes, respectively, thereby completing the positioning and classification of diseased targets.
6. The adaptive airport runway underground structure disease automatic detection method according to claim 1 is characterized by: In step 6, the method of calculating the inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3 by using the unsupervised domain adaptation technology is: The inter-domain distance between the two features p3 extracted from the source domain dataset and the target domain dataset in step 3 is calculated using the following formula: Among them, X S represents the source domain data, X T represents the target domain data, x s and t denote samples from the source domain and the target domain, respectively, and Φ(·) is a mapping function, usually a kernel function.
7. The adaptive airport runway underground structure disease automatic detection method according to claim 1 is characterized by: In step 7, the positioning and classification results obtained in step 5 are subjected to classification loss and regression loss calculation with the real disease target label, and then the total loss function of the runway underground structure disease detection model is calculated in combination with the inter-domain distance obtained in step 6. The model is then optimized by back propagation, and the method of determining the final weight of the runway underground structure disease detection model by continuously iterating the optimization until the total loss value converges is as follows: The positioning and classification results obtained in step 5 are compared with the actual disease target label to calculate the classification loss and regression loss, and the classification loss values L are obtained respectively. cls And the regression loss value L box , and then combined with the inter-domain distance MMD obtained in step 6, the total loss value of the runway underground structure disease detection model is calculated using the following formula: L=L cls +L box +λMMD(X S ,X T ) (3) Among them, the hyperparameter λ represents the importance that the model assigns to the difference between the two domains. According to experimental verification, the best effect is achieved when the hyperparameter λ is set to 0.05; After obtaining the total loss value, the model is optimized through back propagation. After continuous iterative optimization until the total loss value converges, the final weight of the runway underground structure disease detection model is determined.
Citation Information
Patent Citations
SF-SSD algorithm-based method for detecting underground diseases of airport runway
CN113298155A
Universal domain adaptive node classification method and device
CN117435970A
System for fast and accurate visual domain adaptation
US10839269B1
Transfer model training method and apparatus, and fault detection method and apparatus
US20230063148A1
Neural network distillation method and apparatus
US20230153615A1
Cited By
Abnormal body detection model and method constructed for airport management and maintenance large model knowledge base
CN122112829A
