Airport runway pavement real-time detection system and method based on machine vision

Through the real-time detection system of airport runway surfaces based on machine vision, combined with telephoto cameras and convolutional neural networks, the abnormalities of runway surfaces are identified and evaluated, and the problems of untimely and misjudgment of traditional detection methods are solved, achieving efficient and accurate Real-time monitoring and early warning ensure the safety and efficiency of airport operations.

CN118918550APending Publication Date: 2024-11-08CIVIL AVIATION UNIV OF CHINA
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202410938917.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Traditional airport runway surface detection methods rely on manual inspection or intermittent equipment, resulting in untimely inspection, omission or misjudgment, and lack of real-time monitoring and early warning capabilities around the clock, so that the runway surface defects cannot be discovered in time.

Method used

Adopt a real-time detection system for airport runways based on machine vision, including a track image acquisition module, a machine vision model module, an edge detection module, anomaly identification module and defect evaluation and early warning module. Panoramic images are collected through telephoto cameras and convolutional nerves are used. The network performs preprocessing and training, combines gradient calculation and dual threshold mechanism to identify abnormalities, automatically sends patrol cars to shoot and evaluate defects, and generates an early warning mechanism.

Benefits of technology

Achieve comprehensive real-time monitoring and analysis of airport runway surfaces, improve the accuracy and efficiency of detection, reduce missed inspections and misjudgments, ensure the safety and stability of operations, reduce maintenance costs and manpower investment, and improve The airport service level.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918550A_ABST
    Figure CN118918550A_ABST
Patent Text Reader

Abstract

The invention discloses an airport runway pavement real-time detection system and method based on machine vision, and relates to the technical field of visual detection.The system can comprehensively monitor and analyze an airport runway pavement through organic combination of five modules, firstly, a long-focus camera is used for collecting an airport runway pavement panoramic image in real time, and the panoramic image of the airport runway pavement is obtained; therefore, high quality and comprehensiveness of the data are ensured, and then the machine vision model carries out preprocessing and training on the image, so that the accuracy and generalization ability of detection are improved. Afterwards, the edge detection module effectively recognizes and locates the abnormality of the runway surface by calculating the gradient magnitude Tf of the image and applying a double-threshold mechanism, and achieves the precise detection of the abnormal position. And finally, an abnormity identification module and a defect evaluation and early warning module are combined to realize further analysis and defect degree evaluation of an abnormal position image, so that a corresponding early warning mechanism is generated, and the safety and the stability of the operation of the airport runway pavement are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of visual detection technology, and specifically to a real-time detection system and method for airport runway pavement based on machine vision. Background Art

[0002] Airport runway pavement inspection usually refers to the regular inspection and evaluation of the airport runway pavement and related facilities to ensure its safety and reliability. This work is usually undertaken by the airport management department or a dedicated operations team. The inspection content includes the flatness of the runway pavement surface, the effectiveness of the drainage system, the operating status of signs and lights, etc. These inspections can be carried out through manual inspections, the use of field inspection vehicles, ground radar or visual sensor technology.

[0003] At present, traditional airport runway pavement inspection methods have some shortcomings. Traditional methods usually rely on manual inspections or intermittent inspection equipment. This method is not only time-consuming and labor-intensive, but may also result in untimely inspections, omissions or misjudgments, leading to immeasurable losses. In addition, traditional methods are often not easy to achieve all-weather real-time monitoring of runway pavements, and lack the ability to promptly detect and warn of runway pavement defects. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a system and method for real-time detection of airport runway pavement based on machine vision, which solves the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: including a road surface image acquisition module, a machine vision model module, an edge detection module, an anomaly recognition module and a defect assessment and warning module;

[0006] The pavement image acquisition module is used to install a telephoto camera inside the airport to acquire a panoramic image of the airport runway pavement and obtain a panoramic image data set of the airport runway pavement;

[0007] The machine vision model module is used to construct a machine vision model, preprocess the collected panoramic images of the airport runway pavement, and perform training and testing based on the preprocessed panoramic images of the airport runway pavement;

[0008] The edge detection module is used to detect and segment the target of the panoramic image of the airport runway pavement using computer vision technology, and calculate the horizontal gradient value Sp and the vertical gradient value Cz through the edge detection algorithm, and then perform correlation calculation to obtain the gradient amplitude Tf of each pixel point, and then analyze it through the dual threshold mechanism to accurately identify and locate the runway pavement anomaly from the image;

[0009] The anomaly recognition module is used to automatically send out a field patrol vehicle to take a separate photo of the abnormal road surface of the runway after locating the abnormality of the runway pavement, and then send the separately taken photo to the machine vision model, and use the machine vision model to analyze the runway pavement defect characteristics of the extracted abnormal position image, and calculate and obtain the defect characteristic coefficient Qx;

[0010] The defect assessment and early warning module is used to compare and evaluate the preset defect threshold Q with the acquired defect characteristic coefficient Qx, analyze the defect degree of the abnormal position image, and generate an early warning mechanism according to the defect degree.

[0011] Preferably, the pavement image acquisition module provides a fixed viewing angle inside the airport, and a telephoto camera with a focal length greater than 400 mm is arranged inside the airport, and a gimbal stabilization structure is installed at the bottom of the telephoto camera to perform real-time panoramic shooting of the airport runway pavement, thereby obtaining a panoramic image data set of the airport runway pavement.

[0012] Preferably, the machine vision model module includes a machine vision model unit and an image preprocessing unit;

[0013] The machine vision model unit is used to select a convolutional neural network (CNN) as a machine vision model framework, collect a data set of panoramic images of airport runway pavement from the Internet, and these images include images of runway pavement under various weather conditions and defects of airport runway pavement, including cracks, potholes and wear. After collection, the data set of panoramic images of airport runway pavement is preprocessed, including image size standardization, noise removal and image enhancement, and defective areas in the image are annotated. The marked panoramic image data set of airport runway pavement is divided into a training set and a test set, and the selected machine vision model is trained through the training set. During the training process, the selected loss function, optimization algorithm and learning rate adjustment strategy are used to optimize the accuracy and generalization ability of the machine vision model, and the accuracy, recall rate and F1 score indicators are used to evaluate the effect of the machine vision model through the test set.

[0014] Preferably, the image preprocessing unit is used to remove noise in the airport runway pavement panoramic image data set collected in real time through a Gaussian filter and a median filter, and convert the color image in the airport runway pavement panoramic image data set into a grayscale image, and then enhance the grayscale image through histogram equalization and contrast enhancement methods to enhance the edge features in the image, and at the same time divide the airport runway pavement panoramic image into different areas through an image segmentation algorithm, and locate the images of different areas and the actual positions.

[0015] Preferably, the edge detection module includes an image gradient calculation unit and a dual threshold detection unit;

[0016] The gradient calculation unit includes a directional gradient calculation unit and a gradient amplitude calculation unit;

[0017] The image gradient calculation unit is used to perform a convolution operation on each grayscale image after segmentation using a gradient operator Sobel to calculate a horizontal gradient value Sp and a vertical gradient value Cz of the runway pavement panoramic segmentation image;

[0018] The horizontal gradient value Sp and the vertical gradient value Cz are obtained by the following formula:

[0019]

[0020] Preferably, the gradient amplitude calculation unit calculates the gradient amplitude Tf of each pixel in the image by performing a convolution operation on the image with two convolution kernels in the horizontal direction gradient value Sp and the vertical direction gradient value Cz.

[0021] The gradient amplitude Tf is obtained by the following formula:

[0022]

[0023] Preferably, the dual threshold detection unit is used to set a first threshold F1 and a second threshold F2, the first threshold F1 is usually used to determine a strong edge, and the second threshold F2 is used to determine a weak edge, and then compared with the obtained gradient amplitude Tf for evaluation and analysis to classify the pixels in the image. The specific classification scheme is as follows;

[0024] When the gradient amplitude Tf> the first threshold F1, the pixel point in the segmented image is marked as a strong edge. At this time, the current segmented image is actually positioned, and further abnormality recognition analysis is performed;

[0025] When the second threshold F2≤gradient amplitude Tf≤first threshold F1, the pixel point in the segmented image is marked as a weak edge, and the current segmented image is actually positioned, and further abnormality recognition analysis is performed;

[0026] When the gradient amplitude Tf is less than the second threshold F2, the pixel point in the segmented image is marked as background, and no operation is required at this time.

[0027] Preferably, the anomaly identification module includes a pavement defect feature extraction unit and an anomaly feature calculation unit;

[0028] The pavement defect feature extraction unit is used to be triggered after a strong edge and a weak edge are detected, and after being triggered, automatically sends positioning and take-off instructions to the field patrol vehicle, and the field patrol vehicle automatically flies to the airport runway pavement positioning position, collects images of the runway pavement in the positioning area, and sends the collected image data to the machine vision recognition model, analyzes the runway pavement defect features of the image, and summarizes and obtains a runway pavement defect feature set in the image;

[0029] The runway pavement defect feature set includes the number of cracks Lcd, the crack area Lmj, the pothole depth Ksd and the pothole area Kmj;

[0030] The abnormal feature calculation unit is used to perform dimensionless processing on the runway pavement defect feature set, and then summarize and calculate to obtain the defect feature coefficient Qx, wherein the defect feature coefficient Qx is obtained by the following formula:

[0031]

[0032] Preferably, the defect assessment and early warning module is used to preset a defect threshold Q, and then compare and evaluate the abnormality with the acquired defect characteristic coefficient Qx to generate a corresponding early warning strategy. The specific assessment scheme and early warning strategy are as follows;

[0033] When the defect characteristic coefficient Qx ≥ defect threshold Q, it means that the current airport runway pavement is seriously damaged. At this time, a "level one warning" is generated, prohibiting aircraft from taking off and landing, and notifying relevant staff to repair the runway pavement immediately;

[0034] When the defect characteristic coefficient Qx is less than the defect threshold Q, it means that the current airport runway pavement is slightly damaged. At this time, a "secondary warning" is generated to notify the staff to repair the airport runway pavement during the period when there are no aircraft takeoffs and landings.

[0035] A real-time detection method for airport runway pavement based on machine vision comprises the following steps:

[0036] S1. First, install a telephoto camera inside the airport, ensure that the viewing angle is fixed and the focal length is greater than 400mm, install a pan-tilt stabilization structure at the bottom, and deploy a pan-tilt stabilization structure to take real-time panoramic photos of the airport runway surface;

[0037] S2. Select the convolutional neural network (CNN) as the machine vision model framework, collect runway pavement images under various weather conditions and airport runway pavement panoramic image datasets with pavement defects, preprocess the runway pavement panoramic image dataset, divide the dataset into a training set and a test set, use the training set to train the machine vision model, and evaluate the model effect through the test set;

[0038] S3, using the gradient operator Sobel to calculate the horizontal gradient value Sp and the vertical gradient value Cz, and calculating the gradient amplitude Tf of each pixel point according to the gradient value, setting a double threshold mechanism, classifying the pixels into strong edges, weak edges and background, and realizing accurate positioning of abnormal positions in the image;

[0039] S4. When the abnormal position is located, the field patrol vehicle is automatically triggered to take pictures of the abnormal area and send the pictures to the machine vision model. The machine vision model analyzes the abnormal position image, extracts defect features, and calculates the defect feature coefficient Qx;

[0040] S5. Compare and evaluate the preset defect threshold Q with the calculated defect characteristic coefficient Qx to generate a corresponding evaluation plan and early warning strategy.

[0041] Beneficial Effects

[0042] The present invention provides a real-time detection system and method for airport runway pavement based on machine vision. It has the following beneficial effects:

[0043] (1) The key modules of the real-time detection system of airport runway pavement based on machine vision include pavement image acquisition, machine vision model construction, edge detection, anomaly recognition and defect assessment and warning. Through the organic combination of these modules, comprehensive monitoring and analysis of airport runway pavement can be achieved. First, the panoramic image of the airport runway pavement is collected in real time using a telephoto camera, which ensures the high quality and comprehensiveness of the data. Then, the machine vision model preprocesses and trains the image to improve the accuracy and generalization ability of detection. Subsequently, the edge detection module effectively identifies and locates the runway pavement anomaly by calculating the image gradient amplitude Tf and applying the dual threshold mechanism, thus realizing the accurate detection of the anomaly position. Finally, the anomaly recognition module and the defect assessment and warning module are combined to realize further analysis of the abnormal position image and defect degree assessment, thereby generating a corresponding warning mechanism to ensure the safety and stability of airport runway pavement operation.

[0044] (2) The system realizes real-time shooting of panoramic images of airport runway pavement through the deployment of pavement image acquisition modules, avoiding the time-consuming and inconvenient manual inspections in traditional methods. The application of machine vision models further improves detection efficiency and accuracy, eliminating subjective misjudgments that may exist in traditional methods. The use of edge detection modules makes the positioning of abnormal positions more accurate, greatly reducing the possibility of missed detection and misjudgment, and helping to promptly discover and deal with problems with airport runway pavement. At the same time, the combined effect of abnormal identification and defect assessment and early warning modules makes the assessment and treatment of different defects more scientific and efficient, helping to prevent potential safety hazards in advance and ensure the smooth operation of airport operations.

[0045] (3) The operation process of the system and the functions of its various modules cooperate with each other to form a complete airport runway pavement real-time detection and early warning system. The establishment of this system not only improves the safety and efficiency of airport operations, but also effectively reduces maintenance costs and manpower investment. In addition, the degree of automation and real-time nature of the system make the identification and processing of abnormal locations faster and more accurate, which helps to improve the airport's service level and reputation. In summary, the application of this system will bring many benefits to airport management departments and provide strong support for the development and safety of the aviation industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a real-time detection system for airport runway pavement based on machine vision according to the present invention;

[0047] Figure 2 A schematic diagram of the steps of a real-time detection method for airport runway pavement based on machine vision according to the present invention; DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Example 1

[0050] See also Figure 1 The present invention provides a real-time detection system for airport runway pavement based on machine vision. To achieve the above purpose, the present invention is implemented through the following technical scheme: including a pavement image acquisition module, a machine vision model module, an edge detection module, an abnormality recognition module and a defect assessment and warning module;

[0051] The pavement image acquisition module is used to install a telephoto camera inside the airport to collect panoramic images of the airport runway pavement and obtain a panoramic image dataset of the airport runway pavement;

[0052] The machine vision model module is used to build a machine vision model, pre-process the collected panoramic images of the airport runway pavement, and perform training and testing based on the pre-processed panoramic images of the airport runway pavement;

[0053] The edge detection module is used to detect and segment the targets in the panoramic image of the airport runway pavement using computer vision technology, and calculate the horizontal gradient value Sp and the vertical gradient value Cz through the edge detection algorithm, and then perform correlation calculations to obtain the gradient amplitude Tf of each pixel point, and then analyze it through a dual threshold mechanism to accurately identify and locate the runway pavement anomalies in the image;

[0054] The abnormality recognition module is used to locate the abnormality of the airport runway pavement. At this time, the runway patrol vehicle is automatically dispatched to take a separate photo of the abnormal road surface of the located runway pavement, and then the separately taken photos are sent to the machine vision model. The machine vision model is used to analyze the runway pavement defect characteristics of the extracted abnormal position image, and the defect characteristic coefficient Qx is calculated;

[0055] The defect assessment and early warning module is used to compare and evaluate the preset defect threshold Q with the acquired defect characteristic coefficient Qx, analyze the defect degree of the abnormal position image, and generate an early warning mechanism based on the defect degree.

[0056] In this embodiment, the system realizes comprehensive monitoring and analysis of the airport runway pavement through the organic combination of five key modules. First, the pavement image acquisition module uses a telephoto camera installed inside the airport to realize the real-time acquisition of panoramic images of the airport runway pavement, ensuring the comprehensiveness and high quality of the data. Secondly, the machine vision model module preprocesses and trains the image, improves the accuracy and generalization ability of detection, and realizes the effective processing of runway pavement images under various weather conditions. The edge detection module accurately identifies and locates runway pavement anomalies by calculating gradient values ​​and applying a dual threshold mechanism, effectively reducing the possibility of missed detection and misjudgment. The anomaly recognition module realizes further analysis of the abnormal position image and defect feature extraction by automatically dispatching a field patrol car to take separate photos of the abnormal position, providing accurate data support for subsequent defect assessment. Finally, the defect assessment and early warning module compares and evaluates the preset defect threshold Q and the defect feature coefficient Qx, generates a corresponding early warning mechanism, prevents potential safety hazards in advance, and ensures the safety and stability of the airport runway pavement operation. Compared with current technical means, the system has significantly improved monitoring accuracy, real-time and automation, providing airport management departments with more efficient and reliable detection and early warning services, further improving the level and safety of airport operations.

[0057] Example 2

[0058] This embodiment is explained in Example 1, please refer to Figure 1Specifically: the pavement image acquisition module provides a fixed viewing angle inside the airport, and deploys a telephoto camera with a focal length greater than 400mm inside the airport. A gimbal stabilization structure is installed at the bottom of the telephoto camera to perform real-time panoramic shooting of the airport runway pavement and obtain a panoramic image data set of the airport runway pavement.

[0059] In this embodiment, firstly, by using the fixed viewing angle and telephoto camera provided inside the airport, a panoramic shot of the airport runway pavement is achieved, ensuring the comprehensiveness and coverage of the image. Secondly, the telephoto camera's focal length of more than 400mm and the gimbal stabilization structure installed at the bottom effectively improve the clarity and stability of the image, avoiding image blur or distortion caused by vibration or jitter, thereby improving the accuracy of subsequent image processing and analysis. In addition, real-time panoramic shooting ensures timely monitoring and feedback of the airport runway pavement status, providing timely data support for abnormal identification and early warning.

[0060] Example 3

[0061] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the machine vision model module includes a machine vision model unit and an ,image preprocessing unit;

[0062] The machine vision model unit is used to select convolutional neural network (CNN) as the machine vision model framework, and collect airport runway pavement panoramic image datasets from the Internet. These images contain runway pavement images under various weather conditions and airport runway pavement defects, including cracks, potholes, and wear. After collection, the airport runway pavement panoramic image dataset is preprocessed, including image size standardization, noise removal, and image enhancement, and the defective areas in the image are annotated. The labeled airport runway pavement panoramic image dataset is divided into a training set and a test set. The selected machine vision model is trained with the training set. During the training process, the selected loss function, optimization algorithm, and learning rate adjustment strategy are used to optimize the accuracy and generalization ability of the machine vision model. At the same time, the accuracy, recall, and F1 score indicators are used to evaluate the effect of the machine vision model through the test set.

[0063] The image preprocessing unit is used to remove the noise in the panoramic image dataset of the airport runway pavement collected in real time through Gaussian filters and median filters, and convert the color images in the panoramic image dataset of the airport runway pavement into grayscale images. Then, the grayscale images are enhanced through histogram equalization and contrast enhancement methods to enhance the edge features in the images. At the same time, the panoramic image of the airport runway pavement is divided into different areas through an image segmentation algorithm, and the images of different areas are located with the actual positions.

[0064] In this embodiment, first, the convolutional neural network CNN is selected as the machine vision model framework, and the data set containing runway pavement images and defects under various weather conditions collected from the Internet can effectively extract image features and identify abnormal conditions of the runway pavement. Secondly, by preprocessing the image, the quality and clarity of the image are improved, which helps to improve the training effect and generalization ability of the machine vision model. In addition, marking the defective areas in the image helps to improve the accuracy and robustness of the model, so that it can more accurately identify the problems of the runway pavement. Finally, the model is trained with the training set and evaluated with the test set, which can timely discover and correct the shortcomings of the model, further improving the performance and reliability of the model. In summary, the design of the machine vision model module has achieved significant beneficial effects in model selection, data preprocessing, training evaluation, etc., and provides important support for the accuracy and reliability of the airport runway pavement real-time detection system. At the same time, the image preprocessing unit removes the noise in the image through a Gaussian filter and a median filter, which improves the clarity and quality of the image, which is conducive to subsequent image processing and analysis. Secondly, converting the color image into a grayscale image and performing histogram equalization and contrast enhancement can highlight the edge features of the image, which helps to more accurately identify defects and abnormalities in the runway surface. Finally, using the image segmentation algorithm to divide the image into different areas and locate the actual position helps to match the image analysis results with the actual situation, improving the accuracy and credibility of the system's runway surface status.

[0065] Example 4

[0066] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the edge detection module includes an image gradient calculation unit and a dual threshold detection unit;

[0067] The gradient calculation unit includes a direction gradient calculation unit and a gradient amplitude calculation unit;

[0068] The image gradient calculation unit is used to perform a convolution operation on each grayscale image after segmentation using the gradient operator Sobel, and calculate the horizontal gradient value Sp and the vertical gradient value Cz of the runway pavement panoramic segmentation image;

[0069] The horizontal gradient value Sp and the vertical gradient value Cz are obtained by the following formula:

[0070]

[0071] The gradient amplitude calculation unit calculates the gradient amplitude Tf of each pixel in the image by performing convolution operation on the image with two convolution kernels in the horizontal direction gradient value Sp and the vertical direction gradient value Cz;

[0072] The gradient amplitude Tf is obtained by the following formula;

[0073]

[0074] The dual threshold detection unit is used to set the first threshold F1 and the second threshold F2. The first threshold F1 is usually used to determine a strong edge, and the second threshold F2 is used to determine a weak edge. Then, the first threshold F1 is compared with the obtained gradient amplitude Tf for evaluation and analysis to classify the pixels in the image. The specific classification scheme is as follows;

[0075] When the gradient amplitude Tf> the first threshold F1, the pixel point in the segmented image is marked as a strong edge. At this time, the current segmented image is actually positioned, and further abnormality recognition analysis is performed;

[0076] When the second threshold F2≤gradient amplitude Tf≤first threshold F1, the pixel point in the segmented image is marked as a weak edge, and the current segmented image is actually positioned, and further abnormality recognition analysis is performed;

[0077] When the gradient amplitude Tf is less than the second threshold F2, the pixel point in the segmented image is marked as background, and no operation is required at this time.

[0078] In this embodiment, the system uses the gradient calculation unit to perform convolution operation on the grayscale image by using the gradient operator Sobel to calculate the horizontal gradient value Sp and the vertical gradient value Cz of each pixel point, thereby accurately capturing the edge information in the image. This precise gradient calculation can help the system accurately locate the abnormal situation of the runway pavement and provide important support for subsequent abnormality identification. Secondly, the dual threshold detection unit sets the first threshold F1 and the second threshold F2, and divides the pixels in the image into three categories: strong edge, weak edge and background, which helps to classify and analyze the abnormal situation of the runway pavement more finely. By setting appropriate thresholds, noise and irrelevant information in the image can be effectively filtered out, and the accuracy and reliability of the system's recognition of abnormal situations are improved. Finally, by classifying the pixels according to different gradient amplitudes Tf, the abnormal position can be determined more accurately, and further abnormality identification analysis can be performed, so as to timely discover and deal with problems on the runway pavement.

[0079] Example 5

[0080] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the anomaly recognition module includes a pavement defect feature extraction unit and an anomaly feature calculation unit;

[0081] The pavement defect feature extraction unit is used to be triggered after the strong edge and weak edge are detected, and then automatically sends positioning and take-off instructions to the field patrol vehicle. The field patrol vehicle automatically flies to the airport runway pavement positioning position, collects images of the runway pavement in the positioning area, and sends the collected image data to the machine vision recognition model to analyze the runway pavement defect features of the image, and summarizes and obtains the runway pavement defect feature set in the image;

[0082] The runway pavement defect feature set includes the number of cracks Lcd, the crack area Lmj, the pothole depth Ksd and the pothole area Kmj;

[0083] The abnormal feature calculation unit is used to perform dimensionless processing on the runway pavement defect feature set, and then summarize and calculate to obtain the defect feature coefficient Qx. The defect feature coefficient Qx is obtained by the following formula:

[0084]

[0085] The defect assessment and early warning module is used to preset the defect threshold Q, and then compare and evaluate the abnormality with the acquired defect characteristic coefficient Qx to generate the corresponding early warning strategy. The specific assessment scheme and early warning strategy are as follows;

[0086] When the defect characteristic coefficient Qx ≥ defect threshold Q, it means that the current airport runway pavement is seriously damaged. At this time, a "level one warning" is generated, prohibiting aircraft from taking off and landing, and notifying relevant staff to repair the runway pavement immediately;

[0087] When the defect characteristic coefficient Qx is less than the defect threshold Q, it means that the current airport runway pavement is slightly damaged. At this time, a "secondary warning" is generated to notify the staff to repair the airport runway pavement during the period when there are no aircraft takeoffs and landings.

[0088] In this embodiment, the system automatically sends positioning and take-off instructions to the runway patrol vehicle through the pavement defect feature extraction unit, and performs automated airport runway pavement positioning and image acquisition, thereby achieving efficient detection of the runway pavement defect position. After positioning, the runway patrol vehicle can comprehensively and comprehensively collect detailed images of the runway pavement at a relatively low altitude, providing sufficient data support for subsequent abnormality identification and evaluation. Secondly, the abnormal feature calculation unit can analyze the runway pavement defect features of the collected image data, extract key feature information, and calculate to obtain the defect feature coefficient Qx. Finally, the defect assessment and early warning module can compare and evaluate the preset defect threshold Q with the calculated defect feature coefficient Qx, and generate a corresponding early warning strategy. This timely abnormal assessment and early warning mechanism can effectively avoid potential safety hazards and ensure the normal operation of the airport runway pavement.

[0089] Example 6

[0090] See also Figure 1 and Figure 2 , a real-time detection method for airport runway pavement based on machine vision, comprising the following steps:

[0091] S1. First, install a telephoto camera inside the airport, ensure that the viewing angle is fixed and the focal length is greater than 400mm, install a pan-tilt stabilization structure at the bottom, and deploy a pan-tilt stabilization structure to take real-time panoramic photos of the airport runway surface;

[0092] S2. Select the convolutional neural network (CNN) as the machine vision model framework, collect runway pavement images under various weather conditions and airport runway pavement panoramic image datasets with pavement defects, preprocess the runway pavement panoramic image dataset, divide the dataset into a training set and a test set, use the training set to train the machine vision model, and evaluate the model effect through the test set;

[0093] S3, using the gradient operator Sobel to calculate the horizontal gradient value Sp and the vertical gradient value Cz, and calculating the gradient amplitude Tf of each pixel point according to the gradient value, setting a double threshold mechanism, classifying the pixels into strong edges, weak edges and background, and realizing accurate positioning of abnormal positions in the image;

[0094] S4. When the abnormal position is located, the field patrol vehicle is automatically triggered to take pictures of the abnormal area and send the pictures to the machine vision model. The machine vision model analyzes the abnormal position image, extracts defect features, and calculates the defect feature coefficient Qx;

[0095] S5. Compare and evaluate the preset defect threshold Q with the calculated defect characteristic coefficient Qx to generate a corresponding evaluation plan and early warning strategy.

[0096] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. A real-time detection system for airport runway pavement based on machine vision, Features: It includes pavement image acquisition module, machine vision model module, edge detection module, anomaly recognition module and defect assessment and early warning module; The pavement image acquisition module is used to deploy a telephoto camera inside the airport to acquire a panoramic image of the airport runway pavement and obtain a panoramic image data set of the airport runway pavement; The machine vision model module is used to construct a machine vision model, preprocess the collected panoramic images of the airport runway pavement, and perform training and testing based on the preprocessed panoramic images of the airport runway pavement; The edge detection module is used to detect and segment the target of the panoramic image of the airport runway pavement using computer vision technology, and calculate the horizontal gradient value Sp and the vertical gradient value Cz through the edge detection algorithm, and then perform correlation calculation to obtain the gradient amplitude Tf of each pixel point, and then analyze it through the dual threshold mechanism to accurately identify and locate the runway pavement anomaly from the image; The anomaly recognition module is used to automatically dispatch a field patrol vehicle equipped with a camera to take a separate photo of the abnormal road surface of the runway after locating the abnormality of the runway pavement at the airport, and then send the separately taken photo to the machine vision model, and use the machine vision model to analyze the runway pavement defect characteristics of the extracted abnormal position image, and calculate and obtain the defect characteristic coefficient Qx; The defect assessment and early warning module is used to compare and evaluate the preset defect threshold Q with the acquired defect characteristic coefficient Qx, analyze the defect degree of the abnormal position image, and generate an early warning mechanism according to the defect degree.

2. According to the machine vision-based real-time detection system for airport runway pavement of claim 1, Features: The pavement image acquisition module provides a fixed viewing angle inside the airport, and deploys a telephoto camera with a focal length greater than 400 mm inside the airport. A pan-tilt stabilization structure is installed at the bottom of the telephoto camera to perform real-time panoramic photography of the airport runway pavement and obtain a panoramic image dataset of the airport runway pavement.

3. According to the machine vision-based real-time detection system for airport runway pavement of claim 1, Features: The machine vision model module includes a machine vision model unit and an image preprocessing unit; The machine vision model unit is used to select a convolutional neural network (CNN) as a machine vision model framework, collect a data set of panoramic images of airport runway pavement from the Internet, and these images include images of runway pavement under various weather conditions and defects of airport runway pavement, including cracks, potholes and wear. After collection, the data set of panoramic images of airport runway pavement is preprocessed, including image size standardization, noise removal and image enhancement, and defective areas in the image are annotated. The marked panoramic image data set of airport runway pavement is divided into a training set and a test set, and the selected machine vision model is trained through the training set. During the training process, the selected loss function, optimization algorithm and learning rate adjustment strategy are used to optimize the accuracy and generalization ability of the machine vision model, and the accuracy, recall rate and F1 score indicators are used to evaluate the effect of the machine vision model through the test set.

4. According to claim 3, a real-time detection system for airport runway pavement based on machine vision, Features: The image preprocessing unit is used to remove noise in the airport runway pavement panoramic image data set collected in real time through a Gaussian filter and a median filter, and convert the color image in the airport runway pavement panoramic image data set into a grayscale image, and then enhance the grayscale image through a histogram equalization and contrast enhancement method to enhance the edge features in the image, and divide the airport runway pavement panoramic image into different areas through an image segmentation algorithm, and locate the images of different areas and the actual positions.

5. According to the machine vision-based real-time detection system for airport runway pavement of claim 1, Features: The edge detection module includes an image gradient calculation unit and a dual threshold detection unit; The gradient calculation unit includes a directional gradient calculation unit and a gradient amplitude calculation unit; The image gradient calculation unit is used to perform a convolution operation on each grayscale image after segmentation using a gradient operator Sobel to calculate a horizontal gradient value Sp and a vertical gradient value Cz of the runway pavement panoramic segmentation image; The horizontal gradient value Sp and the vertical gradient value Cz are obtained by the following formula:

6. The real-time detection system for airport runway pavement based on machine vision according to claim 5, Features: The gradient amplitude calculation unit calculates the gradient amplitude Tf of each pixel in the image by performing a convolution operation on the image with two convolution kernels in the horizontal direction gradient value Sp and the vertical direction gradient value Cz; The gradient amplitude Tf is obtained by the following formula:

7. The real-time detection system for airport runway pavement based on machine vision according to claim 5, Features: The dual threshold detection unit is used to set a first threshold F1 and a second threshold F2, and then compare and evaluate the obtained gradient amplitude Tf to classify the pixels in the image. The specific classification scheme is as follows: When the gradient amplitude Tf> the first threshold F1, the pixel point in the segmented image is marked as a strong edge. At this time, the current segmented image is actually positioned, and further abnormality recognition analysis is performed; When the second threshold F2≤gradient amplitude Tf≤first threshold F1, the pixel point in the segmented image is marked as a weak edge, and the current segmented image is actually positioned, and further abnormality recognition analysis is performed; When the gradient amplitude Tf is less than the second threshold F2, the pixel point in the segmented image is marked as background, and no operation is required at this time.

8. The real-time detection system for airport runway pavement based on machine vision according to claim 1, Features: The anomaly recognition module includes a pavement defect feature extraction unit and an anomaly feature calculation unit; The pavement defect feature extraction unit is used to be triggered after a strong edge and a weak edge are detected, and after being triggered, automatically sends a positioning and image acquisition instruction to the field patrol vehicle, and the field patrol vehicle automatically drives to the airport runway pavement positioning position, collects images of the runway pavement in the positioning area, and sends the collected image data to the machine vision recognition model, analyzes the runway pavement defect features of the image, and summarizes and obtains a runway pavement defect feature set in the image; The runway pavement defect feature set includes the number of cracks Lcd, the crack area Lmj, the pothole depth Ksd and the pothole area Kmj; The abnormal feature calculation unit is used to perform dimensionless processing on the runway pavement defect feature set, and then summarize and calculate to obtain the defect feature coefficient Qx, wherein the defect feature coefficient Qx is obtained by the following formula:

9. The real-time detection system for airport runway pavement based on machine vision according to claim 8, Features: The defect assessment and early warning module is used to preset the defect threshold Q, and then compare and evaluate the abnormality with the acquired defect characteristic coefficient Qx to generate a corresponding early warning strategy. The specific assessment scheme and early warning strategy are as follows; When the defect characteristic coefficient Qx ≥ defect threshold Q, it means that the current airport runway pavement is seriously damaged. At this time, a "level one warning" is generated, prohibiting aircraft from taking off and landing, and notifying relevant staff to repair the runway pavement immediately; When the defect characteristic coefficient Qx is less than the defect threshold Q, it means that the current airport runway pavement is slightly damaged. At this time, a "secondary warning" is generated to notify the staff to repair the airport runway pavement during the period when there are no aircraft takeoffs and landings.

10. A real-time detection method for airport runway pavement based on machine vision, comprising the real-time detection system for airport runway pavement based on machine vision as claimed in any one of claims 1 to 9, Features: The following steps are involved: S1. First, install a telephoto camera inside the airport, ensure that the viewing angle is fixed and the focal length is greater than 400mm, install a pan-tilt stabilization structure at the bottom, and deploy a pan-tilt stabilization structure to take real-time panoramic photos of the airport runway surface; S2. Select the convolutional neural network (CNN) as the machine vision model framework, collect runway pavement images under various weather conditions and airport runway pavement panoramic image datasets with pavement defects, preprocess the runway pavement panoramic image dataset, divide the dataset into a training set and a test set, use the training set to train the machine vision model, and evaluate the model effect through the test set; S3, using the gradient operator Sobel to calculate the horizontal gradient value Sp and the vertical gradient value Cz, and calculating the gradient amplitude Tf of each pixel point according to the gradient value, setting a double threshold mechanism, classifying the pixels into strong edges, weak edges and background, and realizing accurate positioning of abnormal positions in the image; S4. When the abnormal position is located, the field patrol vehicle is automatically triggered to take pictures of the abnormal area and send the pictures to the machine vision model. The machine vision model analyzes the abnormal position image, extracts defect features, and calculates the defect feature coefficient Qx; S5. Compare and evaluate the preset defect threshold Q with the calculated defect characteristic coefficient Qx to generate a corresponding evaluation plan and early warning strategy.

Citation Information

Cited By

  • Airfield pavement evaluation method and device based on defect detection

    CN121257991A

  • An airport pavement evaluation method and device based on defect detection

    CN121257991B

  • Airport runway abnormal target identification method and device and electronic equipment

    CN121527433A