A real-time detection method and system for abnormal state of airport pavement

By using the optimized YOLOv5 and DDRNet models in the airport pavement detection system, combined with classic image processing algorithms and image acceleration decoder, the problems of low efficiency and poor accuracy in the existing technology are solved, real-time and accurate detection of abnormal states of the airport pavement are realized, and the safe operation of the airport is ensured.

CN114240901BActive Publication Date: 2025-05-02CIVIL AVIATION CHENGDU ELECTRONIC TECH CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111564694.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-20
Publication Date
2025-05-02
Estimated Expiration
2041-12-20

AI Technical Summary

Technical Problem

The existing airport road surface detection technology has problems such as large workload, low efficiency, poor accuracy, easy to miss detection and strong subjectivity. In particular, the lidar is expensive and the detection accuracy is difficult to meet the requirements of small-sized cracks. Image processing technology also lacks real-time and environmental adaptability.

Method used

The optimized YOLOv5 object detection model and DDRNet semantic segmentation model are adopted, combined with classic image processing algorithms and image acceleration decoder, and are deployed on the movable patrol body to detect abnormal states of the airport road surface in real time, including apparent diseases of the road surface, foreign objects and abnormal wear of the marking line.

Benefits of technology

Real-time detection of abnormal status of airport road surfaces is realized, detection accuracy and efficiency are improved, road surface abnormalities can be detected in a timely manner, and the safe and intelligent operation of the airport is ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114240901B_ABST
    Figure CN114240901B_ABST
Patent Text Reader

Abstract

The present invention provides a method for real-time detection of abnormal conditions of airport pavement, comprising: optimizing a trained YOLOv5 target detection model and a DDRNet semantic segmentation model based on a preset inference optimizer; deploying the optimized YOLOv5 and DDRNet, a classic image processing algorithm and an image acceleration decoder on an image detection device of a movable body; controlling the movable inspection body to travel according to a preset driving path, and controlling the image acquisition device to acquire images of the airport pavement; the image acceleration decoder acquires the acquired airport pavement images in real time and decodes them; using YOLOv5, DDRNet and a classic image processing algorithm to detect abnormal conditions in the decoded images, the abnormal conditions including apparent pavement diseases, pavement foreign matter and abnormal wear of pavement marking lines. The present invention also provides a system for real-time detection of abnormal conditions of airport pavement. The present invention can detect abnormal conditions of airport pavements in real time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of airport pavement detection, and in particular to a real-time detection method and system for abnormal status of an airport pavement. Background Art

[0002] The airport pavement is an important facility of the airport. Under the influence of human factors, aircraft takeoff and landing, and various climatic environments, the appearance of incomplete markings, defects, and foreign objects on the airport pavement will affect the safe operation of the airport.

[0003] Currently, airports mainly use manual visual inspections to inspect airport pavements. The inspection process is that one inspector drives a car while another staff member observes the surface condition of the pavement. However, this traditional manual inspection method has disadvantages such as large workload, low efficiency, poor accuracy, easy to miss inspections, and strong subjectivity.

[0004] With the rapid growth of air traffic, automated inspection systems are used to replace manual inspections. Among them, radar detection technology and image processing technology are the current mainstream automatic detection technologies, but lidar is relatively expensive, and the detection effect on airport pavement markings is poor, and the detection accuracy is difficult to meet the detection requirements of small-sized cracks; in image processing, the accuracy of target detection algorithms and semantic segmentation algorithms is constantly improving, but the target detection algorithm cannot extract the contour information of the cracks, so it cannot judge the severity of the cracks; the semantic segmentation algorithm can extract the contour information of the cracks, but the real-time detection is poor. In terms of classical image processing, the image processing process is manually involved, but this processing method has poor environmental adaptability and cannot directly realize the abnormal state detection of airport pavements. Summary of the invention

[0005] In view of the above technical problems, the technical solution adopted by the present invention is:

[0006] An embodiment of the present invention provides a method for real-time detection of abnormal state of an airport pavement, comprising the following steps:

[0007] S100, respectively optimizing and building the YOLOv5 convolutional neural network model and the DDRNet convolutional neural network model and training them to obtain the trained YOLOv5 target detection model and DDRNet semantic segmentation model;

[0008] S200, based on the preset inference optimizer, optimizes the trained YOLOv5 object detection model and DDRNet semantic segmentation model to improve the inference speed of the model;

[0009] S300, deploying the optimized YOLOv5 target detection model and DDRNet semantic segmentation model, the classic image processing algorithm and the image acceleration decoder on the image detection device of the movable body; the movable inspection body is also provided with an image acquisition device;

[0010] S400, controlling the movable inspection body to travel along a preset travel path, and controlling the image acquisition device to acquire an image of the airport pavement;

[0011] S500, the image acceleration decoder acquires the collected airport pavement image in real time and decodes it;

[0012] S600 uses the optimized YOLOv5 target detection model, DDRNet semantic segmentation model and classic image processing algorithm to detect abnormal conditions in the decoded image and generate corresponding detection results. The abnormal conditions include pavement surface diseases, pavement foreign objects and abnormal wear of pavement markings.

[0013] Another embodiment of the present invention provides a real-time detection system for abnormal state of airport pavement, comprising a movable body and a control terminal connected in communication, wherein the movable body is provided with an image acquisition device and an image detection device;

[0014] The image detection device includes an image acceleration decoder, a classic image processing algorithm, a YOLOv5 target detection model optimized by a preset inference optimizer, and a DDRNet semantic segmentation model;

[0015] The movable inspection body is used to perform inspections along a preset driving path based on control instructions sent by the control terminal;

[0016] The image acquisition device is used to collect the airport pavement image during the inspection process of the movable inspection body and send it to the image detection device;

[0017] The image detection device is used for:

[0018] Acquire and decode the collected airport pavement images in real time through an image acceleration decoder; and

[0019] The abnormal conditions in the decoded image are detected through the optimized YOLOv5 target detection model, DDRNet semantic segmentation model and classic image processing algorithm, and the corresponding detection results are generated and sent to the control terminal. The abnormal conditions include pavement surface diseases, pavement foreign objects and abnormal wear of pavement markings.

[0020] The real-time detection method and system for abnormal conditions of airport pavements provided in the embodiments of the present invention use a thrust optimizer to optimize the YOLOv5 target detection model and the DDRNet semantic segmentation model, and use an image acceleration decoder to quickly decode the collected airport pavement images and are equipped with a classic image processing algorithm. Therefore, during the inspection process, detection results including surface diseases of the pavement, foreign objects on the pavement, and abnormal wear of pavement markings can be obtained in real time, and abnormal conditions of the airport pavement can be discovered in time, thereby ensuring the safe and intelligent operation of the airport. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic flow chart of a method for real-time detection of abnormal conditions of an airport pavement provided by an embodiment of the present invention;

[0023] Figure 2 This is a structural block diagram of a system for real-time detection of abnormal conditions of airport pavement provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] Figure 1 The following is a flow chart of a method for real-time detection of abnormal conditions of airport pavement provided by an embodiment of the present invention. Figure 1 As shown, the method for real-time detection of abnormal state of airport pavement provided by the embodiment of the present invention comprises the following steps:

[0026] S100, respectively optimize and build the YOLOv5 convolutional neural network model and the DDRNet convolutional neural network model and train them to obtain the trained YOLOv5 target detection model and DDRNet semantic segmentation model.

[0027] In an embodiment of the present invention, the training sample set can be obtained in the following manner: 40,000 images of normal targets such as people (such as airport employees), vehicles and airplanes on the airport pavement and 10,000 images of pavement surface defects such as cracks, broken seams, peeling of plate corners, patches, potholes and abnormal pavement conditions such as wear of marking lines are obtained from multiple airports and third-party institutions, and the obtained images are annotated and the data is enhanced using Mosaic and MixUp data enhancement technologies, obtaining a total of 500,000 images. 400,000 images are used as training sets and 200,000 images are used as test sets, which are placed on a training server equipped with 4 NVIDIA Tesla V100s (32GB video memory) for training and accuracy testing, and then a trained YOLOv5 target detection model and DDRNet semantic segmentation model are obtained, wherein YOLOv5 is responsible for detecting normal targets such as employees, vehicles and airplanes, marking lines and pavement surface defects such as broken seams, peeling of plate corners, patches, potholes; DDRNet is responsible for detecting pavement areas, pavement cracks and marking lines.

[0028] In this embodiment of the present invention, tensorflow2.5 is selected as the deep learning framework to build a YOLOv5 target detection model and a DDRNet semantic segmentation model.

[0029] In the embodiment of the present invention, the YOLOv5 target detection model uses the k-means clustering algorithm to perform cluster analysis on the target detection frame in the defect data set image, obtains the new number and size of anchors suitable for airport pavement defect detection, and modifies the number and size of anchors in the YOLOv5 network configuration file according to the parameters obtained by the k-means clustering algorithm; in order to improve the model's detection ability for small-sized defects such as cracks and potholes, the DDRNet semantic segmentation model uses the OhemCELoss loss function for training. The construction, training and testing of the YOLOv5 and DDRNet convolutional neural networks are all carried out on a training server equipped with 4 NVIDIA Tesla V100s (32GB video memory).

[0030] S200 optimizes the trained YOLOv5 target detection model and DDRNet semantic segmentation model based on the model inference optimizer TensorRT to improve the model's inference speed.

[0031] In an embodiment of the present invention, the model inference optimizer may use TensorRT launched by Nvidia. The optimization of the convolutional neural network model by TensorRT mainly includes reducing the amount of model inference calculation and reconstructing the model, specifically including:

[0032] The parameters of the FP32 data type in the convolutional neural network model are reduced to the FP16 or INT8 data type through mapping operations to reduce the amount of model inference calculations; and the network layers in the convolutional neural network model are simplified through inter-layer fusion and tensor fusion to simplify the model structure. Specifically, the common block structures in the convolutional neural network model can be merged into one network layer through inter-layer fusion, multiple network layers with the same structure can be merged into one network layer through tensor fusion, and model simplification methods such as automatic deletion of connection layers can be used to simplify the model structure.

[0033] By refactoring the model, the CUDA core usage and calling frequency of the GPU are reduced, making the entire model structure smaller, faster, and more efficient.

[0034] S300, deploying the optimized YOLOv5 target detection model and DDRNet semantic segmentation model, the classic image processing algorithm and the image acceleration decoder on the image detection device of the movable body; the movable inspection body is also provided with an image acquisition device.

[0035] In an illustrative embodiment of the present invention, the image acceleration decoder may be DeepStream. DeepStream is an SDK launched by Nvidia to accelerate streaming video analysis. Through this SDK, it is possible to quickly complete the process from video stream capture to output of deep learning algorithm recognition results on the GPU. A stream processing pipeline is built through DeepStream, and the camera video stream is input into the pipeline; the fast decoding module quickly decodes the video stream through the GPU and obtains sequence frame images; the image preprocessing module performs color channel conversion, size scaling and normalization on the image through the GPU, so that the image meets the input requirements of the YOLOv5 algorithm.

[0036] In an embodiment of the present invention, the movable inspection body may be an inspection vehicle that can move automatically. The image acquisition device may be composed of three industrial cameras, wherein one camera observes the front lower part of the inspection vehicle, and the other two cameras observe the side lower part of the inspection vehicle.

[0037] In a specific embodiment of the present invention, the image detection device may include an AI edge computing platform, and the Jetson AGX Xavier development kit may be used as the AI ​​edge computing platform of the embodiment of the present invention. The optimized YOLOv5 target detection model and DDRNet semantic segmentation model, the classic image processing algorithm and the image acceleration decoder can be deployed on the AI ​​edge computing platform. Specifically, the model inference optimizer TensorRT and the image acceleration decoder DeepStream are installed on the AI ​​edge computing platform, and the trained YOLOv5 target detection model and DDRNet model file (.PB) are optimized using TensorRT to generate an .engine file, and the optimized YOLOv5 target detection model and DDRNet model are loaded through the image acceleration decoder DeepStream to complete the deployment of the model.

[0038] S400, controlling the movable inspection body to travel along a preset travel path, and controlling the image acquisition device to acquire an image of the airport pavement.

[0039] The preset driving path can be determined based on the inspection task and can be generated by the control terminal.

[0040] S500, the image acceleration decoder acquires the collected airport pavement image in real time and decodes it.

[0041] S600 uses the optimized YOLOv5 target detection model, DDRNet semantic segmentation model and classic image processing algorithm to detect abnormal conditions in the decoded image and generate corresponding detection results. The abnormal conditions include pavement surface diseases, pavement foreign objects and abnormal wear of pavement markings.

[0042] In an embodiment of the present invention, the pavement surface defects include first surface defects and second surface defects, wherein the first surface defects include broken joints, peeling of board corners, patches, and potholes, and the second surface defects include cracks.

[0043] Furthermore, S600 further includes:

[0044] S610, using the optimized YOLOv5 target detection model to detect the first apparent disease, the marker line and the normal target in the decoded image, and marking the detected first apparent disease, the marker line and the normal target.

[0045] In an embodiment of the present invention, normal targets may include people, airplanes, vehicles, etc. Using an optimized YOLOv5 target detection model to detect the first apparent disease, marker line, and normal target in the decoded image, and marking the detected first apparent disease, marker line, and normal target may be a prior art. In an illustrative embodiment, an external rectangular frame may be used to mark the detected first apparent disease, marker line, and normal target. In addition, the optimized YOLOv5 target detection model will also obtain the type and location of the detected disease, etc.

[0046] S620, using the optimized DDRNet semantic segmentation model to perform the following operations:

[0047] S6210, respectively extracting the contours of the detected first apparent disease and the marker line to obtain contour information of the first apparent disease and the marker line.

[0048] S6220, detecting a second apparent defect in the decoded image, and extracting contour information of the second apparent defect when the second apparent defect is detected.

[0049] S6230, determining whether the marking line has abnormal wear based on the identification of the marking line and the contour information of the marking line.

[0050] Specifically, if the area A of the contour of the extracted marker line R The area of ​​the circumscribed rectangular frame A C Satisfy A R / A C <a, it is determined that the marking line has abnormal wear, and a is a set threshold value, which can be an empirical value, for example, 0.3.

[0051] S6240, extracting the contour of the road surface area in the decoded image to obtain first road surface contour information.

[0052] In this step, the road surface ROI in the image can be first extracted, and then the contour of the road surface area can be detected and extracted through the DDRNet semantic segmentation model.

[0053] S6250, removing the detected first apparent defect, second apparent defect, marker line and normal target from the first road surface contour information to obtain second road surface contour information.

[0054] In an embodiment of the present invention, the method of detecting diseases and extracting contours using a DDRNet semantic segmentation model may be prior art.

[0055] S630: Detect foreign matter in the second surface profile information using a classical image processing algorithm, and identify the foreign matter when it is detected.

[0056] This step may specifically include:

[0057] S6310, extracting the edge contour of the object to be identified from the second contour information using a classical image processing algorithm, and calculating the area enclosed by the edge contour of the object to be identified.

[0058] The object to be identified is an object detected in the second contour information and needs to be further determined whether it is a foreign object.

[0059] S6320: If the calculated area exceeds a preset value, the object to be identified is determined to be a foreign object, and a circumscribed rectangular frame is used to mark the determined foreign object. The preset value may be an empirical value, for example, 70 pixels*70 pixels.

[0060] In the embodiment of the present invention, the classical image processing algorithm may include spatial domain filtering, image edge detection, erosion and dilation algorithms, etc.

[0061] Figure 2 The structure block diagram of a real-time detection system for abnormal state of airport pavement provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, the real-time detection system for airport pavement defects provided by an embodiment of the present invention includes: a movable body 1, a control terminal 2 and a display terminal 8, the movable body includes an image acquisition device 3, an image detection device 4, a positioning device 5, a vehicle-mounted control device 6 and a communication device 7, and the movable body 1 and the display terminal 8 are respectively connected to the control terminal 2 for communication.

[0062] In the embodiment of the present invention, the movable body 1 is used to travel along a preset inspection path based on the motion control instruction sent by the control terminal 2, and the movable body 1 is an inspection vehicle that can move automatically.

[0063] The positioning device 5 can be composed of a Beidou antenna and an inertial navigation device, and is used to locate the inspection vehicle and send the positioning information to the remote control terminal 1. The vehicle-mounted control device 6 is used to execute the inspection control instructions of the control terminal 2 to the inspection vehicle, and control the inspection body to travel according to the set driving path. The communication device 7 is used for real-time communication between the inspection vehicle and the control terminal 2, for example, 5G can be used for real-time communication.

[0064] The control terminal 2 is equipped with a patrol control system, including a vehicle path planning module and a remote control module, which is used to send patrol control instructions to the vehicle control device 6 to automatically control or remotely temporarily control the movement route, i.e., the driving path, of the patrol vehicle. In the specific implementation, the control terminal 2 is equipped with a web-based program of the airport pavement intelligent patrol system, which has two main functions: one is to receive and save pavement patrol images and pavement abnormality detection results, and transmit pavement patrol images and pavement abnormality information to the display terminal 8; the other is to send the vehicle control system instructions to the vehicle control module on the patrol vehicle through the 5G network to control the movement of the patrol vehicle.

[0065] In the embodiment of the present invention, the image acquisition device 3 is composed of three industrial cameras, one of which is used to acquire the front and lower images of the movable body 1, and the other two industrial cameras are used to acquire the lower images of the movable body 1. It is used to acquire the image of the airport pavement and send it to the control terminal 2 through the 5G network, and transmit the image to the image detection device 4 through the data line.

[0066] Furthermore, in an embodiment of the present invention, the image detection device 4 includes an AI edge computing platform and a YOLOv5 target detection model and a DDRNet semantic segmentation model, an image acceleration decoder and a classic image processing algorithm optimized by a preset inference optimizer and deployed on the AI ​​edge computing platform.

[0067] Specifically, the inference optimizer can be TensorRT, and the image acceleration decoder can be DeepStream. The specific optimization process of the YOLOv5 target detection model and the DDRNet semantic segmentation model, as well as the specific deployment of the optimized YOLOv5 target detection model and the DDRNet semantic segmentation model, the image acceleration decoder, and the classic image processing algorithm can be referred to in the above embodiments.

[0068] The image detection device 4 is used to perform the following operations based on the received airport pavement image acquired by the image acquisition device 3:

[0069] Acquire and decode the collected airport pavement images in real time through an image acceleration decoder; and

[0070] The abnormal conditions in the decoded image are detected through the optimized YOLOv5 target detection model, DDRNet semantic segmentation model and classic image processing algorithm, and the corresponding detection results are generated and sent to the control terminal 2. The abnormal conditions include surface diseases of the road surface, foreign objects on the road surface and abnormal wear of the road surface marking lines.

[0071] Among them, the abnormal state in the decoded image is detected by using the optimized YOLOv5 target detection model, DDRNet semantic segmentation model and classic image processing algorithm, which can be specifically referred to S600 in the aforementioned embodiment.

[0072] In the embodiment of the present invention, the display terminal 8 is used to display the detection results. In the specific implementation, the display terminal 8 is used to display the web page program of the airport pavement intelligent inspection system built on the control terminal 2, and its main page content is the real-time picture of the pavement, the disease record column on the right, and the vehicle control module column below.

[0073] In specific applications, when the airport pavement abnormal state real-time detection system provided by the embodiment of the present invention is used to inspect the airport pavement, the following steps may be included:

[0074] Step 1: In the vehicle control module column of the airport pavement intelligent inspection system web page, select an inspection task and determine the inspection path of the inspection vehicle. The navigation device on the inspection vehicle sends the positioning information to the control terminal through the 5G network. The control terminal obtains the corresponding motion control instructions based on the inspection vehicle's location information and the planned inspection path, and sends the motion control instructions to the on-board control device on the inspection vehicle through the 5G network. The on-board control device controls the movement of the inspection vehicle to perform automatic inspections.

[0075] Step 2: During the vehicle inspection, the image acquisition device collects road surface images and transmits the images to the control terminal through the 5G network, and transmits the images to the image detection device through the data line. The image detection device detects abnormal conditions on the road surface images, and sends the detected abnormal conditions and location information of the road surface, such as apparent diseases, abnormal wear of marking lines, and foreign objects, to the control terminal through the 5G network.

[0076] Step 3: The control terminal receives and stores pavement images and abnormal status information. The pavement inspection images are stored as videos, and the pavement abnormal status and the location information of the inspection vehicle are saved in the memory. At the same time, the control terminal displays the pavement inspection images in real time on the web program of the airport pavement intelligent inspection system, and marks the type and location of the detected abnormal status in real time in the form of boxes and labels, and displays the defect pictures with outline marks in the defect record column in a timely manner.

[0077] Step 4: When the operator finds on the display that the system detects an abnormal pavement condition, he or she can manually remotely control the movement of the inspection vehicle through the control buttons on the remote control module in the vehicle control module column of the airport pavement intelligent inspection system web page to further confirm the abnormal pavement condition.

[0078] Step 5: After the abnormal status is confirmed, the operator selects to continue the inspection according to the inspection route in the vehicle control module column of the airport pavement intelligent inspection system web page. When the inspection vehicle reaches the end of the inspection route, the entire pavement inspection operation is completed.

[0079] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will also be appreciated by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A real-time detection method for abnormal state of airport pavement, characterized in that: The following steps are involved: S100, respectively optimizing and building a YOLOv5 target detection model and a DDRNet semantic segmentation model and training them to obtain a trained YOLOv5 target detection model and a DDRNet semantic segmentation model; S200, based on the preset inference optimizer, optimizes the trained YOLOv5 object detection model and DDRNet semantic segmentation model to improve the inference speed of the model; S300, deploying the optimized YOLOv5 target detection model and DDRNet semantic segmentation model, the classic image processing algorithm and the image acceleration decoder on the image detection device of the movable inspection body; the movable inspection body is also provided with an image acquisition device; S400, controlling the movable inspection body to travel along a preset travel path, and controlling the image acquisition device to acquire an image of the airport pavement; S500, the image acceleration decoder acquires the collected airport pavement image in real time and decodes it; S600 uses the optimized YOLOv5 target detection model, DDRNet semantic segmentation model and classic image processing algorithm to detect abnormal conditions in the decoded image and generate corresponding detection results. The abnormal conditions include pavement surface diseases, pavement foreign objects and abnormal wear of pavement markings. The pavement surface defects include first surface defects and second surface defects; S600 further includes: S610, using the optimized YOLOv5 target detection model to detect the first apparent disease, the marker line, and the normal target in the decoded image, and marking the detected first apparent disease, the marker line, and the normal target; S620, using the optimized DDRNet semantic segmentation model to perform the following operations: S6210, respectively extracting the contours of the detected first apparent disease and the marker line to obtain contour information of the first apparent disease and the marker line; S6220, detecting a second apparent defect in the decoded image, and extracting contour information of the second apparent defect when the second apparent defect is detected; S6230, determining whether the marking line has abnormal wear based on the identification of the marking line and the contour information of the marking line; S6240, extracting the contour of the road surface area in the decoded image to obtain first road surface contour information; S6250, removing the detected first apparent defect, second apparent defect, marker line and normal target from the first road surface contour information to obtain second road surface contour information; S630: Detect foreign matter in the second surface profile information using a classical image processing algorithm, and identify the foreign matter when it is detected.

2. The method according to claim 1, characterized in that Using an external rectangular frame to mark the detected marking line; S6230 further includes: If the area A of the contour of the extracted marker line R The area of ​​the circumscribed rectangular frame A C Satisfy A R / A C <a, it is determined that the marking line has abnormal wear, and a is a set threshold.

3. The method according to claim 1, characterized in that S630 further includes: S6310, extracting an edge contour of the object to be identified from the second contour information using a classical image processing algorithm, and calculating an area enclosed by the edge contour of the object to be identified; S6320: If the calculated area exceeds a preset value, the object to be identified is determined to be a foreign object, and an external rectangular frame is used to mark the determined foreign object.

4. According to the method of claim 1, the inference optimizer is TensorRT.

5. According to the method of claim 1, the image acceleration decoder is DeepStream.

6. The method according to claim 1, wherein the first apparent defects include broken joints, peeling of board corners, patches, and potholes, and the second apparent defects include cracks. The method according to claim 1 , wherein the normal targets include people, vehicles and airplanes.

8. A real-time detection system for abnormal conditions of airport pavement, characterized in that: It includes a movable inspection body and a control terminal which are in communication connection, and the movable inspection body is provided with an image acquisition device and an image detection device; The image detection device includes an image acceleration decoder, a classic image processing algorithm, a YOLOv5 target detection model optimized by a preset inference optimizer, and a DDRNet semantic segmentation model; The movable inspection body is used to perform inspections along a preset driving path based on control instructions sent by the control terminal; The image acquisition device is used to collect the airport pavement image during the inspection process of the movable inspection body and send it to the image detection device; The image detection device is used for: Acquire and decode the collected airport pavement images in real time through an image acceleration decoder; and The abnormal conditions in the decoded image are detected through the optimized YOLOv5 target detection model, DDRNet semantic segmentation model and classic image processing algorithm, and the corresponding detection results are generated and sent to the control terminal. The abnormal conditions include pavement surface diseases, pavement foreign objects and abnormal wear of pavement markings.

9. The system according to claim 8, characterized in that It also includes a display terminal for visually presenting the detection results.

Citation Information

Patent Citations

  • Airfield pavement disease foreign matter detection method based on fusion of multiple convolutional neural networks

    CN113111703A

  • Airfield pavement disease foreign matter detection method and system based on deep learning

    CN113111704A