Airport pavement foreign object detection method, server and storage medium

Abstract

This application discloses a method, server, and storage medium for detecting foreign objects on airport pavement, comprising the following steps: Step 1: Acquire comprehensive pavement data information; Step 2: Transfer the comprehensive pavement data information to independently distributed data points using pixel coordinates and color attributes to form unstructured data; Step 3: Preprocess the unstructured data and perform lateral convolution after preprocessing; Step 4: Process the unstructured data after lateral convolution using gradient downsampling; Step 5: Input the processed unstructured data set into an airport foreign object detection network for feature extraction, and output the extracted foreign object information. By using lateral convolution and gradient downsampling to reduce data jumps caused by noise, uneven driving, and lighting, the airport foreign object detection network is modeled based on data self-drive, automatically eliminates errors, and is able to adapt to different airport environments.

Description

Technical Field

[0001] This application belongs to the field of detection data processing, and in particular relates to a method, server, and storage medium for detecting foreign objects on airport pavement. Background Art

[0002] Traditional airport foreign object detection primarily relies on imaging and radar radio waves. Imaging technology primarily uses cameras to capture images of the airport pavement. The acquired data is then processed using frame subtraction, grayscale mutation segmentation, and other techniques to obtain information about foreign objects on the road surface. This method is low-cost and easy to implement, but is significantly affected by visible and ambient light. Image acquisition can be affected by vehicle vibrations and the environment, resulting in image distortion, low detection accuracy, and inefficiency. Extracting information about foreign objects from these distorted images and accurately and efficiently detecting them remains a pressing challenge.

[0003] Radar radio waves are widely used. The principle is that radar equipment transmits electromagnetic wave energy to the free space of the entire airport surface. If the electromagnetic wave passes through a foreign object, it returns to the radar. The radar receives the echo signal and sends it to the receiving equipment for processing, obtaining information about the foreign object's location. However, radio waves can affect aircraft takeoff and landing, and can be affected by clutter, which can limit detection accuracy.

[0004] Currently, laser scanning 3D imaging is the most widely used method for detecting foreign objects at airports. This requires modeling the airport, which is a significant workload. With the widespread adoption of automated modeling and feature extraction technologies, this workload has been significantly reduced, and detection efficiency has been improved, even with large data volumes. However, the training required before detection remains a significant workload. Furthermore, the development of detection systems requires highly qualified personnel, resulting in high R&D costs.

[0005] Therefore, a new method for detecting foreign objects on airport pavement is needed to solve the above problems. Summary of the Invention

[0006] In order to address the shortcomings of the existing technology, the present application provides a method, server and storage medium for detecting foreign objects on airport pavement. It uses unstructured fusion data composed of multiple acquisition sources to obtain a three-dimensional unstructured data set of the road surface. The problem of data jump caused by noise, uneven driving, lighting and other environmental factors is reduced through the methods of lateral convolution and gradient downsampling. The airport foreign object detection network is modeled based on data self-drive, eliminates errors by itself, and can adapt to different airport environments.

[0007] The technical effects to be achieved by this application are achieved through the following solutions:

[0008] According to a first aspect of the present invention, a method for detecting foreign objects on an airport pavement is provided, comprising the following steps:

[0009] Step 1: Obtain comprehensive road surface data information;

[0010] Step 2: Transferring the comprehensive road surface data information to independently distributed data points through the coordinates and color attributes of the pixel points to form unstructured data;

[0011] Step 3: Preprocess the unstructured data and perform horizontal convolution processing after the preprocessing;

[0012] Step 4: The unstructured data after horizontal convolution is processed by gradient downsampling;

[0013] Step 5: Input the processed unstructured data set into the airport foreign object detection network for feature extraction, and output the extracted foreign object information.

[0014] Preferably, the comprehensive road surface data information includes at least multispectral data, depth data and road surface spatial position data information at different angles.

[0015] Preferably, in step 2, a road surface coordinate system is established based on the comprehensive road surface data information, which is then converted into 3D distributed data points through camera calibration technology, and finally the coordinates and color attributes of the pixel points are migrated to the data points so that the road surface has different color information.

[0016] Preferably, in step 3, the preprocessing includes: extracting and segmenting features of mutations and anomalies in the unstructured data of the entire area, and filtering out abnormal values ​​to achieve denoising and filtering operations.

[0017] Preferably, in step 3, the lateral convolution processing is specifically as follows: using a convolution kernel to perform convolution processing on the depth information of the selected area, the convolution operation moves laterally along the input data, and the change in the depth value between the center point and the field point is calculated. The result of the convolution is used as the current depth value, and vibration compensation is performed according to the change in the depth value.

[0018] Preferably, in step 4, the gradient principle is used to calculate the severity of the spatial structure change of the unstructured data of road foreign matter, and then different data points are classified according to the gradient size, and different proportions of sampling are performed on different categories of data.

[0019] Preferably, a gradient threshold is set, and the gradients of two adjacent points are calculated. When the gradient value is greater than the gradient threshold, it is placed in set H, otherwise it is placed in set L. The sampling formula is as follows:

[0020] M=H×T+L×(1-T)

[0021] Where M is the expected number of sampling points and T is the sampling rate of high gradient areas.

[0022] Preferably, the airport foreign object detection network includes an unstructured data input layer, a feature extraction layer and a foreign object information output layer, wherein the feature extraction layer performs feature extraction on the unstructured data input in the unstructured data input layer, and the extracted foreign object information is output by the foreign object information output layer to obtain the detected foreign object.

[0023] According to a second aspect of the present invention, there is provided a server comprising: a memory and at least one processor;

[0024] The memory stores a computer program, and the at least one processor executes the computer program stored in the memory to implement any one of the above-mentioned methods for detecting foreign objects on an airport pavement.

[0025] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed, any one of the above-mentioned methods for detecting foreign objects on an airport pavement is implemented.

[0026] The present invention achieves the beneficial effect of using unstructured fusion data from multiple acquisition sources to acquire a three-dimensional unstructured dataset of the road surface. Due to potential errors caused by vibration, the data needs to be vibration-compensated to improve detection accuracy. Based on this, the present invention proposes a data processing method called lateral convolution to offset vibration errors and designs a gradient downsampling method to address the heavy workload caused by excessive unstructured data.

[0027] By filtering and segmenting unstructured data, and using horizontal convolution to eliminate errors, a three-dimensional unstructured dataset is generated. Smooth data is ignored, and data with anomalies is extracted. The extracted unstructured data is classified according to different gradients, and different categories of data are sampled at different proportions. The data is then input into the airport foreign object detection network for identification and detection.

[0028] Compared with traditional foreign body detection technology and laser detection technology, the present invention has strong generalization ability, combines the advantages of spatial information, fully utilizes the multidimensional attributes of data, makes improvements in feature extraction, improves detection accuracy, and reduces the workload of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0030] Figure 1 This is a flow chart of a method for detecting foreign objects on an airport road surface in one embodiment of the present application;

[0031] Figure 2 This is a structural diagram of a server in one embodiment of the present application. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] like Figure 1 As shown, the airport pavement foreign object detection method in one embodiment of the present application includes the following steps:

[0034] S110: Obtaining comprehensive road surface data information;

[0035] In this step, the multispectral camera, depth camera and linear array camera installed on the cantilever are directed towards the road surface to obtain road surface location information. At the same time, the lidar scans the road surface to comprehensively obtain multispectral data, depth data, color attributes and road surface spatial location data information of road surface foreign objects at different angles. The road surface spatial location data information at different angles is obtained by the lidar and is used to determine the location and shape information of road surface foreign objects from multiple angles.

[0036] S120: Transferring the comprehensive road surface data information to independently distributed data points through the coordinates and color attributes of the pixel points to form unstructured data;

[0037] In this step, a road surface coordinate system is established based on the comprehensive road surface data information, and then converted into 3D distributed data points through camera calibration technology. Finally, the coordinates and color attributes of the pixel points are migrated to the data points to give the road surface different color information, so as to observe the road surface information and foreign object information more intuitively.

[0038] In this embodiment, the independently distributed data points have the color position information of the road surface, and are permutation invariant and rotation invariant. The different road surface information acquired by the camera is transferred to the independently distributed data points through camera calibration and color migration in the color space.

[0039] S130: Preprocessing the unstructured data, and performing horizontal convolution processing after the preprocessing;

[0040] In this step, preprocessing includes: extracting and segmenting the mutations and anomalies of the unstructured data in the entire area, filtering out the outliers to achieve denoising and filtering operations; and then performing further horizontal convolution processing on the processed data to eliminate vibration errors.

[0041] Ideally, road depth data should be parallel to the road surface, excluding foreign objects. Based on this, lateral convolution processing involves convolving the depth information of a local area with a convolution kernel. The convolution operation moves laterally along the input data, calculating the depth change between the center point and the boundary point. The convolution result is used as the current depth value, and vibration compensation is performed based on this change in depth value.

[0042] S140: performing gradient downsampling on the unstructured data after the horizontal convolution processing;

[0043] In this step, the gradient principle is used to calculate the severity of the spatial structure change of the unstructured data of road foreign matter, and then different data points are classified according to the gradient size, and different proportions of sampling are performed on different categories of data. Specifically:

[0044] The unstructured data is recorded as set C, the expected number of sampling points is M, the high gradient area is recorded as H, the low gradient area is recorded as L, and the sampling rate of the high gradient area is T. A gradient threshold is set, and the gradient from the point to the neighboring points is calculated. When the gradient between the point and the neighboring points is greater than the set threshold, the point is placed in set H, otherwise it is placed in set L. Based on this, the unstructured data is divided into two parts, and different areas are sampled. The sampling formula is as follows:

[0045] M=H×T+L×(1-T)

[0046] The threshold is distinguished by the gradient size, and combined with different sampling rates to make the contour features of the target more obvious, which retains the characteristic information of foreign objects and ensures low time complexity.

[0047] S150: Input the processed unstructured data set into the airport foreign object detection network for feature extraction, and output the extracted foreign object information.

[0048] In this step, the airport foreign object detection network includes an unstructured data input layer, a feature extraction layer, and a foreign object information output layer. The feature extraction layer performs feature extraction on the unstructured data input in the unstructured data input layer, and the extracted foreign object information is output by the foreign object information output layer to obtain the detected foreign object.

[0049] After the sensor acquires road surface data through the above steps, it needs to be corrected to eliminate errors caused by vibration or uneven driving. Ideally, the unstructured data captured by the sensor is assumed to be parallel to the road surface. Based on this, a convolution kernel is selected to convolve the unstructured data in a local area. Based on the principle that lasers generate scan data line by line, the unstructured data is convolved horizontally to offset vibration errors. This convolution calculates the change in depth between the center point and neighboring points, and the convolution is performed laterally along the input data. Because the runway surface height changes slowly and is approximately flat, the height difference between foreign objects and the unstructured road surface data in the same area is significant. A sudden change in the data height indicates the presence of a foreign object, and the sudden change in the foreign object depth is preserved. In areas where the depth changes more gradually, the error is assumed to be caused by camera shake.

[0050] After acquiring unstructured data from airport pavement, it needs to be input into the airport foreign object detection network. However, the spatial structure of the pavement is too complex, and the amount of unstructured data obtained is too large, affecting detection efficiency and effectiveness. Based on this, a method for gradient downsampling of unstructured data is proposed. After error correction through lateral convolution, the unstructured data of sections without foreign objects should be horizontally distributed. When foreign objects are present, the spatial data will undergo drastic changes. Based on this, more sampling is performed in areas where the unstructured data has large gradient changes to extract the characteristic outlines of the foreign objects. The gradient principle is used to calculate the degree of spatial structural change in the pavement foreign object data. Different points are then classified according to the gradient size, and different data categories are sampled at different proportions to reduce the data processing workload.

[0051] In one embodiment of the present application, a server is also provided. Figure 2 As shown, it includes: a memory 201 and at least one processor 202;

[0052] The memory 201 stores a computer program, and the at least one processor 202 executes the computer program stored in the memory 201 to implement any of the above-mentioned methods for detecting foreign objects on an airport pavement.

[0053] In one embodiment of the present application, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed, any of the above-mentioned methods for detecting foreign objects on an airport pavement is implemented.

[0054] The present invention achieves the beneficial effect of using unstructured fusion data from multiple acquisition sources to acquire a three-dimensional unstructured dataset of the road surface. Due to potential errors caused by vibration, the data needs to be vibration-compensated to improve detection accuracy. Based on this, the present invention proposes a data processing method called lateral convolution to offset vibration errors and designs a gradient downsampling method to address the heavy workload caused by excessive unstructured data.

[0055] By filtering and segmenting unstructured data, and using horizontal convolution to eliminate errors, a three-dimensional unstructured dataset is generated. Smooth data is ignored, and data with anomalies is extracted. The extracted unstructured data is classified according to different gradients, and different categories of data are sampled at different proportions. The data is then input into the airport foreign object detection network for identification and detection.

[0056] Compared with traditional foreign body detection technology and laser detection technology, the present invention has strong generalization ability, combines the advantages of spatial information, fully utilizes the multidimensional attributes of data, makes improvements in feature extraction, improves detection accuracy, and reduces the workload of data processing.

[0057] It should be noted that the above detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs.

[0058] It should be noted that the terms used herein are intended only to describe specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0059] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0060] In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0061] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used herein to describe the spatial positional relationship of a device or feature to other devices or features as shown in the figures. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figures. For example, if the device in the drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be subsequently positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways, such as rotated 90 degrees or in other orientations, and the spatially relative descriptions used herein are interpreted accordingly.

[0062] In the above detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless the context dictates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be used, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.

[0063] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for detecting foreign objects on airport pavement, characterized in that: The steps include: Step 1: Obtain comprehensive road surface data information; the comprehensive road surface data information includes at least multispectral data, depth data, color attributes, and road surface spatial position data information at different angles; Step 2: Transferring the comprehensive road surface data information to independently distributed data points through the coordinates and color attributes of the pixel points to form unstructured data; Step 3: Preprocess the unstructured data and perform lateral convolution after preprocessing; the preprocessing includes: extracting and segmenting the mutations and anomalies of the unstructured data in the entire area, and filtering out the outliers to achieve denoising and filtering operations; Step 4: The unstructured data after horizontal convolution is processed by gradient downsampling; Step 5: Input the processed unstructured data set into the airport foreign object detection network for feature extraction, and output the extracted foreign object information; the airport foreign object detection network includes an unstructured data input layer, a feature extraction layer, and a foreign object information output layer, wherein the feature extraction layer performs feature extraction on the unstructured data input into the unstructured data input layer, and the extracted foreign object information is output by the foreign object information output layer to obtain the detected foreign object.

2. The method for detecting foreign matter on an airport pavement according to claim 1, characterized in that: In step 2, a road surface coordinate system is established based on the comprehensive road surface data information, which is then converted into 3D distributed data points through camera calibration technology. Finally, the coordinates and color attributes of the pixel points are transferred to the data points to give the road surface different color information.

3. The method for detecting foreign matter on an airport pavement according to claim 1, characterized in that: In step 3, the horizontal convolution processing is specifically as follows: use a convolution kernel to perform convolution processing on the depth information of the selected area, the convolution operation moves horizontally along the input data, calculates the change in depth value between the center point and the field point, and uses the result of the convolution as the current depth value. Vibration compensation is performed according to the change in depth value.

4. The method for detecting foreign matter on an airport pavement according to claim 1, wherein: In step 4, the gradient principle is used to calculate the severity of the spatial structure change of the unstructured data of road foreign matter, and then different data points are classified according to the gradient size, and different proportions of data of different categories are sampled.

5. The method for detecting foreign matter on an airport road surface according to claim 4, characterized in that: Set a gradient threshold and calculate the gradient of two adjacent points. When the gradient value is greater than the gradient threshold, put it into the set H, otherwise put it into the set L. The sampling formula is as follows: ; Where M is the expected number of sampling points and T is the sampling rate of high gradient areas.

6. A server, characterized in that: include: memory and at least one processor; The memory stores a computer program, and the at least one processor executes the computer program stored in the memory to implement the airport pavement foreign object detection method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed, implements the airport pavement foreign object detection method according to any one of claims 1 to 5.

Images