Runway foreign matter detection system based on image recognition

Abstract

The invention relates to the technical field of runway safety detection, and discloses a runway foreign matter detection system based on image recognition. The system comprises a runway block division unit, a reference feature library construction unit, a real-time image analysis unit, a feature deviation analysis unit and an alarm decision unit. According to the system, a runway is divided into a plurality of independent detection blocks, and a reference texture feature set and an optical reflection feature set of each block in a foreign matter-free state are constructed; collecting a real-time image sequence by using a multispectral imaging device, and extracting real-time texture and reflection feature vectors; comparing the real-time feature vector with the reference feature set to generate a texture deviation matrix and a reflection deviation matrix; and generating a foreign matter positioning coordinate and an alarm level instruction based on the conjoint analysis result. The system realizes refined and multi-dimensional detection of the runway foreign matters, can accurately position the foreign matters and give corresponding alarms, and is suitable for runway safety detection scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of runway safety detection, in particular to a runway foreign object detection system based on image recognition. BACKGROUND

[0002] Currently, runway foreign objects are one of the important factors affecting aviation safety. These foreign objects may include gravel, metal fragments, plastic garbage, bird carcasses, etc., which can easily cause serious flight accidents once sucked into the aircraft engine or collide with the landing gear. The existing runway foreign object detection methods mostly rely on manual inspection, and the detection personnel need to regularly check along the runway. This method is limited by labor costs and working hours, and it is difficult to achieve all-weather and full-coverage monitoring. Especially in complex environments such as night and bad weather, the accuracy and efficiency of manual detection are greatly reduced, and there are obvious detection blind spots. Some airports have tried to use single-spectrum imaging devices for auxiliary detection, but such devices can only capture part of the optical characteristics of the object. For foreign objects with colors close to the runway surface or special materials, it is easy to miss detection or misjudge. At the same time, the traditional detection method lacks systematic division of the runway area, and when foreign objects appear, it is difficult to quickly locate their specific position, prolonging the disposal time. The analysis of foreign object features in the prior art is mostly limited to a single dimension, either focusing only on texture changes or relying only on reflection characteristics, without forming a multi-feature collaborative judgment mechanism, resulting in insufficient reliability of the detection results. During the peak period of flight takeoff and landing, this inefficient detection method may also affect the normal use efficiency of the runway, indirectly increasing the operating pressure. SUMMARY

[0003] The purpose of the present application is to provide a runway foreign object detection system based on image recognition to solve the problems raised in the background art.

[0004] To achieve the above purpose, the present application provides a runway foreign object detection system based on image recognition, which comprises: a runway block division unit for dividing the runway surface into a plurality of independent detection blocks based on predefined runway zoning rules; a reference feature library construction unit for storing the reference texture feature set and the optical reflection feature set of each detection block in the absence of foreign objects; a real-time image analysis unit for collecting real-time image sequences of the current detection block through a multispectral imaging device deployed around the runway perimeter, and extracting real-time texture feature vectors and reflection feature vectors thereof; a feature deviation analysis unit configured to dynamically compare the real-time texture feature vector with the reference texture feature set to generate a texture deviation matrix, and to compare the real-time reflection feature vector with the optical reflection feature set to generate a reflection deviation matrix; an alarm decision unit configured to generate foreign object location coordinates and alarm level instructions based on a joint analysis result of the texture deviation matrix and the reflection deviation matrix.

[0005] Preferably, the runway block division unit performs the following operations: acquire actual physical size data and surface material distribution data of the runway; divide the runway into equal-area grid blocks according to a predefined block side length threshold with the runway centerline as the reference axis; assign corresponding material feature codes according to the surface material type of each grid block; bind the material feature codes and block coordinates and transmit them to the reference feature library construction unit.

[0006] Preferably, the reference feature library construction unit performs the following operations: receive the material feature codes and block coordinates from the runway block division unit; continuously collect reference image sets of each block in an undisturbed state through a multispectral imaging device within a predefined reference collection period; perform multiscale feature decomposition on the reference image sets of each block to extract its spatial domain texture features and frequency domain reflection features, respectively; store the spatial domain texture features as the reference texture feature set and store the frequency domain reflection features as the optical reflection feature set.

[0007] Preferably, the real-time image analysis unit includes: an image preprocessing subunit configured to perform illumination equalization and motion blur correction on the collected real-time image sequence; a feature mapping subunit configured to extract a multi-level texture feature mapping from the corrected image through a deep convolutional network; a reflection analysis subunit configured to analyze infrared spectral band data in the image to generate a reflection intensity distribution histogram; a coordinate correlation subunit configured to align the multi-level texture feature mapping and the reflection intensity distribution histogram according to pixel coordinates to generate a real-time feature vector set.

[0008] Preferably, the feature deviation analysis unit performs the following operations: receive the reference texture feature set and the optical reflection feature set from the reference feature library construction unit; acquire the real-time feature vector set generated by the real-time image analysis unit; calculating a Euclidean distance matrix of the real-time texture feature vector and the corresponding block reference texture feature set as a texture deviation matrix; calculating a spectral angle difference value of the real-time reflection feature vector and the corresponding block optical reflection feature set as a reflection deviation matrix; transmitting the texture deviation matrix and the reflection deviation matrix to an alarm decision unit.

[0009] Preferably, the system further comprises a multi-level detection model configuration unit, which comprises: a model initialization subunit for loading a pre-trained foreign matter detection neural network model; a parameter adaptation subunit for dynamically adjusting the model convolution kernel weight according to the reference feature set from the reference feature library construction unit; a feature fusion subunit for tensor splicing the texture deviation matrix and the reflection deviation matrix from the feature deviation analysis unit; a model inference subunit for generating a foreign matter probability heat map by processing the spliced tensor through the adjusted foreign matter detection neural network.

[0010] Preferably, the multi-level detection model configuration unit further comprises: a model integration control subunit for obtaining variance distribution data of each block feature set in the reference feature library construction unit; a weight distribution subunit for calculating the confidence weight of each block detection model according to the variance distribution data; a decision fusion subunit for weighted fusion of foreign matter probability heat maps from different blocks to generate a global detection result.

[0011] Preferably, the alarm decision unit performs the following operations: receiving the global detection result from the multi-level detection model configuration unit; obtaining the texture deviation matrix and the reflection deviation matrix transmitted by the feature deviation analysis unit; when the foreign matter probability in the global detection result exceeds a predefined threshold, locating the foreign matter center coordinates according to the spatial distribution gradient of the texture deviation matrix and the reflection deviation matrix; calculating the threat level of the foreign matter based on the deviation amplitude of the pixels around the foreign matter center coordinates; triggering the corresponding level of alarm protocol according to the threat level.

[0012] Preferably, the alarm decision unit further comprises: a time series verification subunit for continuously obtaining the global detection result of multiple detection periods; a state duration judgment subunit for starting an emergency alarm instruction when the foreign matter threat level maintains an upward trend within a predefined duration. False alarm filtering subunit, ignoring alarm triggering when foreign object threat level exceeds threshold in a single cycle but disappears in the next cycle.

[0013] Preferably, the system further comprises a data management unit, the data management unit comprising: Storage optimization subunit, for obtaining the spatio-temporal distribution data of the texture deviation matrix and the reflection deviation matrix generated by the feature deviation analysis unit; Data compression subunit, initiating lossy compression of the feature matrix when the variance of the spatio-temporal distribution data is continuously below a predefined compression threshold; Storage space allocation subunit, dynamically adjusting the historical data storage period according to the alarm frequency generated by the alarm decision unit.

[0014] Compared with the prior art, the present application has the following advantages: The runway surface is divided into several independent detection blocks by the runway block division unit, making the detection process more targeted and enabling fine-grained monitoring of each block, thus avoiding the omission of details due to excessively large regions during overall detection. The detection data of each independent block is relatively concentrated, facilitating subsequent feature analysis and comparison and reducing the interference of irrelevant region data on the detection result. The baseline texture feature set and the optical reflection feature set of each detection block in the absence of foreign objects stored by the baseline feature library construction unit provide accurate reference standards for real-time detection. These baseline feature sets are established for each block individually, fully considering the inherent differences that may exist in different blocks, such as local changes in runway material and fixed features formed by long-term effects of lighting conditions, making subsequent feature comparison more accurate. The real-time image analysis unit captures real-time image sequences with the aid of multispectral imaging equipment deployed around the perimeter of the runway and extracts real-time texture feature vectors and reflection feature vectors, enabling the capture of various features exhibited by foreign objects under different spectra. Multispectral imaging can obtain more abundant object information than single-spectrum imaging, whether it is a dull-colored gravel or a transparent plastic fragment, which can be effectively identified through its unique texture and reflection characteristics, thus widening the range of detectable foreign objects. The feature deviation analysis unit compares the real-time feature vectors with the baseline feature sets to generate a texture deviation matrix and a reflection deviation matrix, achieving multi-dimensional analysis of foreign object features. The texture deviation matrix can reflect the differences in texture structure between the real-time image and the baseline image, such as surface concave-convex changes caused by foreign objects; the reflection deviation matrix can reflect changes in optical reflection characteristics, such as the difference in reflectivity between metal foreign objects and the runway surface. The combination of the two matrices presents the possible features of foreign objects from different angles, reducing the one-sidedness of single-feature analysis. The alarm decision unit generates the foreign matter positioning coordinates and the alarm level instruction according to the joint analysis result of the texture deviation matrix and the reflection deviation matrix, so that the positioning of the foreign matter is more accurate. Due to the block division in the early stage, combined with the specific position information of the feature change in the deviation matrix, the accurate coordinates where the foreign matter is located can be quickly locked, so that the relevant personnel can quickly go to deal with it. At the same time, based on the alarm level instruction obtained by the joint analysis of multiple features, the potential harm degree of the foreign matter can be more reasonably reflected, which helps the relevant departments to take corresponding disposal measures according to the level, and improves the response efficiency of dealing with the foreign matter problem. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A working principle diagram of the runway foreign matter detection system based on image recognition is described. Figure 2 A flowchart executed by the runway block division unit is described. Figure 3 A flowchart executed by the feature deviation analysis unit is described. Figure 4 A flowchart executed by the multi-level detection model configuration unit is described. Figure 5 A flowchart executed by the alarm decision unit is described. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0017] Please refer to Figure 1 The present application provides a runway foreign matter detection system based on image recognition, which comprises a runway block division unit, a reference feature library construction unit, a real-time image analysis unit, a feature deviation analysis unit and an alarm decision unit, which cooperate with each other. The specific implementation is as follows: The runway block division unit divides the runway surface into independent detection blocks, the reference feature library construction unit stores the feature of each block without foreign matter, the real-time image analysis unit collects and analyzes the image data of the current block, the feature deviation analysis unit compares the real-time feature with the reference feature to generate a deviation matrix, and the alarm decision unit locates the foreign matter coordinates based on the analysis result of the deviation matrix and generates an alarm instruction.

[0018] Embodiment 1: refer to Figure 2The runway block division unit needs to obtain the physical basic data of the runway before performing operations. This unit extracts the actual physical size data of the runway from the airport digital elevation model, including the length, width, and surface curvature parameters of the runway. At the same time, the runway surface material distribution data is collected by a vehicle-mounted infrared spectrum scanner, and the material types of different regions are identified, such as standard concrete, modified asphalt, or reflective marker coating. The data input is stored in a structured format, including the geographical coordinate boundary and the material physical property mapping table. During processing, the unit takes the geographical coordinates of the runway centerline as the reference axis, and the axis definition is generated according to the International Civil Aviation Organization runway identification specification.

[0019] The grid partitioning algorithm is based on a pre-defined block edge length threshold. The edge length threshold is set to 5 meters, and the runway surface is divided into a rectangular grid array through spatial geometry algorithms. The partitioning process uses two-dimensional coordinate transformation techniques, establishes a local coordinate system based on the runway centerline, cuts it at equal intervals along the length direction, and segments it according to the threshold in the width direction. Each grid block generates a unique coordinate identifier, using a binary encoding format of "longitude offset - latitude offset". The division result forms an equal-area block set, and the number of blocks is dynamically calculated according to the runway size, for example, a standard 3600-meter runway is divided into 1728 independent blocks.

[0020] Material feature coding is the core operation of the unit. The system has a built-in material coding table, with a 4-bit fixed-length string: the first bit represents the material category (C for concrete, A for asphalt), and the last three bits represent the reflectivity classification (e.g. 045 for 45% reflectivity). The unit performs spatial superposition analysis to geographically register the material distribution data with the grid block boundaries, matches the dominant material type for each block, and assigns the code. After coding, the unit establishes a mapping table between block coordinates and materials, with a key-value pair structure, where the key is the block coordinate identifier and the value is the material code string. This mapping table is transmitted to the reference feature library construction unit through a distributed message queue, and a timestamp and data check code are attached during transmission.

[0021] The reference feature library construction unit starts reference collection during low-altitude activity periods. The pre-defined reference collection period is set to 02:00-04:00 local time, and the runway clearance status is verified through the airport air traffic control system during this period. The unit controls a multi-spectral imaging device group deployed on the towers on both sides of the runway, which includes a visible light camera and a short-wave infrared sensor. The collection process performs an automated sequence of shots for each block: a 30-second image stream is collected for each block, with a frame rate of 24fps, a visible light resolution of 4096x2160 pixels, and infrared band spectral data collection of 8-14μm. The generated raw reference image set is stored as a time series file set in units of blocks.

[0022] The multi-scale feature decomposition process is divided into two parallel computing streams. The spatial domain texture feature extraction stream processes the visible light image sequence, and a four-level Gaussian pyramid downsampling is used to construct multi-scale spaces. In each level of the scale space, a gray level co-occurrence matrix algorithm is applied to calculate the contrast, correlation and entropy values in the 0°, 45°, 90° and 135° directions, generating a 144-dimensional texture feature vector. The frequency domain reflection feature processing stream focuses on the infrared image sequence. First, the fast Fourier transform is performed on each frame to extract the frequency domain coefficients, and then the energy values of the eight feature bands are calculated. At the same time, a statistical histogram is generated by analyzing the pixel-level reflectivity to quantify the distribution probability of the reflectivity intensity in the [0, 1] interval.

[0023] The feature storage adopts a distributed database architecture for management. The benchmark texture feature set is organized in a tree structure: the root node is the block coordinate ID, and the child nodes contain feature vector sets with different time stamps. The optical reflection feature set is stored as a three-order tensor structure, with the first order corresponding to the block grid position, the second order representing the time sequence index, and the third order containing the frequency band energy values and reflection histogram binning data. The feature library implements a version control mechanism, generating a new feature snapshot for each benchmark update, and preserving historical versions for state rollback. All feature data is stored in a solid-state storage array after AES-256 encryption, and the index system supports millisecond-level feature retrieval.

[0024] The unit performs periodic feature calibration. Automatic benchmark feature verification is performed every 24 hours, and the verification process includes: collecting the latest benchmark image set, extracting the feature vector, and comparing it with the stored benchmark for cosine similarity. When the similarity is lower than the preset tolerance value, the feature library automatic update process is started, and the new feature set is covered after multiple rounds of noise point filtering. The update log records the changed block coordinates and feature difference values, and the feature drift trend is visualized through the system monitoring interface.

[0025] The benchmark feature library provides basic data support for the whole system. The optical parameters of the multi-spectral imaging device are accurately corresponding to the block division position, and the device spatial resolution is set to correspond to an actual 3.2mm x 3.2mm ground area per pixel. The stored benchmark features contain time dimension information, and the single-day benchmark data amount reaches 17TB scale, using columnar compression storage technology to reduce the effective storage volume to 35% of the original data. The unit has a fault recovery mechanism, when the hardware abnormally interrupts the collection, it automatically switches to the redundant device to continue the task, and marks the feature data of the abnormal time period as a to-be-calibrated state.

[0026] Example 2: see Figure 3, real-time image analysis unit processes continuous data stream transmitted by multi-spectral imaging device. Image pre-processing sub-unit receives raw image sequence, which consists of interleaved RGB frames from visible light channel and thermal radiation frames from infrared channel. Visible light frame processing adopts adaptive histogram equalization algorithm, which analyzes regional distribution characteristics of image brightness, and separates high light and shadow regions to adjust gamma value respectively; infrared frame processing focuses on thermal radiation data, and eliminates pixel value drift caused by environmental temperature disturbance. Motion blur correction aims at device jitter problem, and captures attitude angular velocity through gyroscope sensor, calculates point spread function to reconstruct clear image, and dynamically selects kernel size according to wind speed data. Processed dual-channel images are temporarily stored in circular buffer, and the latest five frames of data are reserved for time domain analysis.

[0027] Feature mapping sub-unit configures special hardware accelerator. The unit loads pre-constructed deep convolutional network model, and the input layer of the network receives pre-processed visible light image. The model architecture contains five feature extraction stages: the first stage adopts 7x7 convolution kernel to output 64-channel feature map; the second stage applies 3x3 convolution with a step of 2 to realize down-sampling; the third to fifth stages use residual connection structure to extract multi-level features. The output of each stage is uniformly scaled to the original Figure 1 / 4 size through bilinear interpolation, forming a stack of spatially aligned multi-level feature maps. The bottom layer of the stack retains edge detail features, and the high layer abstracts foreign object outline patterns. The format of the mapping data is floating-point tensor, and each pixel position is attached with a 16-dimensional feature descriptor.

[0028] Reflection analysis sub-unit independently processes infrared spectral data stream. The unit analyzes 8-14 μm waveband information in infrared image, and separates background radiation and target reflection component. Reflection intensity calculation converts thermal radiation value into reflectivity based on Planck's radiation law, and the range is calibrated in the interval of 0.2-0.95 reflectivity. Dynamic binning mechanism is adopted for distribution histogram construction, and the bin width is automatically adjusted according to the current scene reflectivity range, with the minimum binning unit being 0.005 reflectivity value. The histogram output is a 256-dimensional normalized vector, and each dimension represents the pixel ratio of the corresponding reflectivity interval in the block. The processing process is accompanied by abnormal pixel filtering, and the outliers caused by lens stains are removed.

[0029] The coordinate association subunit establishes the correspondence relationship of multi-source data space. The unit receives the multi-level texture feature tensor output by the feature mapping subunit and the reflection intensity vector generated by the reflection analysis subunit. The texture feature tensor is converted to the pixel coordinate system through the coordinate mapping table, which records the projection relationship between the network output feature points and the original image coordinates. The reflection vector is matched with the resolution of the visible light image through spatial resampling, and the data continuity is maintained using bicubic interpolation. The alignment operation is implemented based on the runway coordinate system, and the texture features and reflection data are registered according to the geographical coordinate grid. Each spatial grid point generates a composite feature vector, which includes 16-dimensional texture attributes, 1-dimensional reflection value and 3-dimensional position encoding. The final real-time feature vector set is organized in time sequence, and each feature point is attached with a millisecond-level timestamp and a block ID index.

[0030] The feature deviation analysis unit performs dynamic difference detection. The unit loads the reference data of the reference feature library, and the reference texture feature set is stored as a sparse matrix structure, including standard texture templates under different lighting conditions; the optical reflection feature set includes the reflectivity baseline caused by seasonal changes. When the texture deviation calculation is started, the unit retrieves the real-time texture vector and the corresponding reference data according to the block ID. The Euclidean distance algorithm calculates the feature space distance pixel by pixel, and the distance value is smoothed by a Gaussian window to reduce local mutation interference. The output texture deviation matrix includes the difference value of each grid point, and the data is stored as a single-precision floating-point two-dimensional array.

[0031] The reflection deviation analysis adopts a spectral feature comparison mode. The unit analyzes the real-time reflection vector and the reference reflection set, and regards them as vector elements in a high-dimensional spectral space. The spectral angle mapping algorithm calculates the cosine value of the included angle between the vectors, and outputs the angle difference through the inverse cosine transformation. The difference value is normalized to the interval [0, 1], and 0 represents a complete matching state. The reflection deviation matrix is generated to mark the spectral mismatch area, and the matrix dimension is consistent with the texture deviation matrix. During the analysis process, a block-adaptive threshold is applied, and a wider tolerance is set for blocks with high reflectivity materials.

[0032] The dual deviation matrix transmission adopts a difference encoding technology. The texture deviation matrix and the reflection deviation matrix are packaged and transmitted, and the data packet includes the matrix header information: runway partition version number, acquisition timestamp, and device serial number. The matrix data is losslessly compressed to reduce bandwidth occupation, and the LZW algorithm is used to achieve a compression rate of more than 65%. The transmission interval is fixed at 250 milliseconds, and the retransmission mechanism is triggered in abnormal situations to ensure data integrity. The receiving end caches the deviation matrices of the last three periods for input preparation of the trend analysis module. All matrices are attached with check codes to ensure data reliability, and the previous period data is temporarily used for processing when the check fails.

[0033] The unit implements a fault monitoring mechanism. The self-diagnosis module continuously monitors deviation calculation anomalies: when the mutation rate of consecutive grid points in the texture deviation matrix exceeds 25%, a sensor cleaning warning is triggered; when the reflection deviation is continuously below the detection threshold, the infrared sensor gain setting is automatically adjusted. The processing delay is maintained within 70 milliseconds, and the timeout triggers an acceleration strategy - reducing the feature vector dimension or reducing the spatial sampling rate. Historical deviation data is retained for a 24-hour period to track trends in state changes during the detection process.

[0034] Embodiment 3: Referring to Figure 4 The multi-level detection model configuration unit initializes the stage from the persistent storage to load the pre-trained foreign object detection neural network. The network uses a U-Net architecture topology, and the encoder contains eight levels of down-sampling modules, each containing two layers of convolution operations and a maximum pooling layer; the decoder realizes up-sampling through transposed convolution and cascades with the corresponding encoding layer features. The initial model parameters come from the training results of ten thousand airport scene samples, and the model file format is the Open Neural Network Exchange format. The parameter adaptation sub-unit dynamically adjusts the model parameters to adapt to the current feature environment. This sub-unit receives the block optical reflection feature set transmitted by the reference feature library construction unit, extracts the time series statistics of the feature set as the basis for adjustment. The adjustment process is executed in steps: first, the initial four layers of convolution kernel parameters are frozen to maintain the general feature extraction capability; then the weight update algorithm is applied to the first layer of the decoder, and the convolution kernel gain coefficient is scaled based on the variance data of the block texture features; finally, the output layer is added with a material feature coding embedded vector, which is projected to the feature space through a fully connected layer.

[0035] The feature fusion sub-unit processes dual-source data from the feature deviation analysis unit. The input data includes the texture deviation matrix and the reflection deviation matrix, both of which are single-precision floating-point two-dimensional arrays. The fusion operation is expanded in four-dimensional space: the first dimension indexes the spatial position of different blocks, the second dimension represents the image row coordinate, the third dimension represents the column coordinate, and the fourth dimension is the deviation feature channel. The texture deviation matrix is assigned to channel 0-5 to store 6 groups of texture statistics; the reflection deviation matrix occupies channel 6-11 to record the spectral difference indicators. The splicing operation satisfies the tensor alignment rule, when the input matrix size is MxN, the output tensor size is MxNx12. The output tensor adds device number and acquisition timestamp metadata.

[0036] The model inference subunit performs deep processing of tensor data. The adjusted U-Net network receives the feature fusion tensor as input, and the first layer of the encoding path uses an expanded convolution kernel to expand the receptive field, with a convolution kernel size of 5x5 and an expansion rate of 2. Each convolution layer is followed by a batch normalization layer and a parametric rectified linear unit activation function. The skip connection operation is implemented at the third, fifth, and seventh levels, and the intermediate features of the encoder are concatenated with the up-sampled features in the channel dimension. The output layer applies a spatial pyramid pooling module to capture multi-scale context information through multiple parallel atrous convolutions. The final output is a foreign object probability heat map in the form of a 32-bit floating-point matrix, with the same size as the original input image, and each pixel value in the range [0, 1] representing the probability density of the presence of a foreign object.

[0037] The model integration control subunit manages multi-zone joint inference. This module collects variance distribution data for each zone feature set in the reference feature library construction unit, establishing a zone feature stability quantization index. The system maintains a variance distribution matrix S, with matrix elements representing the variance value of the th feature channel in the th zone, with a 24-hour statistical window for calculation period. The weight allocation subunit calculates the zone confidence based on the variance matrix: Here represents the confidence weight of the th zone, is the total number of feature channels, is the feature variance value. The calculation result is normalized to make the total zone weight sum equal to 1, and a lower weight is assigned to high-variance zones to suppress the influence of unstable regions.

[0038] The decision fusion subunit receives the foreign object probability heat maps from all zones. The input heat map set is spatially calibrated and stitched into a complete runway coverage map based on the zone division coordinates. The fusion process uses a weighted average strategy: the global probability value of each pixel is equal to the heat map value of its zone multiplied by the confidence weight of that zone . The fusion result generates a global detection probability map in the form of a floating-point two-dimensional matrix, with a spatial resolution that maintains a 1:100 scale mapping relationship with the actual size of the runway. The fusion output includes zone boundary marker data for subsequent precise positioning processing.

[0039] The data flow management module optimizes processing efficiency. The hardware accelerator adopts a mixed precision calculation mode: forward inference uses half-precision floating-point numbers to improve throughput, and weight update retains single-precision accuracy. Memory allocation implements a dynamic scheduling strategy: when multiple blocks are processed concurrently, independent video memory space is allocated; when a single block is processed, computing resources are reused. Inference delay control is within a 50-millisecond period, and a model simplification mechanism is started in timeout scenarios, skipping the pyramid pooling module to reduce computational complexity. All processing results are marked with a version number and a time identifier and transmitted to the alarm decision unit via a high-speed serial bus.

[0040] Embodiment 4: refer to Figure 5 The alarm decision unit receives the global detection result data packet transmitted by the multi-level detection model configuration unit, and the data packet structure includes three core fields: the timestamp field records the accurate time of data collection, in the format of UTC time millisecond count; the probability heat map field stores a two-dimensional floating-point array, each element representing the probability of foreign matter existing at the corresponding runway coordinate point; and the block identification field records the block number to which each pixel in the heat map belongs. The unit also obtains the texture deviation matrix and the reflection deviation matrix sent by the feature deviation analysis unit, and the matrix data is attached with spatial calibration parameters to ensure strict alignment with the heat map coordinate system. After receiving the data, the unit first performs format verification, checks the continuity of the timestamp, the matching of the matrix dimensions, and the validity of the numerical values, and when abnormalities are found, the valid frames in the last three frames of data are called for interpolation compensation.

[0041] The foreign matter positioning operation is based on a gradient analysis method. The unit first performs binary processing on the areas in the probability heat map that exceed the threshold value of 0.85 to generate a potential foreign matter mask. For each connected domain, the corresponding area data of the texture deviation matrix and the reflection deviation matrix are extracted. The spatial gradient calculation uses the central difference method to calculate the change rate of the deviation value along the longitudinal and transverse directions of the runway, and identifies the gradient mutation points as foreign matter boundary candidate points. The boundary point set is fitted into a polygonal region through a convex hull algorithm, and the center of gravity coordinates of the polygon are calculated as the center position of the foreign matter. The positioning result includes the following attributes: the geographic coordinates are expressed in the airport local coordinate system with a precision of centimeters; the size feature records the length and width of the circumscribed rectangle of the polygon; and the direction angle describes the included angle between the main extension direction of the foreign matter and the runway center line.

[0042] The threat level assessment module analyzes the multi-dimensional indicators of foreign object features. The assessment indicator system includes two types of data: static attributes and dynamic trends. Static attributes are directly extracted from positioning results, including foreign object projection area, distance from runway centerline, material reflection characteristics, etc. Dynamic trends are obtained by analyzing data changes in five consecutive detection cycles, including area expansion rate, position movement vector, probability value growth slope, etc. The evaluation rule base adopts a hierarchical decision tree structure, with the first layer determining whether the foreign object is in a critical area (such as the area 300 meters before the runway entrance), the second layer analyzing whether the foreign object size exceeds the safety threshold, and the last layer adjusting the level by synthesizing dynamic change trends. The threat level is divided into three levels: level one represents high-risk foreign objects that need to be removed immediately, level two represents medium-risk foreign objects that need to be monitored, and level three represents low-risk foreign objects that can be temporarily handled.

[0043] The time series verification sub-unit maintains a sliding time window data. The window default span is set to 30 seconds, storing the historical records of all detection results within the period. The sub-unit implements a state machine model to manage the life cycle of foreign objects, defining three basic states: initial detection, continuous tracking, and disappearance confirmation. When a foreign object first appears, a tracking instance is created, a unique identifier is assigned, and a feature vector is initialized. The state duration judgment sub-unit monitors the threat level change curve, and when the level is raised for three consecutive periods and the last level reaches two or more, the emergency alarm protocol is triggered. The false alarm filtering mechanism is designed for transient anomalies. If a foreign object appears in a single period but the probability value of the heat map decays to 0.2 or below in subsequent periods, it is marked as environmental noise and not reported.

[0044] The alarm protocol execution phase selects a response strategy according to the threat level. Level one alarm triggers a full system emergency response, including sending a red alert signal to the air traffic control tower, activating the runway light alarm system, and automatically generating a NOTAM navigation notice. Level two alarm starts the inspection plan, sends a work order containing the foreign object coordinates to the field service department, and requires on-site verification within 30 minutes. Level three alarm only records to the runway maintenance log for regular analysis. All alarm events generate structured reports, including the following key fields: event number uses a combination of date and time + block number encoding; positioning data records the center coordinates and boundary vertex set of the foreign object; time series data saves the probability value and feature deviation statistics of the last six periods; the processing state field tracks the response progress of the alarm.

[0045] The following table shows the alarm decision process data record for a typical foreign object event: The system implements multiple safeguard mechanisms to ensure the reliability of the decision. The data consistency check module compares the temporal and spatial characteristics of the heat map and the deviation matrix. When there is a significant contradiction, the manual review process is started. The clock synchronization system keeps the time deviation of all units less than 5 milliseconds to avoid timing analysis errors. The historical case library stores the processing records of typical foreign matter events in the past three years, providing similarity reference for current decision. The unit running state real-time monitoring interface displays the load of each link of the processing pipeline, including data receiving rate, calculation delay, storage throughput and other performance indicators. Abnormal conditions automatically trigger the degradation processing mode. All output data is written into the airport central database at the same time, and is backed up to the off-site disaster recovery center with a backup period of 15 minutes incremental update.

[0046] In embodiment 5, the data management unit continuously receives the spatiotemporal distribution data set output by the feature deviation analysis unit. This data set contains the time series change record of the texture deviation matrix and the reflection deviation matrix, and each record is attached with a collection timestamp and a block position identifier. The spatiotemporal distribution data is organized in a three-dimensional structure: the first dimension represents the time axis, with a new data point generated every 250 milliseconds; the second dimension stores the runway lateral coordinate index; and the third dimension records the longitudinal coordinate index. The matrix data value is stored in single-precision floating-point format, and each element represents the deviation intensity of the corresponding spatiotemporal position. The system generates a unique hash check code for each frame of data, which is calculated based on the value range characteristics of the matrix elements and used for subsequent data integrity verification.

[0047] The storage optimization sub-unit processes the original input data stream. This sub-unit parses the three-dimensional structure of the spatiotemporal distribution data and extracts the spatial distribution pattern of each time slice. The unit generates two-level storage indexes: the time index establishes a directory structure by minute, storing all time slice data of each minute; and the space index establishes a mapping table of block coordinates and physical storage locations. The index file uses a tree structure to accelerate query, and the leaf node records the physical sector address of the data file in the storage array. The storage location selection algorithm is optimized based on access frequency: recent data is stored in the high-speed area of solid state disk, and historical data is migrated to large-capacity mechanical hard disk array. Index update and data write operations are atomic, and after power failure recovery, the index structure can be reconstructed through transaction logs.

[0048] The data compression subunit monitors the statistical characteristics of the spatio-temporal distribution data. The system maintains a global variance analyzer, which calculates the variance sequence of the texture deviation matrix and the reflection deviation matrix every five minutes. The variance calculation range includes all data points in the last three detection periods, and the rolling variance value is output for compression decision. When the rolling variance of the texture deviation matrix is continuously lower than the compression threshold of 0.015 and the rolling variance of the reflection deviation matrix is lower than 0.02, the unit activates the lossy compression process. The compression process is executed on a dedicated coprocessor, using an improved JPEG2000 algorithm: first, normalize the floating-point matrix to the 0-255 integer interval, perform discrete wavelet transform to generate multi-level sub-band coefficients; retain all information of the low-frequency sub-band, and implement quantization processing based on human visual characteristics for the high-frequency sub-band; finally, generate a compressed code stream through the MQ encoder. The compression ratio is controlled between 5:1 and 8:1, and the compressed data is added with a marker header to record the original matrix range and quantization parameters.

[0049] The storage space allocation subunit dynamically regulates the data retention period. This unit obtains the alarm event log generated by the alarm decision unit through the system bus, and the log includes alarm level and trigger time information. The allocation algorithm maintains an alarm frequency counter: when the number of first-level alarm triggers reaches 5 times within a 6-hour statistical window, the storage period of the corresponding block historical data is automatically extended to 72 hours; the no-alarm area defaults to a 24-hour storage period; the second-level alarm area adopts an intermediate value of 48 hours. The data cleaning task is executed every day at 4:00, and the expired data is marked according to the storage period configuration. The cleaning process is divided into two stages: in the first stage, the expired data is moved to the buffer pool and marked as a to-be-deleted state; in the second stage, physical erasure is performed when the system load is less than 30%. The cleaning task generates an operation report, recording the released storage space size and the deleted data time range.

[0050] The storage resource management module coordinates the hardware infrastructure. The system manages multiple storage volumes: the high-performance storage volume uses an NVMe solid state disk array with a capacity of 8TB, responsible for storing near-24-hour raw data; the archival storage volume uses a RAID6 mechanical hard disk group with a total capacity of 192TB, storing compressed historical data. The volume management program monitors the usage of each storage volume, and when the main volume space usage exceeds 85%, it starts automatic data migration: selects the earliest time period of inactive data, performs compression processing, and transfers to the archival volume. The archival data index file is stored independently, supporting fast retrieval by time range and block coordinates. All storage devices are configured with dual power redundancy, and the standby power supply supports 10-minute data cache transfer in case of abnormal power failure.

[0051] Fault-tolerant mechanism guarantees data security. Unit implements real-time data backup channel: synchronously writes each frame of data to main storage array and backup server during normal operation. Backup adopts differential transmission mode, only sends incremental change part. Data recovery function is automatically activated when detecting check failure: locates damaged data position through hash code comparison, retrieves intact copy from backup server for repair. If backup data is also unavailable, starts data reconstruction process: linear interpolation reconstruction based on previous and next time frame data. Unit maintains operation status dashboard, continuously displays key indicators such as storage space usage, data compression ratio, backup synchronization delay, etc. Timestamp calibration system automatically synchronizes with airport reference clock every day, ensures that all recorded time deviations are less than 10 milliseconds.

[0052] It should be noted that the relational terms herein such as first and second and the like are used solely to distinguish one from another entity or action without necessarily requiring or implying that there is any such relationship or order between such entities or actions. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0053] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are only by way of example and that changes can be made to the embodiments without departing from the spirit and the scope of the application, which is defined in the following claims and their equivalents.

Claims

1. An image recognition-based runway foreign object detection system, characterized by, The execution unit comprises: a runway block division unit, configured to divide a runway surface into a plurality of independent detection blocks based on a predefined runway zoning rule; a reference feature library construction unit, configured to store a reference texture feature set and an optical reflection feature set of each detection block in a foreign matter-free state; a real-time image analysis unit, configured to collect a real-time image sequence of a current detection block by a multispectral imaging device deployed around the runway, and extract a real-time texture feature vector and a reflection feature vector therefrom; a feature deviation analysis unit, configured to dynamically compare the real-time texture feature vector with the reference texture feature set to generate a texture deviation matrix, and compare the real-time reflection feature vector with the optical reflection feature set to generate a reflection deviation matrix; an alarm decision unit, configured to generate foreign matter positioning coordinates and alarm level instructions based on a joint analysis result of the texture deviation matrix and the reflection deviation matrix.

2. The runway foreign object detection system based on image recognition according to claim 1, characterized in that, The runway block division unit performs the following operations: obtain actual physical size data and surface material distribution data of the runway; divide the runway into equal-area grid blocks according to a predefined block side length threshold with the center line of the runway as a reference axis; assign a corresponding material feature code to each grid block according to the surface material type of the grid block; bind the material feature code and the block coordinates and transmit them to the reference feature library construction unit.

3. The runway foreign object detection system based on image recognition according to claim 2, characterized in that, The reference feature library construction unit performs the following operations: receive the material feature code and the block coordinates from the runway block division unit; collect a reference image set of each block in an interference-free state by the multispectral imaging device within a predefined reference collection period; perform multi-scale feature decomposition on the reference image set of each block to extract spatial domain texture features and frequency domain reflection features thereof, respectively; store the spatial domain texture features as a reference texture feature set and store the frequency domain reflection features as an optical reflection feature set.

4. The runway foreign object detection system based on image recognition according to claim 3, characterized in that, The real-time image analysis unit comprises: an image preprocessing subunit, configured to perform illumination equalization and motion blur correction on the collected real-time image sequence; a feature mapping subunit, configured to extract a multi-level texture feature mapping graph from the corrected image by a deep convolutional network; a reflection analysis subunit, configured to analyze infrared spectral band data in the image to generate a reflection intensity distribution histogram; a coordinate correlation subunit, configured to align the multi-level texture feature mapping graph and the reflection intensity distribution histogram according to pixel coordinates to generate a real-time feature vector set.

5. The image recognition based runway foreign object detection system of claim 4, wherein, The feature deviation analysis unit performs the following operations: receive the reference texture feature set and the optical reflection feature set from the reference feature library construction unit; obtain the real-time feature vector set generated by the real-time image analysis unit; calculate a Euclidean distance matrix of the real-time texture feature vector and the corresponding block reference texture feature set as a texture deviation matrix; calculate a spectral angle difference value of the real-time reflection feature vector and the corresponding block optical reflection feature set as a reflection deviation matrix; transmit the texture deviation matrix and the reflection deviation matrix to the alarm decision unit.

6. The foreign object detection system on runway based on image recognition according to claim 1, characterized in that, Further comprising a multi-level detection model configuration unit, which comprises: a model initialization subunit, configured to load a pre-trained foreign matter detection neural network model; A parameter adaptation subunit is configured to dynamically adjust model convolution kernel weights according to a reference feature set from the reference feature library construction subunit. A feature fusion subunit is configured to perform tensor splicing on the texture deviation matrix and the reflection deviation matrix from the feature deviation analysis subunit. A model inference subunit is configured to generate a foreign object probability heat map by processing the spliced tensor through the adjusted foreign object detection neural network.

7. The image recognition based runway foreign object detection system of claim 6, wherein, The multi-level detection model configuration subunit further includes: A model integration control subunit is configured to obtain variance distribution data of each block feature set in the reference feature library construction subunit. A weight distribution subunit is configured to calculate confidence weights of each block detection model according to the variance distribution data. A decision fusion subunit is configured to perform weighted fusion on foreign object probability heat maps from different blocks to generate a global detection result.

8. The image recognition based runway foreign object detection system of claim 7, wherein, The alarm decision unit performs the following operations: Receives the global detection result from the multi-level detection model configuration subunit; Obtains the texture deviation matrix and the reflection deviation matrix transmitted by the feature deviation analysis subunit; When the foreign object probability in the global detection result exceeds a predefined threshold, locates the foreign object center coordinates according to the spatial distribution gradient of the texture deviation matrix and the reflection deviation matrix; Calculates the foreign object threat level based on the deviation amplitude of the pixels around the foreign object center coordinates; Triggers an alarm protocol of a corresponding level according to the threat level.

9. The image recognition based runway foreign object detection system of claim 8, wherein, The alarm decision unit further includes: A time series verification subunit is configured to continuously obtain global detection results of multiple detection periods; A state duration judgment subunit is configured to start an emergency alarm instruction when the foreign object threat level maintains an upward trend within a predefined duration; A false alarm filtering subunit is configured to ignore alarm triggering when the foreign object threat level exceeds a threshold in a single period but disappears in the next period.

10. The foreign object detection system on a runway based on image recognition according to claim 1, characterized in that, Further includes a data management unit, which includes: A storage optimization subunit is configured to obtain spatiotemporal distribution data of the texture deviation matrix and the reflection deviation matrix generated by the feature deviation analysis subunit; A data compression subunit is configured to start lossy compression of the feature matrix when the variance of the spatiotemporal distribution data continuously falls below a predefined compression threshold; A storage space allocation subunit is configured to dynamically adjust the historical data storage period according to the alarm frequency generated by the alarm decision unit.

Images