An airport clearance obstacle identification method based on multi-source fusion perception

Abstract

The application discloses an airport clearance obstacle identification method based on multi-source fusion perception, comprising: multi-source perception data acquisition and preprocessing, generating a clearance perception data set; spatial registration and coordinate conversion, constructing a multi-source fusion spatial data field, generating a multi-modal fusion feature representation; constructing a clearance domain relationship graph, performing graph structure iterative updating, separating fixed and dynamic obstacles, and generating a candidate obstacle set; improved RF-DETR network identification, dynamic scale query and height regression, generating an obstacle identification result set; constructing an obstacle space influence domain, performing extension, fusion and dynamic correction, and generating a clearance risk judgment result; time series correlation and trend analysis, outputting a dynamic risk state and early warning result. The application realizes accurate identification, spatial positioning and dynamic risk early warning of obstacles in the airport clearance area through multi-source fusion perception and improved RF-DETR network.

Description

Technical Field

[0001] This invention relates to the field of airport airspace safety monitoring technology, and in particular to a method for identifying airport airspace obstacles based on multi-source fusion perception. Background Technology

[0002] With the continuous expansion of civil aviation transport and the increasing density of airport operations, airport airspace safety has gradually become a crucial foundation for ensuring flight safety. Current airport airspace obstacle identification primarily relies on manual inspections, single video surveillance, or radar monitoring to periodically check surrounding buildings, cranes, trees, and temporary construction facilities. While some technical solutions incorporate image recognition or radar detection for automatic target identification and location, they still largely rely on single data sources, lacking multi-dimensional information support, and exhibit significant shortcomings in stability and accuracy under complex environmental conditions.

[0003] In practical applications, due to the presence of various types, scales, and dynamic changes of obstacles within the airport airspace, a single sensing method is insufficient to fully reflect the spatial structural characteristics of the target. Especially in rainy or foggy weather, low-light environments, or long-distance monitoring scenarios, image data is easily interfered with, radar data resolution is limited, and point cloud data exhibits sparsity. This leads to problems such as temporal asynchrony, inconsistent spatial coordinates, and inconsistent feature representations between different data sources, making it difficult to accurately identify and continuously track small, low-altitude, or temporary obstacles, thus affecting the overall detection performance.

[0004] Existing technologies, after completing obstacle identification, mostly only make static judgments on the target's location or height, lacking a comprehensive analysis of the obstacle's spatial occupancy relationship and changing trends. This makes it difficult to accurately assess the dynamic impact of obstacles on airspace safety. The spatial relationships between different obstacles and the risk propagation process have not been effectively modeled, resulting in lag and discontinuity in risk assessment results, making it difficult to support the real-time monitoring and early warning needs of airport airspace safety.

[0005] Therefore, how to provide a method for airport airspace obstacle identification based on multi-source fusion perception is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an airport airspace obstacle identification method based on multi-source fusion perception. This invention fully utilizes the collaborative fusion capabilities of image data, point cloud data, radar data, infrared data, and geographic elevation data. By constructing a multi-source fusion spatial data field, an airspace occupancy relationship map, and an improved RF-DETR network, it achieves accurate identification, spatial positioning, and height determination of obstacles within the airport airspace area. Furthermore, it completes dynamic risk identification and early warning output through obstacle spatial influence domain analysis. It has the advantages of high identification accuracy, strong resistance to environmental interference, strong ability to identify small and dynamic targets, and high timeliness of risk assessment.

[0007] An airport airspace obstacle identification method based on multi-source fusion sensing according to an embodiment of the present invention includes:

[0008] Collect multi-source sensing data within the airport's airspace, perform preprocessing on the multi-source sensing data, and generate an airspace sensing data set;

[0009] Spatial registration and coordinate transformation are performed on the airspace perception data set to map different data sources to a unified spatial reference coordinate system, construct a multi-source fusion spatial data field, and feature alignment and fusion encoding are performed on each data source to generate a multi-modal fusion feature representation.

[0010] Based on the multi-source fusion spatial data field, a clearance domain relationship graph is constructed, the graph structure is iteratively updated, the target domain boundary and domain evolution state are identified, fixed obstacles and dynamic obstacles are separated, and a candidate obstacle set is generated.

[0011] The fused features corresponding to the candidate obstacle set are input into the improved RF-DETR network. The improved RF-DETR network sets up an image-point cloud cross-modal feature interaction layer and an elevation prior embedding layer in parallel at the encoding end, and sets up a small target dynamic scale query module and a height joint regression head at the decoding end. It performs category recognition, bounding box regression, instance segmentation and height refinement on each candidate target to generate an obstacle recognition result set.

[0012] Based on the obstacle identification result set, an obstacle spatial influence domain is constructed. The spatial influence boundary is calculated according to the three-dimensional coordinates, height information, occupied area and historical change trajectory of each obstacle. Influence domain expansion, overlap fusion and dynamic correction processing are performed to identify the encroachment status, risk propagation direction and influence range of each obstacle and generate the airspace risk assessment result.

[0013] Perform time series correlation and trend analysis on the airspace risk assessment results, output the dynamic risk status, warning level and spatial distribution information of obstacles, and generate airport airspace obstacle identification and warning results.

[0014] Optionally, the multi-source sensing data includes image data, point cloud data, radar data, infrared data, positioning data, and geographic elevation data.

[0015] Optionally, the preprocessing of the multi-source sensing data to generate the airspace sensing data set includes performing time synchronization, coordinate unification, anomaly removal, and data resampling on the multi-source sensing data to generate a unified time series airspace sensing data set.

[0016] Optionally, the generation of multimodal fusion feature representation includes:

[0017] Read image data, point cloud data, radar data, infrared data, positioning data and geographic elevation data from the airspace perception data set, align the data collected at the same time according to the timestamp, remove data frames whose time offset exceeds the preset range, and generate a time-aligned airspace perception data set.

[0018] Using positioning data and geographic elevation data as spatial benchmarks, a unified spatial reference coordinate system corresponding to the airport's airspace is established. The collection location, installation attitude, and spatial resolution of each data source are read, and image data, point cloud data, radar data, and infrared data are converted to the unified spatial reference coordinate system to generate unified spatial coordinate data.

[0019] Based on unified spatial coordinate data, the airport airspace is divided into spatial grid units. Image texture information, point cloud geometric information, radar echo information, infrared radiation information and geographic elevation information within the same spatial grid unit are matched to generate multi-source matching data corresponding to the spatial grid.

[0020] Feature alignment processing is performed on multi-source matching data. Image texture information, point cloud geometric information, radar echo information, infrared radiation information and geographic elevation information are arranged in the same spatial grid cell in sequence. Missing data source locations are marked and filled in to generate spatial grid fusion features.

[0021] According to the arrangement of spatial grid cells in a unified spatial reference coordinate system, the fusion features of each spatial grid are sequentially written into the corresponding spatial location to form a multi-source fused spatial data field containing location, height, geometry, texture, echo, radiation and elevation information. The multi-source fused spatial data field is then subjected to fusion encoding processing to generate a multi-modal fused feature representation.

[0022] Optionally, generating the candidate obstacle set includes:

[0023] Read the multi-source fused spatial data field and multi-modal fused feature representation, scan each spatial grid cell item by item according to the arrangement position of the spatial grid cells, extract the position, height, geometry, texture, echo, radiation and elevation information in each spatial grid cell, and mark the spatial grid cells with target response as occupied grid cells;

[0024] Spatial continuity detection is performed on the occupied grid cells. The occupied grid cells that are spatially adjacent, have continuous feature changes, and have consistent height distribution are merged into initial occupied cells. The center position, outer contour, maximum height, minimum elevation, occupied area, and fusion features of each initial occupied cell are read to generate an initial set of spatial targets.

[0025] Using each spatial target in the initial set of spatial targets as a graph node, and the proximity connection, elevation correlation and motion continuity relationship between spatial targets as graph edges, the spatial position, outline range, height distribution and fusion characteristics of each graph node are recorded, and the proximity direction, elevation difference and temporal displacement state of each graph edge are recorded to construct a net space occupation relationship graph.

[0026] According to the continuous acquisition time, the map structure of the airspace occupation relationship map is iteratively updated. Newly emerging occupation grid cells are incorporated into the corresponding map nodes, and disappearing or separated occupation grid cells are removed from the corresponding map nodes. The target occupation boundary and occupation evolution state are updated according to the contour changes, height changes and position changes of the map nodes.

[0027] Based on the updated target territory boundary and territory evolution state, graph nodes are judged as fixed or dynamic. Graph nodes with stable territory boundaries and position changes below a preset range are marked as fixed obstacles, while graph nodes with continuously moving territory boundaries or continuously changing height states are marked as dynamic obstacles. The spatial position, outline boundary, initial height, and territory range of fixed and dynamic obstacles are summarized to generate a candidate obstacle set.

[0028] Optionally, the generated obstacle recognition result set includes:

[0029] Read the candidate obstacle set and multimodal fusion feature representation, and extract the image texture features, point cloud geometric features, radar echo features, infrared radiation features and geographic elevation features of the corresponding area from the multi-source fusion spatial data field according to the spatial location, contour boundary, initial height and occupied area of ​​each candidate obstacle to generate candidate obstacle fusion feature blocks.

[0030] The candidate obstacle fusion feature block is input into the encoder of the improved RF-DETR network. The point cloud spatial coordinates and image pixel positions corresponding to the candidate obstacle are read. The correspondence between the point cloud spatial position and the image pixel region is established based on the unified spatial reference coordinate system, and the image-point cloud correspondence index is generated.

[0031] In the image-point cloud cross-modal feature interaction layer, the image texture region adjacent to each point cloud position is extracted according to the corresponding index of the image-point cloud, and the point cloud height, point cloud density, point cloud edge and point cloud surface undulation information corresponding to each image texture region are extracted to generate the corresponding feature group of image local features and point cloud local features.

[0032] A bidirectional feature update process is performed on the corresponding feature group. The edge, color and contour information in the image texture region is used to correct the boundary of the point cloud target. The vertical structure of the image target is supplemented by the point cloud height, spatial depth and surface undulation information. The updated image features and point cloud features are aligned by channel and position to generate image-point cloud interactive enhancement features.

[0033] Image-point cloud interactive enhancement features, radar echo features, infrared radiation features, and geographic elevation features are input into the elevation prior embedding layer. Based on the initial height, occupied area, and geographic elevation information of the candidate obstacle, height position embedding, ground elevation correction, and vertical structure enhancement processing are performed to generate candidate obstacle coding features carrying clearance height information.

[0034] The encoded features of candidate obstacles are input into the decoder of the improved RF-DETR network. The small target dynamic scale query module generates target queries based on the outline boundary, occupied area and feature response intensity of the candidate obstacles. The joint height regression head performs category recognition, bounding box regression, instance segmentation and height refinement, and outputs obstacle category labels, two-dimensional bounding boxes, instance segmentation contours, confidence scores, accurate height and three-dimensional coordinate information to generate an obstacle recognition result set.

[0035] Optionally, the generation of the net airspace risk assessment result includes:

[0036] Read the obstacle category labels, two-dimensional bounding boxes, instance segmentation contours, confidence scores, precise heights, and three-dimensional coordinate information from the obstacle recognition result set, and combine them with the occupied area and historical change trajectory in the candidate obstacle set to establish an obstacle spatial state record according to the obstacle number;

[0037] Based on the obstacle spatial state record, the three-dimensional coordinates, height information, occupied area and historical change trajectory of each obstacle are read to determine the occupied boundary of each obstacle in the horizontal position, vertical height and movement direction, and the occupied boundary is extended to the surrounding space to generate the initial spatial influence domain corresponding to each obstacle.

[0038] Based on the category label, precise height, occupied area, historical change trajectory and confidence level of each obstacle, the initial spatial influence domain is expanded. The influence domains corresponding to obstacles with continuously increasing height, continuously approaching the airspace protection area, and continuously increasing occupied area are expanded outward. The influence domains corresponding to obstacles with insufficient confidence or unstable boundaries are conservatively expanded to generate the expanded obstacle spatial influence domain.

[0039] Overlap detection is performed on the expanded obstacle spatial influence domain. Multiple obstacle spatial influence domains that are spatially adjacent, have overlapping influence ranges, or have the same historical change direction are overlapped and merged. The spatial influence boundary is updated according to the merged outer boundary, maximum height, maximum occupied area, and main change direction to generate the merged spatial influence domain.

[0040] Based on the fusion of spatial influence domains, the positional relationships of each obstacle relative to the safety height, horizontal protection range, and entry / exit direction protection range are read. Combined with historical change trajectories, dynamic correction processing is performed to identify the intrusion status, risk propagation direction, and impact range of each obstacle. The airspace risk assessment results are generated according to the obstacle number, intrusion status, risk propagation direction, impact range, and risk level.

[0041] Optionally, the generation of airport airspace obstacle identification and early warning results includes:

[0042] Read the obstacle number, intrusion status, risk propagation direction, impact range, risk level and corresponding timestamp from the airspace risk assessment results, and merge the airspace risk assessment results collected continuously according to the obstacle number to generate an obstacle risk time sequence record.

[0043] Based on the obstacle risk time series record, the intrusion status, impact range and risk level of the same obstacle at consecutive collection times are read in chronological order, and the risk changes at adjacent times are compared to generate an obstacle risk change sequence.

[0044] Based on the obstacle risk change sequence, the risk change trend of obstacles is identified. Obstacles whose impact range continues to expand, whose risk level continues to rise, or whose risk propagation direction continues to approach the airspace protection area are marked as risk rising. Obstacles whose impact range is stable and whose risk level does not change are marked as risk holding. Obstacles whose impact range shrinks or whose risk level decreases are marked as risk weakening. This generates the dynamic risk status of obstacles.

[0045] The warning level is determined based on the obstacle's dynamic risk status, intrusion status, and risk level. The obstacle's dynamic risk status, warning level, risk propagation direction, and impact range are mapped to the corresponding spatial location in a unified spatial reference coordinate system to generate spatial distribution information.

[0046] Based on the obstacle number, the system summarizes the obstacle category label, three-dimensional coordinate information, precise height, intrusion status, obstacle dynamic risk status, warning level, and spatial distribution information to generate airport airspace obstacle identification and warning results.

[0047] The beneficial effects of this invention are:

[0048] This invention constructs a multi-source fusion spatial data field, aligning and fusing heterogeneous data from multiple sources such as images, point clouds, radar, infrared, and geographic elevation under a unified spatial reference coordinate system. This solves the problems of asynchronous time, inconsistent space, and scattered feature representation of multi-source data in existing technologies, enabling effective complementarity between different sensing methods. It improves the completeness of obstacle perception and the consistency of spatial representation under complex environmental conditions, and enhances the detection capability of distant, small-scale, and low-contrast targets.

[0049] This invention introduces a clearance-occupancy relationship diagram and an improved RF-DETR network to jointly model the spatial structural features, occupancy boundaries, and evolutionary states of obstacles. This enables a shift from single-target identification to spatial relationship identification, stably separating fixed and dynamic obstacles in continuous time-series data, and accurately classifying, locating, and estimating the height of targets. This improves the accuracy and stability of obstacle identification and enhances the ability to identify temporary obstacles and dynamic targets.

[0050] This invention constructs an obstacle spatial influence domain and combines it with time series change trends to dynamically analyze the obstacle's intrusion status, risk propagation direction, and impact range. This overcomes the problems of existing technologies that only perform static judgments and lack continuous risk assessments, making the airspace risk judgment results continuous and forward-looking. It can achieve early identification and warning of potential risks, effectively improving the real-time performance and reliability of airport airspace safety management. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0052] Figure 1 This is a flowchart of an airport airspace obstacle identification method based on multi-source fusion perception proposed in this invention;

[0053] Figure 2 This is a schematic diagram of the improved RF-DETR network structure and image-point cloud cross-modal feature interaction process of an airport airspace obstacle recognition method based on multi-source fusion perception proposed in this invention. Detailed Implementation

[0054] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0055] refer to Figure 1 and Figure 2 An airport airspace obstacle identification method based on multi-source fusion perception includes:

[0056] Collect multi-source sensing data within the airport's airspace, perform preprocessing on the multi-source sensing data, and generate an airspace sensing data set;

[0057] Spatial registration and coordinate transformation are performed on the airspace perception data set to map different data sources to a unified spatial reference coordinate system, construct a multi-source fusion spatial data field, and feature alignment and fusion encoding are performed on each data source to generate a multi-modal fusion feature representation.

[0058] Based on the multi-source fusion spatial data field, a clearance domain relationship graph is constructed, the graph structure is iteratively updated, the target domain boundary and domain evolution state are identified, fixed obstacles and dynamic obstacles are separated, and a candidate obstacle set is generated.

[0059] The fused features corresponding to the candidate obstacle set are input into the improved RF-DETR network. The improved RF-DETR network sets up an image-point cloud cross-modal feature interaction layer and an elevation prior embedding layer in parallel at the encoding end, and sets up a small target dynamic scale query module and a height joint regression head at the decoding end. It performs category recognition, bounding box regression, instance segmentation and height refinement on each candidate target to generate an obstacle recognition result set.

[0060] Based on the obstacle identification result set, an obstacle spatial influence domain is constructed. The spatial influence boundary is calculated according to the three-dimensional coordinates, height information, occupied area and historical change trajectory of each obstacle. Influence domain expansion, overlap fusion and dynamic correction processing are performed to identify the encroachment status, risk propagation direction and influence range of each obstacle and generate the airspace risk assessment result.

[0061] Perform time series correlation and trend analysis on the airspace risk assessment results, output the dynamic risk status, warning level and spatial distribution information of obstacles, and generate airport airspace obstacle identification and warning results.

[0062] In this embodiment, the multi-source sensing data includes image data, point cloud data, radar data, infrared data, positioning data, and geographic elevation data.

[0063] In this embodiment, the preprocessing of multi-source sensing data to generate a clearance sensing data set includes performing time synchronization, coordinate unification, anomaly removal, and data resampling on the multi-source sensing data to generate a clearance sensing data set with a unified time series.

[0064] In this embodiment, the generation of multimodal fusion feature representation includes:

[0065] The system reads image data, point cloud data, radar data, infrared data, positioning data, and geographic elevation data from the airspace perception data set. It aligns the data collected at the same time according to the timestamp, removes data frames whose time offset exceeds the preset range, and generates a time-aligned airspace perception data set. The preset range is that when the timestamp deviation between different data sources exceeds 50 milliseconds, it is judged as misaligned and removed.

[0066] Using positioning data and geographic elevation data as spatial benchmarks, a unified spatial reference coordinate system corresponding to the airport's airspace is established. The collection location, installation attitude, and spatial resolution of each data source are read, and image data, point cloud data, radar data, and infrared data are converted to the unified spatial reference coordinate system to generate unified spatial coordinate data.

[0067] Based on unified spatial coordinate data, the airport airspace is divided into spatial grid units. Image texture information, point cloud geometric information, radar echo information, infrared radiation information, and geographic elevation information within the same spatial grid unit are matched to generate multi-source matching data corresponding to the spatial grid. Specifically, the generation of multi-source matching data corresponding to the spatial grid involves:

[0068] Within each spatial grid cell, the center coordinates, horizontal and vertical boundary lengths are extracted, and the following summary features are calculated: an image texture feature vector composed of the image texture mean, texture variance, and principal direction gradient; a point cloud geometric feature vector composed of point cloud density, average height, and curvature; a radar echo feature vector composed of the radar echo amplitude mean and Doppler velocity mean; an infrared radiation feature vector composed of the infrared radiation average temperature and maximum temperature difference; and an elevation feature vector composed of the geographic elevation average and range. These vectors are then concatenated in a uniform order, and the grid cell index number in a unified spatial reference coordinate system is added to form spatial grid matching data entries. All data entries corresponding to all grid cells constitute the multi-source matching data corresponding to the spatial grid.

[0069] Feature alignment processing is performed on the multi-source matching data. Image texture information, point cloud geometric information, radar echo information, infrared radiation information, and geographic elevation information are arranged sequentially according to the same spatial grid cell. Missing data source locations are marked and filled in to generate spatial grid fusion features. Specifically, the generation of spatial grid fusion features is as follows:

[0070] Within each spatial grid cell, image texture feature vectors, point cloud geometric feature vectors, radar echo feature vectors, infrared radiation feature vectors, and geographic elevation feature vectors are written sequentially. Zero-value placeholders are inserted at missing data source locations, and missing identifier bits are added. All written vectors are padded to a uniform length to ensure consistent length. The padded vectors are then concatenated sequentially according to fixed channel numbers. Spatial grid index, center coordinates, and boundary size information are added before the concatenation result. The number of valid data sources within the grid cell is appended after the concatenation result, and the output serves as the fusion feature entry for the spatial grid cell.

[0071] Based on the arrangement of spatial grid cells in a unified spatial reference coordinate system, the fusion features of each spatial grid are sequentially written into their corresponding spatial locations, forming a multi-source fused spatial data field containing location, height, geometry, texture, echo, radiation, and elevation information. Fusion encoding processing is then performed on the multi-source fused spatial data field to generate a multimodal fused feature representation. Specifically, the multi-source fused spatial data field containing location, height, geometry, texture, echo, radiation, and elevation information is as follows:

[0072] Following the order of the east, north, and vertical axes of the unified spatial reference coordinate system, starting from the lowest elevation layer, all spatial grid cells are traversed sequentially. The aforementioned fused feature entries are written to the corresponding three-dimensional array positions according to the grid column index, row index, and layer index. During the writing process, the spatial grid center coordinates are used as the physical reference for the row, column, and layer indices. The position coordinates are written to the spatial index channel, the maximum and average heights within the grid are written to the height channel, the point cloud geometric features are written to the geometry channel, the image texture features are written to the texture channel, the radar echo features are written to the echo channel, the infrared radiation features are written to the radiation channel, and the geographic elevation features are written to the elevation channel. If data in a certain channel is missing, a zero value is written and the missing flag is maintained. All channel data are kept aligned under the same index, ultimately forming a multi-source fused spatial data field with three-dimensional coordinates as the index and multi-channel vectors as the content.

[0073] In this embodiment, generating the candidate obstacle set includes:

[0074] Read the multi-source fused spatial data field and multi-modal fused feature representation, scan each spatial grid cell item by item according to the arrangement position of the spatial grid cells, extract the position, height, geometry, texture, echo, radiation and elevation information in each spatial grid cell, and mark the spatial grid cells with target response as occupied grid cells;

[0075] Spatial continuity detection is performed on the occupied grid cells. The occupied grid cells that are spatially adjacent, have continuous feature changes, and have consistent height distribution are merged into initial occupied cells. The center position, outer contour, maximum height, minimum elevation, occupied area, and fusion features of each initial occupied cell are read to generate an initial set of spatial targets.

[0076] Using each spatial target in the initial set of spatial targets as a graph node, and the proximity connections, elevation relationships, and motion continuity relationships between spatial targets as graph edges, the spatial location, contour range, height distribution, and fusion characteristics of each graph node are recorded. The adjacent directions, elevation differences, and temporal displacement states corresponding to each graph edge are also recorded to construct a clearance-occupancy relationship graph. Specifically, the construction of the clearance-occupancy relationship graph is as follows:

[0077] Each spatial target in the initial spatial target set is assigned a unique number, and a node attribute table is established. The node attribute table is written sequentially with the node number, 3D center coordinates, circumscribed contour length, width, and height, highest and lowest heights, fused feature vector, and the most recent observation timestamp. For any two nodes, their horizontal center distance, overlap area ratio, and height difference are compared using thresholds. When the horizontal center distance is less than the proximity threshold and the overlap area ratio is higher than the domain threshold, a proximity connection edge is generated between the two nodes. When the horizontal projections of the two nodes overlap and the height difference is lower than the elevation threshold, an elevation association edge is generated between the two nodes. The position difference of the same target at adjacent timestamps in consecutive observation frames is read. When the position difference direction is consistent and the displacement amplitude is lower than the displacement threshold, a motion continuous edge is generated between the nodes corresponding to adjacent timestamps. For each graph edge, the proximity direction vector, elevation difference, and time-series displacement vector are written, and the corresponding adjacency relationship and attribute index are constructed in the graph structure. Finally, a net domain relationship graph containing the node attribute set and the edge attribute set is formed.

[0078] According to the continuous acquisition time, the map structure of the airspace occupation relationship map is iteratively updated. Newly emerging occupation grid cells are incorporated into the corresponding map nodes, and disappearing or separated occupation grid cells are removed from the corresponding map nodes. The target occupation boundary and occupation evolution state are updated according to the contour changes, height changes and position changes of the map nodes.

[0079] Based on the updated target territory boundary and territory evolution state, graph nodes are judged as fixed or dynamic. Graph nodes with stable territory boundaries and position changes below a preset range are marked as fixed obstacles, while graph nodes with continuously moving territory boundaries or continuously changing height states are marked as dynamic obstacles. The spatial position, outline boundary, initial height, and territory range of fixed and dynamic obstacles are summarized to generate a set of candidate obstacles. The preset range is that within a continuous observation window of 3 frames or 1 second, the cumulative displacement of the horizontal center coordinate of the obstacle does not exceed 0.5 meters and the change in the height of the highest point does not exceed 0.3 meters.

[0080] In this embodiment, the generation of the obstacle recognition result set includes:

[0081] Read the candidate obstacle set and multimodal fusion feature representation. Based on the spatial location, contour boundary, initial height, and occupied area of ​​each candidate obstacle, extract the image texture features, point cloud geometric features, radar echo features, infrared radiation features, and geographic elevation features of the corresponding region from the multi-source fused spatial data field to generate candidate obstacle fusion feature blocks. Specifically, the generation of candidate obstacle fusion feature blocks is as follows:

[0082] For each candidate obstacle, the 3D center coordinates, horizontal bounding rectangle boundary, maximum height, and occupied area are read. The corresponding spatial grid index range is located in the multi-source fused spatial data field. The interception window is determined by expanding one grid cell outward from the horizontal boundary. Within the interception window, the data vectors of the image texture channel, point cloud geometry channel, radar echo channel, infrared radiation channel, and geographic elevation channel of all grid cells are extracted, maintaining a one-to-one correspondence between each channel in the spatial index. For missing channels, zero-value placeholders and missing identifier bits are written. After performing intra-channel normalization on the extracted channel data, they are arranged in the order of image texture, point cloud geometry, radar echo, infrared radiation, and geographic elevation. The arranged multi-channel data is flattened in row priority order. The candidate obstacle number, center coordinates, and maximum height are added to the front, and the occupied area and channel missing identifier are added to the back. The corresponding candidate obstacle fusion feature is then generated by splicing.

[0083] The candidate obstacle fusion feature block is input into the encoder of the improved RF-DETR network. The point cloud spatial coordinates and image pixel positions corresponding to the candidate obstacles are read. The correspondence between the point cloud spatial positions and image pixel regions is established based on a unified spatial reference coordinate system, generating an image-point cloud correspondence index. Specifically, the generation of the image-point cloud correspondence index is as follows:

[0084] The algorithm reads the set of 3D point cloud coordinates of candidate obstacles in a unified spatial reference coordinate system and the corresponding 2D image projection field parameters. For each point cloud coordinate, it uses the camera extrinsic parameter matrix to retrieve the number of the imaging sensor it belongs to, and organizes the point cloud coordinates according to the sensor number. For point cloud coordinates within the same sensor number group, it calls the sensor's intrinsic parameter matrix to map the 3D point cloud coordinates to the 2D pixel coordinates of the image, obtaining the pixel row and column index corresponding to each point cloud coordinate. It performs a boundary validity check on the mapped pixel row and column index, removes index records that fall outside the valid area of ​​the image, and records the original corresponding point cloud identifier. It forms an index tuple with the valid pixel row and column index and its corresponding point cloud identifier, sorts it according to the sensor number-pixel row-pixel column as the joint primary key, and writes it into the index table. The index table also writes the row and column channel offset of the pixel position in the multimodal fusion feature. Finally, it outputs the image-point cloud corresponding index.

[0085] In the image-point cloud cross-modal feature interaction layer, image texture regions adjacent to each point cloud location are extracted according to the corresponding index of the image and point cloud. The point cloud height, density, edge, and surface undulation information corresponding to each image texture region are also extracted. This generates corresponding feature sets between image local features and point cloud local features. Specifically, the generation of these corresponding feature sets involves:

[0086] The image-point cloud correspondence index table is traversed sequentially. For each index record, its pixel row and column coordinates and corresponding point cloud identifier are read. Centered on the pixel row and column coordinates, a local texture region is extracted from the image texture channel according to a preset window size. The average brightness, mean color value, gray-level co-occurrence matrix energy, principal direction gradient magnitude, and local edge density of the local texture region are calculated and combined to form an image local texture feature vector. In the point cloud geometry channel, all point cloud coordinates adjacent to the pixel row and column coordinate mapping are retrieved. The average height, maximum height, point cloud density, normal vector variance, and local curvature change rate of the adjacent point clouds are calculated and combined to form a point cloud local geometric feature vector. The image local texture feature vector and the point cloud local geometric feature vector are concatenated in a fixed order, and the pixel row and column coordinates, point cloud identifier, and sensor number of the index record are appended to generate an image-point cloud corresponding feature entry. After processing all index records, they are written to the feature cache sequentially according to the sensor number and pixel row and column order. Finally, the corresponding feature groups of image local features and point cloud local features are obtained.

[0087] A bidirectional feature update process is performed on the corresponding feature group. Edge, color, and contour information from the image texture region are used to correct the point cloud target boundary. Point cloud height, spatial depth, and surface undulation information are used to supplement the vertical structure of the image target. The updated image features and point cloud features are then aligned by channel and position to generate image-point cloud interactive enhancement features. Specifically, the generation of image-point cloud interactive enhancement features involves:

[0088] Each image-point cloud feature entry in the corresponding feature group is read sequentially. Using the pixel row and column coordinates in the entry as the center, the maximum gradient direction and salient color distribution are located within the local texture feature vector of the image. These are then projected onto the local geometric feature vector of the point cloud corresponding to the same row and column coordinates. The horizontal coordinates of the boundary points in the local geometric features of the point cloud are adjusted according to the maximum gradient direction, and the vertical coordinates of the boundary points in the local geometric features of the point cloud are adjusted according to the edge contraction rate of the salient color distribution. Subsequently, the average height, maximum height, and local curvature change rate in the local geometric feature vector of the point cloud are read. The average height is written into the vertical depth channel of the local texture feature vector of the image, and the maximum height is written into the image... The top edge channel of the local texture feature vector is used to write the local curvature change rate into the surface undulation channel of the local texture feature vector of the image. After bidirectional update, the updated local texture feature vector of the image and the local geometric feature vector of the point cloud are aligned in channel order, with the image channel first and the point cloud channel second. After alignment, the position index is rewritten according to the pixel row and column coordinates so that the two types of features maintain a one-to-one correspondence under the same channel number. The aligned multi-channel feature vector is written into the corresponding position of the channel stacking tensor in the order of row priority and column priority. The mapping table between the candidate obstacle number and the channel starting offset is maintained to complete the generation of image-point cloud interactive enhancement features.

[0089] Image-point cloud interactive enhancement features, radar echo features, infrared radiation features, and geographic elevation features are input into the elevation prior embedding layer. Based on the initial height, occupied area, and geographic elevation information of the candidate obstacles, elevation location embedding, ground elevation correction, and vertical structure enhancement processing are performed to generate candidate obstacle coding features carrying clearance height information. Specifically, the generated candidate obstacle coding features carrying clearance height information are as follows:

[0090] The initial offset of the image-point cloud interactive enhancement feature tensor in the channel stacking tensor is read to extract the complete multi-channel feature block of the corresponding candidate obstacle. In the elevation prior embedding layer, the initial height value, occupied area, and geographic elevation value of the candidate obstacle are read respectively. The initial height value is mapped to a height position vector, the geographic elevation value is mapped to a ground elevation vector, and the occupied area is mapped to a vertical structure scale vector. The height position vector is written into the height position channel of the multi-channel feature block, the ground elevation vector is written into the ground reference channel of the multi-channel feature block, and the vertical structure scale vector is written into the vertical structure channel of the multi-channel feature block. In the elevation prior embedding layer, the vertical structure enhancement module is called to perform joint convolution on the elevation position channel, ground reference channel and vertical structure channel in the multi-channel feature block to enhance the coupling relationship between elevation difference, relative height and vertical scale. After the vertical structure enhancement is completed, the enhanced elevation-related channel is re-stitched with the original image-point cloud channel, radar echo channel, infrared radiation channel and geographic elevation channel in a fixed order. The feature content of the corresponding region of the candidate obstacle in the channel stacking tensor is updated, and the new channel length offset is recorded in the mapping table to generate candidate obstacle coding features carrying clearance height information.

[0091] The encoded features of candidate obstacles are input into the decoder of the improved RF-DETR network. A small-target dynamic scale query module generates target queries based on the outline boundaries, occupied areas, and feature response intensity of the candidate obstacles. A joint height regression head then performs category recognition, bounding box regression, instance segmentation, and height refinement, outputting obstacle category labels, 2D bounding boxes, instance segmentation contours, confidence scores, precise heights, and 3D coordinate information, generating an obstacle recognition result set.

[0092] The small target dynamic scale query module generates target queries based on the contour boundaries, occupied areas, and feature response intensity of candidate obstacles. Specifically:

[0093] Read the outline bounding length, width, and height, occupied area, and maximum feature response from the candidate obstacle's encoded features. Map the length, width, and height and the occupied area to scale factors, and map the maximum feature response to weight factors. Based on the scale factors, retrieve the three closest scale templates from a pre-defined query scale library. Perform a weighted average on each retrieved scale template according to the weight factors to generate a scale adaptation weight vector. Use the scale adaptation weight vector to adjust the baseline query vector of the corresponding scale in the query vector pool. Write the spatial position embedding item in the query vector into the center coordinates of the candidate obstacle's encoded features, and write the height position embedding item in the query vector into the precise initial height value of the encoded features. Add the occupied area number and channel offset identifier to the adjusted query vector to form a target query, and write it into the target query queue in the order of candidate obstacle numbers.

[0094] The system performs category recognition, bounding box regression, instance segmentation, and height refinement using a highly joint regression head.

[0095] The system receives query vectors from the target query queue, performs interactive attention with candidate obstacle encoding features in the channel dimension, inputs the interactive attention results into the category recognition branch, and outputs obstacle category labels and confidence scores based on the category probability distribution of the query vector. It then inputs the interactive attention results into the bounding box regression branch, regresses the two-dimensional bounding box coordinates according to the four-dimensional parameters of center coordinates, width, and height, and appends the bounding box confidence score to the output. In parallel, it inputs the interactive attention results into the instance segmentation branch, outputting instance segmentation contours through feature upsampling and channel masking. Simultaneously, it inputs the interactive attention results into the height refinement branch, reads the initial height written in the query vector and the height-related channels in the encoding features, performs vertical error correction, ground elevation correction, and multi-frame height fusion, and outputs precise height values. Finally, in the output convergence unit, it integrates category labels, two-dimensional bounding boxes, instance segmentation contours, confidence scores, precise heights, and three-dimensional coordinate information according to the candidate obstacle number, and writes them into the obstacle recognition result set.

[0096] This invention structurally optimizes the RF-DETR network, proposing an improved RF-DETR network for airport airspace obstacle recognition scenarios. An image-point cloud cross-modal feature interaction layer is introduced at the encoding end. Based on the point cloud spatial coordinates of candidate obstacles in a unified spatial reference coordinate system and the image pixel positions, a correspondence index is established between the image texture region and the point cloud geometric region. Edge, color, and contour information in the image are bidirectionally updated with height, depth, density, and surface undulation information in the point cloud. This enables the network to simultaneously perceive the two-dimensional appearance features and three-dimensional spatial structure features of obstacles, effectively enhancing the model's ability to recognize small obstacles, low-contrast obstacles, and obstacles with blurred boundaries.

[0097] To address the high accuracy requirements for determining the height of obstacles in airport airspace, this invention further incorporates an elevation prior embedding layer at the encoding stage. This layer incorporates the initial height, occupied area, and geographic elevation information of candidate obstacles into their encoding features. Ground elevation correction and vertical structure enhancement are then performed, enabling the network to combine ground reference height, the highest point of the target, and the vertical occupied area simultaneously for feature representation during target detection, rather than relying solely on image texture or planar boundaries. This processing allows the model to more accurately distinguish between ground background, low-lying targets, and tall obstacles posing an airspace risk, thus improving the accuracy of obstacle height estimation and airspace risk assessment.

[0098] To enhance the model's adaptability to distant small targets and temporary dynamic obstacles, this invention incorporates a small target dynamic scale query module and a height joint regression head at the decoding end. The small target dynamic scale query module generates target queries at corresponding scales based on the contour boundaries, occupied areas, and feature response intensity of candidate obstacles, enabling the decoding process to adaptively adjust the scope of interest for obstacles of different sizes. The height joint regression head simultaneously performs category recognition, bounding box regression, instance segmentation, and height refinement, ensuring that obstacle category, 2D contour, instance boundary, precise height, and 3D coordinate information are jointly output during the same decoding process. This invention not only retains the advantages of end-to-end target detection using RF-DETR networks but also achieves deep fusion of obstacle planar recognition, spatial localization, and height refinement in airport airspace scenarios, improving the completeness, stability, and reliability of obstacle recognition results for risk warning.

[0099] In this embodiment, generating the airspace risk assessment result includes:

[0100] Read the obstacle category labels, two-dimensional bounding boxes, instance segmentation contours, confidence scores, precise heights, and three-dimensional coordinate information from the obstacle recognition result set, and combine them with the occupied area and historical change trajectory in the candidate obstacle set to establish an obstacle spatial state record according to the obstacle number;

[0101] Based on the obstacle spatial state record, the three-dimensional coordinates, height information, occupied area, and historical change trajectory of each obstacle are read. The occupied boundaries of each obstacle in the horizontal position, vertical height, and direction of movement are determined, and the occupied boundaries are extended to the surrounding space to generate the initial spatial influence domain corresponding to each obstacle. Specifically, the generation of the initial spatial influence domain corresponding to each obstacle is as follows:

[0102] For each obstacle, the 3D center coordinates, horizontal bounding rectangle dimensions, highest point height, lowest point height, and historical motion trajectory are read. Based on the horizontal bounding rectangle boundary, the obstacle's occupancy extremes in the east and north directions are determined. These extremes are then expanded outward by 10% of the obstacle's width to obtain the horizontal influence radius. The vertical occupancy interval is calculated based on the difference between the highest and lowest point heights. Vertically, this difference is expanded upward by 5% and downward by 0.2 meters from the ground reference to cover height deviations caused by measurement errors and ground undulations. Then, based on the historical motion trajectory, the average motion direction vector and maximum displacement amplitude of the obstacle in the most recent three frames are obtained. The motion direction vector is simultaneously expanded outward by 50% of the maximum displacement amplitude in both the horizontal and vertical directions to form a motion redundancy zone. Finally, the expanded horizontal influence radius, vertical occupancy interval, and motion redundancy zone are subjected to a 3D volume union operation to generate a 3D polyhedron structure containing the location, horizontal expansion boundary, vertical expansion interval, and motion redundancy zone, which serves as the initial spatial influence domain of the obstacle.

[0103] Based on the category label, precise height, occupied area, historical change trajectory, and confidence level of each obstacle, the initial spatial influence domain is expanded. The influence domains corresponding to obstacles whose height continuously increases, whose location continuously approaches the airspace protection area, and whose occupied area continuously expand are expanded outwards. The influence domains corresponding to obstacles with insufficient confidence or unstable boundaries are conservatively expanded, generating an expanded obstacle spatial influence domain. Specifically, the generated expanded obstacle spatial influence domain is as follows:

[0104] Read the horizontal expansion boundary, vertical expansion range, and motion redundancy zone of the initial spatial influence domain. For obstacles with the "tall" category label (such as tower cranes, high-mast lights, or building structures) and whose height has been continuously increasing in the last three frames, increase the horizontal expansion boundary by 15% in both the east and north directions, and raise the vertical expansion range by 0.5 meters. For obstacles with dynamic categories such as drones and suspended platform vehicles whose historical motion trajectories continuously point towards the center of the airspace protection area, extend the horizontal expansion boundary along the direction of motion by 80% of the maximum displacement amplitude, and add a warning buffer layer of equal width to the extension distance at the leading edge of the motion redundancy zone. For obstacles whose area has increased by more than 10% in the last three frames, the horizontal expansion boundary will be proportionally increased by the same factor, while no additional adjustment will be made to the vertical expansion range. When the confidence level is below 0.6 or the instance segmentation contour fluctuates continuously, causing boundary instability, the initial horizontal expansion boundary will remain unchanged, and only a vertical expansion redundancy of 0.2 meters will be added upwards and marked as a conservative expansion state. After the classification expansion is completed, the horizontal expansion boundary, vertical expansion range and motion redundancy zone of all obstacles will be merged and updated to generate an expanded obstacle spatial influence domain that includes expansion parameters, expansion strategy identifiers and initial risk level information.

[0105] Overlap detection is performed on the expanded obstacle spatial influence domain. Multiple obstacle spatial influence domains that are spatially adjacent, have overlapping influence ranges, or have the same historical change direction are overlapped and merged. The spatial influence boundary is then updated according to the merged outer boundary, maximum height, maximum occupied area, and main change direction to generate a merged spatial influence domain. Specifically, the generation of the merged spatial influence domain is as follows:

[0106] Based on the three-dimensional outer boundary of the expanded obstacle space influence domain, the overlap relationship between any two domains is detected using an axis-aligned bounding box method. If two domains overlap in the horizontal outer boundary and their vertical outer expansion intervals overlap each other, or if the angle between the motion redundancy zones of the two domains is less than 20 degrees, they are determined to be fusionable objects. For objects determined to be fusionable, the minimum and maximum coordinates of their horizontal outer boundary in the east and north directions are taken as new horizontal outer boundaries, and the lowest and highest points of their vertical outer expansion intervals are taken as new vertical outer expansion intervals. The motion redundancy zone direction vectors of the two domains are merged, and the direction with the larger amplitude is taken as the main change direction. If a third domain is found to overlap with the already merged domain during the fusion process, the iterative fusion is continued according to the aforementioned rules until there is no overlap within the same group of domains. For each group of domains that have been fused, the updated outer boundary size, maximum height, maximum occupied area, and main change direction are calculated and written into a new influence domain attribute table. At the same time, the original domain numbers and fusion order of the domains participating in the fusion are recorded. Finally, all the domains that have been fused and the individual domains without overlap are summarized to generate a fused space influence domain.

[0107] Based on the fusion of spatial influence domains, the positional relationships of each obstacle relative to the safety height, horizontal protection range, and entry / exit direction protection range are read. Combined with historical change trajectories, dynamic correction processing is performed to identify the intrusion status, risk propagation direction, and impact range of each obstacle. The airspace risk assessment results are generated according to the obstacle number, intrusion status, risk propagation direction, impact range, and risk level.

[0108] In this embodiment, generating airport airspace obstacle identification and early warning results includes:

[0109] Read the obstacle number, intrusion status, risk propagation direction, impact range, risk level and corresponding timestamp from the airspace risk assessment results, and merge the airspace risk assessment results collected continuously according to the obstacle number to generate an obstacle risk time sequence record.

[0110] Based on the obstacle risk time series record, the intrusion status, impact range and risk level of the same obstacle at consecutive collection times are read in chronological order, and the risk changes at adjacent times are compared to generate an obstacle risk change sequence.

[0111] Based on the obstacle risk change sequence, the risk change trend of obstacles is identified. Obstacles whose impact range continues to expand, whose risk level continues to rise, or whose risk propagation direction continues to approach the airspace protection area are marked as risk rising. Obstacles whose impact range is stable and whose risk level does not change are marked as risk holding. Obstacles whose impact range shrinks or whose risk level decreases are marked as risk weakening. This generates the dynamic risk status of obstacles.

[0112] The warning level is determined based on the obstacle's dynamic risk status, intrusion status, and risk level. The obstacle's dynamic risk status, warning level, risk propagation direction, and impact range are mapped to the corresponding spatial location in a unified spatial reference coordinate system to generate spatial distribution information.

[0113] Based on the obstacle number, the system summarizes the obstacle category label, three-dimensional coordinate information, precise height, intrusion status, obstacle dynamic risk status, warning level, and spatial distribution information to generate airport airspace obstacle identification and warning results.

[0114] Example 1: During a continuous airport airspace monitoring cycle, the system received multi-source sensing data from the airport airspace. The raw data included 7200 frames of image data with a resolution of 1920×1080 pixels; 2880 frames of point cloud data with an average of 186,000 points per frame; 5760 frames of radar data with an average of 36 echo targets per frame; 2880 frames of infrared data with a resolution of 640×512 pixels; 7200 location data entries; and geographic elevation data covering an area of ​​3600m×2400m. During this monitoring cycle, there were construction equipment, trees, high-mast lights, temporary scaffolding, engineering vehicle booms, and low-altitude dynamic targets outside the perimeter fence. Among these, 11 targets were similar in color to the background in the images, 6 targets were partially obscured by tree canopies, and 3 dynamic targets showed significant height changes in consecutive frames.

[0115] After the data enters the processing flow, the system first reads the timestamps of image data, point cloud data, radar data, infrared data, positioning data, and geographic elevation data. Using the acquisition time of the image data as a benchmark, the point cloud, radar, and infrared data are matched to adjacent timestamps, with a preset time offset range of ±50ms. In the original multi-source data, a total of 1482 data frames had time offsets, of which 1186 offsets were less than 50ms and were aligned, while 296 offsets exceeding 50ms were discarded. After processing, a set of 6904 time-aligned clearance sensing data sets is formed, with the average time offset reduced from 63.4ms before processing to 18.7ms.

[0116] After completing the temporal alignment, the system establishes a unified spatial reference coordinate system using positioning data and geographic elevation data. The airport airspace is divided into spatial grid cells of 1m horizontally and 0.5m vertically, resulting in 8.64 million spatial grid cells. The system then writes image texture information, point cloud geometric information, radar echo information, infrared radiation information, and geographic elevation information into the corresponding grid cells one by one. For example, in a certain spatial grid cell, the image texture mean is 0.46, the average point cloud height is 12.8m, and the point cloud density is 42 points / m². 3The radar echo intensity was 0.71, the average infrared temperature was 31.6℃, and the geographic elevation was 4.2m. For missing data source locations, the system wrote zero-value placeholders and added missing flag bits. After processing, 6.883 million multi-source matching data records were generated, of which complete five-source matching data accounted for 82.6%, four-source matching data accounted for 13.9%, and three-source or lower matching data accounted for 3.5%.

[0117] When forming spatial grid fusion features, the system arranges them in the following order: image texture features, point cloud geometric features, radar echo features, infrared radiation features, and geographic elevation features. Taking the grid containing a high-mast lamp target as an example, its fusion feature entries include spatial grid index, center coordinates, boundary dimensions, maximum height of 15.4m, point cloud edge intensity of 0.68, image edge density of 0.53, radar echo amplitude of 0.62, average infrared radiation of 29.8℃, and elevation of 4.1m. All spatial grid fusion features are written to the corresponding three-dimensional array positions in the order of east, north, and vertical, ultimately forming a multi-source fused spatial data field. In this spatial data field, there are 146,312 effective target response grids and 8,493,688 background grids, with the target response grids accounting for 1.69%.

[0118] The system then scans the multi-source fused spatial data field item by item, marking grids with continuous height response, edge response, and echo response as occupied grid cells. After spatial continuity detection, 146,312 target response grids are merged into 214 initial occupied cells. The system assigns a node number to each initial spatial target and writes its 3D center coordinates, circumscribed contour length, width, and height, highest height, lowest height, and fused feature vector. For any two nodes, if the horizontal center distance is less than 5m, the horizontal projection overlap ratio is greater than 0.25, or the height difference is less than 1.2m, a neighboring connection edge or an elevation-related edge is generated; if the position difference direction is consistent in consecutive frames and the displacement amplitude is less than 0.5m, a motion continuous edge is generated. The final net space occupied relationship graph contains 214 graph nodes and 367 graph edges. After three frame iterations, the system marks targets with stable domain boundaries, cumulative horizontal center coordinate displacement not exceeding 0.5m, and highest point height change not exceeding 0.3m as fixed obstacles, identifying a total of 153 fixed obstacles; targets with continuously moving domain boundaries or continuously changing heights are marked as dynamic obstacles, identifying a total of 27 dynamic obstacles; the remaining 34 targets are judged as background interference and removed, generating 180 candidate obstacles.

[0119] The training phase utilized manually verified multi-source samples, totaling 28,600 training samples, 6,400 validation samples, and 6,000 test samples. Among the training samples, 4,200 were fixed buildings (average height 9.8m); 3,600 were high-mast lights (average height 14.6m); 5,100 were trees (average height 7.2m); 3,300 were tower cranes and booms (average height 21.4m); 2,900 were engineering vehicle lifting booms (average height 6.5m); 3,800 were temporary scaffolding (average height 8.7m); 2,500 were small dynamic targets (average height 3.2m); and 3,200 were background non-obstacle targets. Each sample included a category label, a 2D bounding box, an instance segmentation profile, 3D coordinates, maximum height, occupied area, and risk status.

[0120] In the process of improving the RF-DETR network recognition, the system extracts candidate obstacle fusion feature blocks from 180 candidate obstacles. Taking an engineering vehicle lifting arm as an example, its horizontal outer boundary is 4.8m × 2.1m, and its maximum height increases from 5.9m to 6.8m, with the occupied area shifting 1.42m in the approach direction. The system generates an image-point cloud correspondence index based on the point cloud 3D coordinates and image pixel positions, with 1846 valid index records. The image-point cloud cross-modal feature interaction layer reads each index record, extracts local texture regions in the image texture channel, and extracts average height, maximum height, point cloud density, normal vector variance, and local curvature change rate in the point cloud geometry channel. Before the interaction, the average boundary response of the target image was 0.54, and the average boundary response of the point cloud was 0.61; after the bidirectional feature update, the image boundary response improved to 0.78, the point cloud boundary response improved to 0.81, and the target contour edge break length decreased from 1.9m to 0.4m.

[0121] In the elevation prior embedding layer, the system reads the initial height, occupied area, and geographic elevation information of candidate obstacles, and performs height position embedding and ground elevation correction on the lifting arm of the engineering vehicle. Before correction, the estimated height of the highest point was 6.42m, which differed from the manually verified height of 6.83m by 0.41m; after correction, the height of the highest point was 6.76m, and the error was reduced to 0.07m. For long-distance high-mast lighting targets, traditional image recognition yielded a height of 13.2m, while manual verification yielded a height of 14.7m, with an error of 1.5m; this invention, after image-point cloud interaction and elevation prior embedding, outputs a height of 14.4m, with an error of 0.3m.

[0122] After entering the decoding stage, the small target dynamic scale query module generates a target query based on the candidate obstacle's outline boundary, occupied area, and feature response intensity. For an obstacle with an occupied area of ​​only 1.6m... 2For small, dynamic targets, the maximum feature response value is 0.74. The system selects three near-scale templates from the query scale library and generates a target query. After decoding, the output class confidence score is 0.87, the 2D bounding box error is 0.18m, and the 3D center coordinate error is 0.36m. The height joint regression head simultaneously completes class identification, bounding box regression, instance segmentation, and height refinement. Among 180 candidate obstacles, 174 were ultimately confirmed as valid obstacles, and 6 false positives were removed.

[0123] During the risk assessment phase, the system established spatial status records for 174 obstacles and generated initial spatial influence domains. For fixed obstacles, the horizontal outer boundary was expanded by 10% of the obstacle width, and vertically by 5% of the height difference upwards and 0.2m downwards. For dynamic targets, the system expanded the maximum displacement amplitude by 50% along the average movement direction of the most recent three frames. For towering targets with continuously increasing height, the system increased the horizontal boundary width by 15% each in the east and north directions, and increased the vertical expansion range upwards by 0.5m. For dynamic targets moving towards the airspace protection zone, the maximum displacement amplitude was further extended by 80% along the movement direction. In this monitoring, the system identified 9 high-risk targets, 21 medium-risk targets, and 144 low-risk targets, among which 3 dynamic targets were warned an average of 5.8 seconds before entering the manually set risk boundary.

[0124] In the comparative experiment, the overall obstacle recognition accuracy of the traditional single-video method on 6000 test samples was 86.4%, the traditional video plus radar method was 90.7%, and the present invention was 96.8%; the recall rates for small obstacles were 71.6%, 78.9%, and 91.5%, respectively; the recognition accuracy for dynamic obstacles was 76.8%, 84.2%, and 94.1%, respectively; the accuracy rates for distinguishing between fixed and dynamic obstacles were 79.3%, 85.6%, and 95.7%, respectively; the average errors of the three-dimensional center coordinates were 1.86m, 1.21m, and 0.43m, respectively; the average errors of the highest point height were 1.42m, 0.96m, and 0.31m, respectively; the false negative rates in complex environments were 12.8%, 8.7%, and 3.1%, respectively; and the average processing time per single target was 0.82s, 0.64s, and 0.41s, respectively.

[0125] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for airport airspace obstacle identification based on multi-source fusion perception, characterized in that, include: Collect multi-source sensing data within the airport's airspace, perform preprocessing on the multi-source sensing data, and generate an airspace sensing data set; Spatial registration and coordinate transformation are performed on the airspace perception data set to map different data sources to a unified spatial reference coordinate system, construct a multi-source fusion spatial data field, and feature alignment and fusion encoding are performed on each data source to generate a multi-modal fusion feature representation. Based on the multi-source fusion spatial data field, a clearance domain relationship graph is constructed, the graph structure is iteratively updated, the target domain boundary and domain evolution state are identified, fixed obstacles and dynamic obstacles are separated, and a candidate obstacle set is generated. The fused features corresponding to the candidate obstacle set are input into the improved RF-DETR network. The improved RF-DETR network sets up an image-point cloud cross-modal feature interaction layer and an elevation prior embedding layer in parallel at the encoding end, and sets up a small target dynamic scale query module and a height joint regression head at the decoding end. It performs category recognition, bounding box regression, instance segmentation and height refinement on each candidate target to generate an obstacle recognition result set. Based on the obstacle identification result set, an obstacle spatial influence domain is constructed. The spatial influence boundary is calculated according to the three-dimensional coordinates, height information, occupied area and historical change trajectory of each obstacle. Influence domain expansion, overlap fusion and dynamic correction processing are performed to identify the encroachment status, risk propagation direction and influence range of each obstacle and generate the airspace risk assessment result. Perform time series correlation and trend analysis on the airspace risk assessment results, output the dynamic risk status, warning level and spatial distribution information of obstacles, and generate airport airspace obstacle identification and warning results.

2. The airport airspace obstacle identification method based on multi-source fusion perception according to claim 1, characterized in that, The multi-source sensing data includes image data, point cloud data, radar data, infrared data, positioning data, and geographic elevation data.

3. The airport airspace obstacle identification method based on multi-source fusion perception according to claim 1, characterized in that, The preprocessing of multi-source sensing data to generate a clearance sensing data set includes performing time synchronization, coordinate unification, anomaly removal, and data resampling on the multi-source sensing data to generate a clearance sensing data set with a unified time series.

4. The airport airspace obstacle identification method based on multi-source fusion perception according to claim 1, characterized in that, The generated multimodal fusion feature representation includes: Read image data, point cloud data, radar data, infrared data, positioning data and geographic elevation data from the airspace perception data set, align the data collected at the same time according to the timestamp, remove data frames whose time offset exceeds the preset range, and generate a time-aligned airspace perception data set. Using positioning data and geographic elevation data as spatial benchmarks, a unified spatial reference coordinate system corresponding to the airport's airspace is established. The collection location, installation attitude, and spatial resolution of each data source are read, and image data, point cloud data, radar data, and infrared data are converted to the unified spatial reference coordinate system to generate unified spatial coordinate data. Based on unified spatial coordinate data, the airport airspace is divided into spatial grid units. Image texture information, point cloud geometric information, radar echo information, infrared radiation information and geographic elevation information within the same spatial grid unit are matched to generate multi-source matching data corresponding to the spatial grid. Feature alignment processing is performed on multi-source matching data. Image texture information, point cloud geometric information, radar echo information, infrared radiation information and geographic elevation information are arranged in the same spatial grid cell in sequence. Missing data source locations are marked and filled in to generate spatial grid fusion features. According to the arrangement of spatial grid cells in a unified spatial reference coordinate system, the fusion features of each spatial grid are sequentially written into the corresponding spatial location to form a multi-source fused spatial data field containing location, height, geometry, texture, echo, radiation and elevation information. The multi-source fused spatial data field is then subjected to fusion encoding processing to generate a multi-modal fused feature representation.

5. The airport airspace obstacle identification method based on multi-source fusion perception according to claim 1, characterized in that, The generated candidate obstacle set includes: Read the multi-source fused spatial data field and multi-modal fused feature representation, scan each spatial grid cell item by item according to the arrangement position of the spatial grid cells, extract the position, height, geometry, texture, echo, radiation and elevation information in each spatial grid cell, and mark the spatial grid cells with target response as occupied grid cells; Spatial continuity detection is performed on the occupied grid cells. The occupied grid cells that are spatially adjacent, have continuous feature changes, and have consistent height distribution are merged into initial occupied cells. The center position, outer contour, maximum height, minimum elevation, occupied area, and fusion features of each initial occupied cell are read to generate an initial set of spatial targets. Using each spatial target in the initial set of spatial targets as a graph node, and the proximity connection, elevation correlation and motion continuity relationship between spatial targets as graph edges, the spatial position, outline range, height distribution and fusion characteristics of each graph node are recorded, and the proximity direction, elevation difference and temporal displacement state of each graph edge are recorded to construct a net space occupation relationship graph. According to the continuous acquisition time, the map structure of the airspace occupation relationship map is iteratively updated. Newly emerging occupation grid cells are incorporated into the corresponding map nodes, and disappearing or separated occupation grid cells are removed from the corresponding map nodes. The target occupation boundary and occupation evolution state are updated according to the contour changes, height changes and position changes of the map nodes. Based on the updated target territory boundary and territory evolution state, graph nodes are judged as fixed or dynamic. Graph nodes with stable territory boundaries and position changes below a preset range are marked as fixed obstacles, while graph nodes with continuously moving territory boundaries or continuously changing height states are marked as dynamic obstacles. The spatial position, outline boundary, initial height, and territory range of fixed and dynamic obstacles are summarized to generate a candidate obstacle set.

6. The airport airspace obstacle identification method based on multi-source fusion perception according to claim 1, characterized in that, The generated obstacle recognition result set includes: Read the candidate obstacle set and multimodal fusion feature representation, and extract the image texture features, point cloud geometric features, radar echo features, infrared radiation features and geographic elevation features of the corresponding area from the multi-source fusion spatial data field according to the spatial location, contour boundary, initial height and occupied area of ​​each candidate obstacle to generate candidate obstacle fusion feature blocks. The candidate obstacle fusion feature block is input into the encoder of the improved RF-DETR network. The point cloud spatial coordinates and image pixel positions corresponding to the candidate obstacle are read. The correspondence between the point cloud spatial position and the image pixel region is established based on the unified spatial reference coordinate system, and the image-point cloud correspondence index is generated. In the image-point cloud cross-modal feature interaction layer, the image texture region adjacent to each point cloud position is extracted according to the corresponding index of the image-point cloud, and the point cloud height, point cloud density, point cloud edge and point cloud surface undulation information corresponding to each image texture region are extracted to generate the corresponding feature group of image local features and point cloud local features. A bidirectional feature update process is performed on the corresponding feature group. The edge, color and contour information in the image texture region is used to correct the boundary of the point cloud target. The vertical structure of the image target is supplemented by the point cloud height, spatial depth and surface undulation information. The updated image features and point cloud features are aligned by channel and position to generate image-point cloud interactive enhancement features. Image-point cloud interactive enhancement features, radar echo features, infrared radiation features, and geographic elevation features are input into the elevation prior embedding layer. Based on the initial height, occupied area, and geographic elevation information of the candidate obstacle, height position embedding, ground elevation correction, and vertical structure enhancement processing are performed to generate candidate obstacle coding features carrying clearance height information. The encoded features of candidate obstacles are input into the decoder of the improved RF-DETR network. The small target dynamic scale query module generates target queries based on the outline boundary, occupied area and feature response intensity of the candidate obstacles. The joint height regression head performs category recognition, bounding box regression, instance segmentation and height refinement, and outputs obstacle category labels, two-dimensional bounding boxes, instance segmentation contours, confidence scores, accurate height and three-dimensional coordinate information to generate an obstacle recognition result set.

7. The airport airspace obstacle identification method based on multi-source fusion perception according to claim 1, characterized in that, The generation of net airspace risk assessment results includes: Read the obstacle category labels, two-dimensional bounding boxes, instance segmentation contours, confidence scores, precise heights, and three-dimensional coordinate information from the obstacle recognition result set, and combine them with the occupied area and historical change trajectory in the candidate obstacle set to establish an obstacle spatial state record according to the obstacle number; Based on the obstacle spatial state record, the three-dimensional coordinates, height information, occupied area and historical change trajectory of each obstacle are read to determine the occupied boundary of each obstacle in the horizontal position, vertical height and movement direction, and the occupied boundary is extended to the surrounding space to generate the initial spatial influence domain corresponding to each obstacle. Based on the category label, precise height, occupied area, historical change trajectory and confidence level of each obstacle, the initial spatial influence domain is expanded. The influence domains corresponding to obstacles with continuously increasing height, continuously approaching the airspace protection area, and continuously increasing occupied area are expanded outward. The influence domains corresponding to obstacles with insufficient confidence or unstable boundaries are conservatively expanded to generate the expanded obstacle spatial influence domain. Overlap detection is performed on the expanded obstacle spatial influence domain. Multiple obstacle spatial influence domains that are spatially adjacent, have overlapping influence ranges, or have the same historical change direction are overlapped and merged. The spatial influence boundary is updated according to the merged outer boundary, maximum height, maximum occupied area, and main change direction to generate the merged spatial influence domain. Based on the fusion of spatial influence domains, the positional relationships of each obstacle relative to the safety height, horizontal protection range, and entry / exit direction protection range are read. Combined with historical change trajectories, dynamic correction processing is performed to identify the intrusion status, risk propagation direction, and impact range of each obstacle. The airspace risk assessment results are generated according to the obstacle number, intrusion status, risk propagation direction, impact range, and risk level.

8. The airport airspace obstacle identification method based on multi-source fusion perception according to claim 1, characterized in that, The generated airport airspace obstacle identification and early warning results include: Read the obstacle number, intrusion status, risk propagation direction, impact range, risk level and corresponding timestamp from the airspace risk assessment results, and merge the airspace risk assessment results collected continuously according to the obstacle number to generate an obstacle risk time sequence record. Based on the obstacle risk time series record, the intrusion status, impact range and risk level of the same obstacle at consecutive collection times are read in chronological order, and the risk changes at adjacent times are compared to generate an obstacle risk change sequence. Based on the obstacle risk change sequence, the risk change trend of obstacles is identified. Obstacles whose impact range continues to expand, whose risk level continues to rise, or whose risk propagation direction continues to approach the airspace protection area are marked as risk rising. Obstacles whose impact range is stable and whose risk level does not change are marked as risk holding. Obstacles whose impact range shrinks or whose risk level decreases are marked as risk weakening. This generates the dynamic risk status of obstacles. The warning level is determined based on the obstacle's dynamic risk status, intrusion status, and risk level. The obstacle's dynamic risk status, warning level, risk propagation direction, and impact range are mapped to the corresponding spatial location in a unified spatial reference coordinate system to generate spatial distribution information. Based on the obstacle number, the system summarizes the obstacle category label, three-dimensional coordinate information, precise height, intrusion status, obstacle dynamic risk status, warning level, and spatial distribution information to generate airport airspace obstacle identification and warning results.

Images