Airport all-weather bird detecting and repelling method based on multi-mode self-adaptive fusion

Abstract

The invention discloses an airport all-weather bird detecting and repelling method based on multi-modal adaptive fusion, and the method comprises the following steps: obtaining the initial track data of a moving target, and generating a candidate region through an ultra-light model; in response to the candidate area, guiding a non-radar sensor to carry out directional observation so as to obtain high-precision multi-modal observation data; determining fusion weight parameters according to the environment features, and performing weighted fusion on the multi-source data to determine a final target object; and finally generating a disposal instruction based on the security policy. According to the invention, through a hierarchical detection mechanism, the operation load of the system is reduced and the response speed is improved; the recognition accuracy and the all-weather working capability under various weather illumination conditions are ensured through environment self-adaptive weighted fusion; and the operation safety of airport personnel and key facilities is guaranteed through a multi-layer safety verification system penetrating through the flow.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of airport detection, in particular to an airport all-weather bird detection and driving method based on multi-modal adaptive fusion. BACKGROUND

[0002] Bird damage prevention of the airport flight area and its surrounding environment is an important work to ensure the safe operation of the aircraft. In order to effectively manage bird activities and reduce the safety risks caused by bird strike events, the airport management party usually deploys a special bird detection and driving system.

[0003] In the prior art, a bird damage prevention system usually includes detection and disposal. The detection part tends to use a multi-sensor fusion scheme, for example, using radar for preliminary search in a large range, and then combining visible light cameras or infrared thermal imaging and other optical devices for fine tracking and identification. The disposal part starts the corresponding driving device for operation according to the detection result.

[0004] However, this multi-sensor system has a key problem in actual application, that is, the overall detection accuracy of the system fluctuates with the change of the external environment. Specifically, the performance of the optical sensor is closely related to the weather and light conditions, and its imaging quality and identification accuracy will be affected at night or in bad weather such as rain and fog. The fluctuation of the system accuracy caused by the change of the environment will bring safety risks in the busy and complex airport environment. The airport ground personnel and vehicles will carry out normal operation in the flight area. If the bird detection and driving system misjudges the target at a certain moment due to the influence of the weather, and misidentifies the ground personnel as a target that needs to be disposed, the subsequent automatic or semi-automatic disposal action of the system may pose a direct safety threat to the personnel. SUMMARY

[0005] In order to overcome the fluctuation of the system accuracy in different weather conditions, ensure accurate identification of the target in any condition, and avoid harm to non-bird targets such as ground personnel, the present application provides an airport all-weather bird detection and driving method based on multi-modal adaptive fusion.

[0006] The airport all-weather bird detection and driving method based on multi-modal adaptive fusion provided by the present application adopts the following technical scheme: An airport all-weather bird detection and driving method based on multi-modal adaptive fusion, comprising the following steps: S1. Obtain preliminary track data in the monitored airspace based on radar detection; wherein the preliminary track data at least represents initial position and speed information of a moving target; S2. inputting the preliminary track data into a super-lightweight model to generate a preliminary candidate region; wherein the super-lightweight model is pre-trained based on radar data; and the preliminary candidate region is a sub-region in the monitoring airspace where a bird target is suspected to exist; S3. in response to the output of the preliminary candidate region, directing at least one non-radar sensor to perform directional observation on the preliminary candidate region to obtain high-precision multi-modal observation data in the preliminary candidate region; S4. detecting and obtaining environmental features of the monitoring airspace, and determining fusion weight parameters for a multi-modal fusion model corresponding to the environmental features of the monitoring airspace; S5. fusing the preliminary track data, the high-precision multi-modal observation data, and the fusion weight parameters, and inputting the multi-modal fusion model for multi-modal recognition to determine a final target object; S6. generating a handling instruction based on the final target object and preset safety policy data.

[0007] By adopting the above technical solution, the method processes the preliminary track data obtained by a large-range radar detection through a lightweight model. The principle of this processing is to serve as a high-speed, low-power consumption pre-filter to quickly identify and output a small-range preliminary candidate region. This enables the subsequent visual module equipped with high-precision optical and infrared sensors to no longer need to continuously scan the entire monitoring airspace, but only perform directional observation with high resolution on the candidate region. This hierarchical filtering approach separates wide-area search and directional precision measurement, avoiding the huge computational load caused by processing massive high-precision image data. The direct benefits are reduced performance requirements for the system hardware and shortened overall delay from preliminary discovery to accurate target locking.

[0008] When determining the target identity, the system determines a set of fusion weight parameters corresponding to real-time environmental features. The principle of this weight parameter is to dynamically adjust the analysis weight of data from different sensors in the multi-modal fusion model. For example, in foggy weather with low visibility, the parameter automatically increases the proportion of radar track data in decision-making; and in poor lighting conditions at night, the proportion of infrared thermal imaging data is increased. This adaptive fusion mechanism always focuses on the most reliable data source, overcoming the performance degradation of a single sensor due to environmental changes, thereby maintaining high recognition accuracy in various weather conditions. This directly benefits the safety of ground personnel, as it reduces the risk of the system misjudging personnel or work vehicles as bird targets in adverse weather conditions.

[0009] Finally, the information of the final target object identified with high confidence is used together with the preset safety policy data to generate a disposal instruction. The generation of the disposal instruction not only depends on the accurate identification of the target, but also must comply with the rules defined in the safety policy data. The principle is to forcibly bind the technical judgment result of target identification with the safety rules of airport operation. For example, even if a bird target is accurately identified, if its position is in the personnel activity area or important equipment area defined by the safety policy, the generation of the disposal instruction will be suspended or modified. This ensures that any subsequent physical disposal action will not harm the ground personnel in the airport, nor will it damage the key facilities.

[0010] Optionally, the S1 comprises the following steps: S11. receiving a radar raw echo signal of the monitored airspace to obtain an original data set containing target signals and clutter signals; S12. applying a dynamically updated clutter map to process the original data set to suppress the clutter signals in the original data set, wherein the clutter signals include ground clutter or weather clutter; S13. performing constant false alarm rate detection on the data set after the clutter suppression to extract independent radar point tracks; S14. logically associating multiple groups of radar point tracks appearing in continuous multiple frames of scanning, and creating a temporary candidate track archive for a group of radar point tracks when the group of radar point tracks meets preset track initiation logic; S15. data associating newly extracted radar point tracks in subsequent scanning frames with the existing candidate track archive, and updating and predicting the state vector of the successfully associated candidate track archive using a Kalman filter; S16. confirming and outputting the candidate track archive as the preliminary track data when the stable state of the candidate track archive meets preset confirmation conditions.

[0011] By adopting the above technical solution, after the radar raw echo signal containing target signals and clutter is received, a dynamically updated clutter map is first applied for processing. The principle of this processing is to use the fixed or slowly changing interference source information recorded in the clutter map in advance to cancel the original signal, thereby suppressing the stable clutter from the ground buildings or specific weather conditions. Then the system performs constant false alarm rate detection on the purified data. This detection method can dynamically adjust the detection threshold according to the local noise background, so as to control the number of false point tracks caused by random noise at a low level while ensuring the target discovery probability. The two continuous signal processing steps significantly reduce the computational burden of the subsequent track processing link by removing a large amount of invalid information at the source.

[0012] After the independent radar point tracks are extracted, the system logically associates the point tracks in continuous multiple frames of scanning, and only creates a temporary candidate track archive for the point track sequence satisfying the preset kinematic logic. The principle of the track initiation logic is that a real target has motion continuity, while a random false alarm point track does not have this feature, so this step can effectively filter out isolated or irregular false targets generated by accidental factors. For the created candidate track, the system uses a Kalman filter to perform data association and state updating. The principle of the Kalman filter is to fuse the predicted value of the motion model of the target and the actual measurement value of the sensor, smooth and correct the measurement data full of noise, so as to obtain the optimal estimation of the real position and speed of the target.

[0013] Finally, only when a candidate track archive is stably updated for multiple times and the error of the state vector thereof converges to below a preset threshold, the candidate track archive is confirmed as valid preliminary track data and output. Through this whole processing flow from the original echo to the stable track, the method overcomes the problem that the original radar data is full of noise and uncertainty, and provides a high-quality and high-reliability input source for the subsequent operation of the whole system.

[0014] Optionally, the S2 comprises the following steps: S21. dynamically determining a current danger area defined by the real-time position of a person or a vulnerable target in a monitored airspace; S22. performing a geographical position comparison between the position of a moving target represented by the preliminary track data and the current danger area, and if the moving target is located in the current danger area, eliminating the preliminary track data and aborting subsequent processing; S23. if the moving target is located outside the current danger area, analyzing the preliminary track data to extract flight motion features, and performing classification and discrimination through the ultra-lightweight model to obtain a target category attribution; S24. when the target category attribution is a preset bird target, generating the preliminary candidate area based on the current position information of the moving target.

[0015] By adopting the above technical solution, when the preliminary track data is processed, a current danger area defined by the real-time position of a person or a vulnerable target is first determined. The system compares the position of each moving target with the danger area in terms of geographical position, and if the target is located in the area, the corresponding track data is directly eliminated and subsequent processing is aborted. The principle is to establish a front-end, geographical position-based dynamic safety filter. This ensures that the system does not invest processing resources in targets in the area of personnel activities under any circumstances, thereby fundamentally eliminating the risk of accidental harm to personnel or critical equipment due to identification errors in subsequent links.

[0016] Only when the moving target is confirmed to be outside the current dangerous area, the system will continue to analyze its track data to extract flight motion features and make a preliminary classification of the bird target. This conditional processing flow based on safety verification reserves the system's computing resources only for unknown targets located in the allowed engagement area. This principle avoids unnecessary analysis of targets that are identified as birds but cannot be handled due to safety restrictions, thereby saving the system's computing power and enabling it to respond more efficiently to potential threats that truly require attention.

[0017] Optionally, the S3 comprises the following steps: S31. Direct the optical axis of the at least one non-radar sensor to the preliminary candidate area; S32. Capture a wide-field image of the preliminary candidate area and identify the pixel coordinates of the moving target within the wide-field image; S33. Based on the pixel coordinates, adjust the optical focal length of the non-radar sensor to optically magnify the moving target and establish stable tracking of the moving target; S34. In the stable tracking state, synchronously collect visible light image data and infrared thermal imaging data of the moving target to obtain high-precision multi-modal observation data.

[0018] By adopting the above technical solution, the method controls the optical axis of the non-radar sensor to point to the preliminary candidate area determined by the radar data, and first captures a wide-field image of the area to identify the specific pixel coordinates of the moving target. This working principle combines the wide-area guidance capability of the radar with the fast imaging capability of the optical sensor, achieving fast target capture from a blurred area to an accurate pixel. This mechanism avoids time-consuming large-scale scanning search by the optical sensor, shortens the time interval from preliminary radar discovery to optical locking, and improves the overall response speed of the system.

[0019] After determining the pixel coordinates of the target, the system adjusts the optical focal length of the sensor to magnify the target and establishes stable tracking. In this stable tracking state, the system synchronously collects visible light images and infrared thermal imaging data of the target to obtain high-precision multi-modal observation data. This closed-loop tracking and synchronous acquisition principle ensures that the data obtained is high-resolution and strictly corresponds in time. The final output of high-definition images and thermal feature data provides key detailed features about the target's shape and physical properties that radar cannot provide, thereby enhancing the ability to distinguish between birds and other small objects such as drones.

[0020] Optionally, the S4 comprises the following steps: S41. Real-time quantification of the environmental characteristics of the monitoring airspace is performed to obtain a set of continuously changing physical parameters representing the current environmental state, including at least ambient light intensity, atmospheric visibility, and precipitation rate; S42. For each data modality of the pre-trained multi-modal fusion model, a non-normalized reference weight is determined for each data modality based on the set of continuously changing physical parameters by calculating a pre-set weight function corresponding to the data modality; S43. All non-normalized reference weights are normalized to generate the final fusion weight parameter.

[0021] By using the above technical solution, the method obtains a high-fidelity description of the current environmental state by real-time quantification of a set of continuously changing physical parameters, such as ambient light intensity and atmospheric visibility. The system pre-sets an independent weight function for each data modality, which receives the above-mentioned physical parameters as input and calculates a non-normalized reference weight representing the current performance of the modality. This principle replaces discrete rule judgment with a continuous mathematical function, enabling the system to respond proportionally and smoothly to light transition states such as dusk and dawn, or different severity weather conditions such as thin fog and light rain, overcoming the limitations of traditional fixed rules in handling transitional or superimposed environmental states.

[0022] After determining the reference weights of each data modality, the system normalizes all these reference weights to generate the final fusion weight parameter, which ensures that when the weight of one sensor (such as visible light) decreases due to environmental deterioration, the relative weight of other more reliable sensors (such as radar) will automatically increase. This dynamic and self-balancing weight distribution mechanism ensures that the final multi-modal fusion decision is always based on the most reliable information combination, thereby maintaining a high level of recognition accuracy in various complex real-world scenarios.

[0023] Optionally, the S5 includes the following steps: S51. Time stamp alignment processing is performed on the filtered preliminary track data associated with the high-precision multi-modal observation data to generate a set of strictly synchronized data sets in the time dimension; S52. Based on the data sets to be processed, independent modality feature vectors are extracted for radar, visible light, and infrared thermal imaging data modalities, respectively; S53. The fusion weight parameter is applied to weight the independent modality feature vectors to generate a set of weighted modality feature vectors representing the priority of each modality feature in the current environment; S54. inputting the weighted modal feature vectors into a pre-trained fusion network, and performing deep fusion through a cross-modal attention mechanism within the fusion network to generate a comprehensive feature representation; S55. inputting the comprehensive feature representation into a classifier for probability classification to output a preliminary identification result containing a target category and a corresponding confidence level; S56. comparing the preliminary identification result with a preset physical constraint database to perform a physical constraint compliance check, and determining the preliminary identification result as the final target object only when the check passes.

[0024] By adopting the above technical solution, the method first performs timestamp alignment processing on the track data and observation data from different sensors, the principle of which is to unify data streams of different refresh rates to a common time reference to generate a set of strictly synchronized data in time dimension. The system then extracts independent feature vectors for each modality from the synchronized data set, and applies fusion weight parameters for weighting. The weighted feature vectors are input into a fusion network using a cross-modal attention mechanism. The principle of this mechanism is to let the model learn and focus on the most critical internal relations between different modal features, such as focusing on the wingspan feature in the image when analyzing track features, thereby generating a comprehensive feature representation with higher resolution than simple data splicing.

[0025] After generating the comprehensive feature representation, the system performs probability classification through the classifier to output a preliminary identification result containing a target category and a corresponding confidence level. Finally, the preliminary identification result also needs to be checked for compliance with the preset physical constraint database. The principle of this check is to serve as a logical safety gate independent of the artificial intelligence model, which is used to filter out results that may be statistically valid but impossible in the physical world, such as a target identified as a bird flying at a speed exceeding the limit of the species. Only the results that pass this check will be determined as the final target object. This series of processes from deep fusion to probability output and then to physical check collectively ensure the high accuracy and high reliability of the final output result, effectively avoiding dangerous disposal intentions caused by model misjudgment, thereby ensuring the safety of personnel and equipment.

[0026] Optionally, the S6 includes the following steps: S61. performing real-time threat level assessment on the final target object based on the flight trajectory, category attribute, and real-time operation state data of the airport to obtain a threat score; S62. predicting and generating a disposal path from a disposal unit at a preset location to the final target object, and comparing the disposal path with an absolute safety area defined in the preset safety policy data to perform static safety check; S63. When and only when the static safety check passes, the disposal path is compared with real-time updated airspace and ground traffic data for dynamic conflict analysis, and a dynamic safety permission signal is outputted; S64. Based on the threat score, the result of the static safety check and the dynamic safety permission signal, the best disposal strategy is decided from a preset disposal strategy library; S65. When the best disposal strategy needs to execute physical attack, disposal parameters for guiding the disposal unit are calculated, the disposal parameters at least including azimuth angle, elevation angle and target lead time; S66. For disposal strategies needing human intervention, the best disposal strategy and the disposal parameters are packaged into an authorization request and submitted to a human-computer interaction interface, and an affirmative authorization instruction is waited to be received; S67. Based on the affirmative authorization instruction or disposal strategy decision without human intervention, the disposal instruction containing the disposal parameters is generated and issued to the disposal unit.

[0027] By adopting the above technical solution, the method combines the motion attribute of the final target object and the airport real-time running state data, performs threat level evaluation to obtain a quantitative threat score. The principle makes the threat judgment no longer based on the target itself, but integrates the space-time situation where the target is located, so that the system can distinguish the different dangerous levels caused by birds wandering in the open area and birds invading the take-off and landing route, thereby realizing proportional and differentiated response. Before generating the disposal decision, the system will predict the disposal path of the disposal unit to the target, and first compare the path with the preset absolute safety area including the personnel activity area. Only after passing the static safety check, the system will continue to perform dynamic conflict analysis on the path and real-time air-ground traffic data. This double-layer serial safety check mechanism can not only ensure that the disposal action does not violate the fixed safety bottom line, but also can avoid real-time conflict with dynamic targets such as moving aircraft and vehicles.

[0028] Based on the threat score and the result of the double-layer safety check, the system will decide the best disposal strategy from the strategy library. When the strategy needs to execute physical attack, the system will calculate the accurate disposal parameters including the target lead time. For high-risk strategies, the system will package the decision scheme into an authorization request and submit it to the human-computer interaction interface, and must obtain the affirmative authorization instruction of the operator before continuing to execute. The mechanism ensures that the finally generated disposal instruction is verified multiple times in safety, and is authorized by humans in high-risk scenarios.

[0029] In summary, the present application includes at least one of the following beneficial technical effects: 1、The method uses a hierarchical detection mechanism, first uses radar data to perform rapid and large-scale preliminary screening to generate a small range of candidate areas, and then guides high-precision optical sensors to perform directional observation. This mechanism avoids the massive data processing burden brought by high-precision sensors for wide-area inspection, reduces system operation delay, and improves the overall response speed from target discovery to accurate locking.

[0030] 2、The method is based on the quantitative collection of environmental physical parameters, dynamically adjusts the proportion of each sensor data in the fusion decision through a weight function, and combines the kinematic characteristics of radar, the morphological characteristics of visible light, and the thermal characteristics of infrared thermal imaging for comprehensive identification. This adaptive fusion mechanism ensures that the system always focuses on the most reliable data source for judgment under different environmental conditions such as day and night, rain and fog, significantly improving the accuracy of target identification and the reliability of all-weather operation.

[0031] 3、The method establishes a multi-layer safety verification system throughout the entire process, which pre-eliminates any target located in the dynamic dangerous area defined by personnel or vulnerable target at the initial detection stage, and again performs dual conflict verification of static safety area and dynamic traffic on the disposal path before generating the final disposal instruction. This multi-layer safety verification system ensures that all disposal decisions of the system are made under the protection of multiple safety redundancies, fundamentally avoiding the risk of accidental injury to airport ground personnel and critical facilities. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The flowchart of the airport all-weather bird detection method based on multi-modal adaptive fusion in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only intended to explain the present application and not to limit the present application.

[0034] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the inventive concept. As part of the description, some of the diagrams in the present disclosure are represented in block diagram form to avoid obscuring the disclosed principles. Not all features of a practical implementation are necessary in order to describe the disclosed principles. Further, the language used in the present disclosure has been principally selected for readability and instructional purposes and can not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such

[0035] The terms "a," "an," and "the" are not intended to refer to singular entities, but include the general class of which a specific example can be used for illustration. Thus, the use of the term "a" or "an" can mean any number of including "a," "one or more," "at least one," and "one or more than one." The term "or" means any one of the alternatives, as well as any combination of the alternatives, including all of the alternatives, unless the alternatives are expressly indicated to be mutually exclusive. The phrase "at least one of" in combination with a list of items is intended to refer to a single item from the list of items, or any combination of items from the list of items, unless expressly limited otherwise. The phrase "at least one of" is not intended to refer to all of the items in the list.

[0036] In the operating environment of modern airports, bird activity is an objective factor that affects the safety of aircraft, especially during the take-off and landing stages. In order to reduce the potential risk of bird strike incidents, airports usually deploy various bird control measures.

[0037] Existing control methods, such as independent radar detection systems or independent photoelectric monitoring systems, have some inherent technical problems when dealing with complex and variable airport environments. First, the detection accuracy of these systems fluctuates with environmental changes, and their ability to discover and identify targets is limited to varying degrees in low-visibility conditions such as night, heavy fog, or rain. Second, there is often a lack of close automatic linkage between the detection system and the disposal system, resulting in delays in the response process from discovering bird activity to executing the drive-off. More importantly, in dynamic scenarios with frequent personnel and vehicle activity, the combination of unstable detection accuracy and system response lag makes it challenging to ensure the safety of ground personnel and critical equipment.

[0038] To this end, an airport all-weather bird detection method based on multi-modal adaptive fusion is proposed in the embodiments of the present application, which comprises the following steps S1-S6. The method first efficiently discovers and locks potential targets through a hierarchical detection process from coarse to fine; then introduces an environment-adaptive fusion mechanism in the identification stage to ensure the identification accuracy under various weather and light conditions; finally, embeds multi-level safety check logic throughout the decision-making stage to ensure the absolute safety of the disposal action.

[0039] S1. Obtain preliminary track data in a monitored airspace based on radar detection; wherein the preliminary track data at least represents initial position and velocity information of a moving target.

[0040] The preliminary track data mentioned here is not the raw signal output by the radar sensor, but the structured data obtained after a series of deep signal processing and data fusion processes, which aims to overcome the problems of ground clutter, weather clutter and measurement noise commonly existing in the original radar echo signal.

[0041] Specifically, in an embodiment, S1 comprises the following steps S11-S16.

[0042] S11. Receive the radar raw echo signal of the monitored airspace to obtain an original data set containing target signals and clutter signals.

[0043] S12. Apply a dynamically updated clutter map to process the original data set to suppress the clutter signals in the original data set, wherein the clutter signals include ground clutter or weather clutter.

[0044] S13. Perform constant false alarm rate detection on the data set after clutter suppression to extract independent radar point tracks.

[0045] S14. Perform logical association on multiple groups of radar point tracks appearing in consecutive multiple frames, and create a temporary candidate track archive for a group of radar point tracks when the group of radar point tracks meets the preset track initiation logic.

[0046] S15. Data associate the newly extracted radar point tracks in the subsequent scan frames with the existing candidate track archive, and update and predict the state vector of the successfully associated candidate track archive using a Kalman filter.

[0047] S16. When the stable state of a candidate track archive meets the preset confirmation condition, confirm and output the candidate track archive as the preliminary track data.

[0048] The radar raw echo signal is the electrical signal form of the electromagnetic wave backscattering signal received by the radar antenna without any processing, which contains the reflection information of all objects in the monitored airspace, including the desired target, fixed terrain, weather particles, etc. After the raw echo signal in one or more scanning periods is digitized, the raw data set is formed. In the data set, the ground clutter refers to the radar echo generated by the ground, buildings, hills and other fixed or slowly changing targets, which is characterized by fixed position but possibly high intensity. The weather clutter is the echo generated by rain, snow, hail and other weather water condensates, which is characterized by wide spatial distribution and dynamic change with the weather.

[0049] In order to extract effective information from the raw data set full of interference, the system first applies a dynamically updated clutter map to suppress the clutter signal. The clutter map is a database that stores the long-term, stable background reflection intensity in each cell of the monitored airspace. Dynamic updating means that the clutter map continuously learns and adapts to the environment by averaging or statistically processing the echoes of multiple scanning periods, so as to accurately depict the fixed and unchanging ground clutter background. The suppression process is at the signal processing level, which cancels or filters the raw data set collected in real time with the corresponding cell value in the clutter map, so as to remove the stable interference such as ground clutter. Then, the purified data set is subjected to constant false alarm rate detection. Constant false alarm rate detection is an adaptive signal detection algorithm that does not use a fixed detection threshold, but dynamically sets a decision threshold that keeps the false alarm probability constant by estimating the average power of noise and residual clutter in the local area around each unit to be detected. This ensures that the system can extract independent radar tracks with stable low false alarm rate in different regions and different weather backgrounds.

[0050] After obtaining discrete radar tracks, the system needs to concatenate the tracks belonging to the same target into a track. This process is first realized through logical association, the system will predict a "gate" area that may appear at the next time for each existing temporary track, and then find the track falling into the gate in the track obtained in the new scanning frame as the candidate association object. The track initiation logic is used to "create a track from scratch", which is a set of preset rules for judging whether a new, unowned but spatiotemporal continuous track is a new target. For example, a typical initiation logic can be the "two-thirds" criterion, that is, in three consecutive scans, if there are at least two radar tracks on a predicted motion trajectory, the system considers that a new target is discovered and creates a candidate track file for it. The candidate track file is a data structure for storing the motion state vector of a target, such as position, velocity, acceleration, etc. It is temporary because the track has just been established and its authenticity and stability have not been fully verified, and it may be formed by accidental noise coincidence.

[0051] For the candidate track files that successfully achieve data association, the system updates and predicts their state vectors using Kalman filter. Kalman filter is a recursive optimal estimation algorithm. It first predicts the state of the target at the current time based on the physical model of the target's motion. Then, it fuses the predicted value with the actual measured radar plot position at the current time, and calculates the optimal estimation value of the target state at the current time. This estimation value is smoother and more accurate than the measurement value or the prediction value. Finally, the preset confirmation condition is used as the criterion for converting a temporary track to a formal track. This criterion is usually composite, for example, a candidate track file must be successfully updated for at least 5 times in succession, and the state estimation error covariance output by the Kalman filter is less than a preset stable threshold. Only the candidate track that meets these conditions at the same time will be finally confirmed by the system, and its state vector will be output as a reliable preliminary track data.

[0052] For example, assume that on a rainy day, a seagull flies over the parking lot beside the runway of an airport. In S11, the original echo signal received by the radar will mix the weak signal of the seagull, the strong ground clutter signal of the aircraft on the parking lot, and the large range of rainfall weather clutter signal. The system applies the clutter map, and since the aircraft on the parking lot is a fixed target, its ground clutter signal is effectively suppressed. The constant false alarm rate detection algorithm dynamically sets the threshold according to the rainfall clutter background around the seagull, and successfully extracts the signal of the seagull as an independent radar plot. In three consecutive radar scans, the system finds radar plots within the association gate that meets the flight trajectory of the seagull, which meets the “three-thirds” track initiation logic, thereby creating a temporary candidate track file for the seagull. In the fourth and subsequent scans, the system continues to associate the newly detected seagull plots with the candidate track file, and uses the Kalman filter to continuously smooth and correct the track, so that the output position and speed of the track are closer and closer to the true values. When the candidate track file of the seagull is successfully updated for 5 times, and its state error converges to a small range, it meets the confirmation condition, and the system formally confirms the file as a preliminary track data, and outputs the accurate position and speed information contained in the file to the subsequent steps.

[0053] S2. inputting the preliminary track data into a super-lightweight model to generate a preliminary candidate region; wherein the super-lightweight model is pre-trained based on radar data; and the preliminary candidate region is a sub-region in the monitoring airspace where a bird target is suspected to exist.

[0054] This step is used to perform a fast and low-cost pre-screening on a large number of preliminary track data, to preliminarily determine which tracks are worth investing in subsequent more valuable and high-cost refined detection resources. The reason why a super-lightweight model needs to be used in this step is that the conventional airport surveillance radar is good at searching in a large range and at a long distance, and can efficiently find moving targets in the airspace, but its resolution is limited, and it is usually unable to accurately identify the specific type of the target, nor can it provide sufficiently accurate pointing information to directly guide the narrow field of view optical device. If you want to rely only on radar to achieve accurate detection and identification in a small range, you need to use advanced phased array radar technology, which will greatly increase the deployment cost of the system.

[0055] Therefore, the present application uses a super-lightweight model to analyze the preliminary track data provided by the conventional radar, and the requirements for such a model are low computational complexity, small memory occupation, and extremely short reasoning delay, to ensure that this screening step will not become a bottleneck of the entire real-time data processing link. The function of this model is not accurate identification, but based on the kinematic characteristics of the target, such as flight height, speed, maneuvering mode, etc., to quickly filter out targets that are obviously not birds (such as civil aviation aircraft) or meaningless clutter tracks, and generate a rough candidate area for those tracks that exhibit bird characteristics in kinematics.

[0056] In a specific implementation, the super-lightweight model can be a classic machine learning model or a simple structure neural network. For example, a support vector machine (SVM) model can be used, which can efficiently learn and build a decision hyperplane that divides "bird characteristics" and "non-bird characteristics" based on several key motion features extracted from the track data. Another optional implementation is to use a simplified recurrent neural network (RNN) or gated recurrent unit (GRU) network, which can directly process the time series of track data to learn and identify the flight patterns unique to birds at a lower computational cost.

[0057] Specifically, in an embodiment, the S2 includes the following steps S21-S24.

[0058] S21. Dynamically determine the current dangerous area in the monitored airspace defined by the real-time position of personnel or vulnerable targets.

[0059] S22. Geographical position comparison between the position of the moving target represented by the preliminary track data and the current dangerous area. If the moving target is located in the current dangerous area, the preliminary track data is discarded and the subsequent processing is terminated.

[0060] S23. If the moving target is outside the current danger zone, resolve the preliminary track data to extract flight motion features, and classify by the ultra-lightweight model to obtain target category attribution.

[0061] S24. When the target category attribution is a preset bird target, generate the preliminary candidate region based on the current location information of the moving target.

[0062] The system first performs a pre-emptive dynamic safety filtering process, which aims to eliminate any moving target located within a danger zone defined by personnel or vulnerable targets at the earliest stage of data processing. The personnel here can include ground maintenance personnel, baggage handlers, aircraft marshalling personnel, or security patrol personnel working in the airport flight zone, apron, or taxiway area. Vulnerable targets can include aircraft being refueled or maintained, airport fuel depots, navigation and communication equipment antennas, and other critical infrastructure.

[0063] In order to dynamically determine the current danger zone, the system can be data interfaced with the personnel and vehicle high-precision positioning system of the airport, such as the Global Positioning System (GPS) or Ultra-Wideband (UWB) positioning system. For each positioned personnel or vulnerable target, the system automatically generates a virtual safety buffer zone with a preset radius or a preset polygon boundary around its real-time geographic coordinates. The union of all these independent safety buffer zones in geographic space constitutes the current danger zone that the system needs to avoid at the current time.

[0064] After S1 outputs a preliminary track data, the system compares the three-dimensional geographic coordinates of the moving target represented by the track data with the current danger zone. To achieve fast comparison with low computational power consumption, the irregular danger zone formed by the merging of multiple safety buffer zones can be pre-processed and mapped onto a two-dimensional or three-dimensional rasterized map. The coordinate of the moving target queries the rasterized map, and only one memory access is needed to determine whether it falls into the marked dangerous grid, thereby achieving efficient position relationship comparison.

[0065] Only when the moving target is outside the current danger zone, the system will invoke the ultra-lightweight model to classify it. The model receives the flight motion features resolved from the preliminary track data, and its principle is to classify by learning the motion pattern differences of different categories of targets. For example, the characteristics of ordinary civil aviation aircraft are high speed, high altitude, and low curvature stable track; the characteristics of ground vehicles are constant altitude on the ground and speed conforming to land regulations; and the characteristics of birds are usually low speed and height, and high trajectory curvature and maneuverability. The ultra-lightweight model, such as a lightweight support vector machine or recurrent neural network, outputs the target category attribution of the moving target by comprehensive analysis of these motion features.

[0066] After obtaining the discrimination result of the target class belonging to the bird class, the system generates a preliminary candidate region based on the current position information of the moving target to guide the subsequent optical sensor. The way to generate the region is optional. One way is to generate a fixed size rectangular or circular region centered on the real-time position of the target. Another better way is to generate an elliptical region along the direction of the target's current speed, with the major axis parallel to the speed vector, so as to better cover the position where the target is most likely to appear at the next moment. The preliminary candidate region is dynamically updated. As S1 continuously outputs new preliminary track data, the candidate region will be refreshed with the same update frequency, continuously centered on the latest position of the moving target, to ensure that the subsequent optical sensor is always guided to the correct observation direction.

[0067] Continue to take the example of the seagull in the previous embodiment. Assume that while the seagull is flying over the parking lot, a ground maintenance team is working on an aircraft 100 meters in front of its flight path, and all members are wearing GPS positioning devices. In S21, the system receives the real-time GPS coordinates of all members of the ground maintenance team and generates a circular region with a radius of 50 meters around the overall position of the team as the current danger region. In S22, the system obtains the preliminary track data of the seagull and compares its current position with the 50-meter radius danger region. Since the seagull's position is outside the danger region, the geographic position comparison passes, and the process continues. In S23, the system analyzes the flight motion characteristics of the seagull, such as a flight height of 15 meters, a flight speed of 45 kilometers per hour, a large trajectory curvature, and a super-lightweight model, and determines that the target class of the target belongs to "bird target". In S24, based on the current position coordinates of the seagull, the system generates a square preliminary candidate region covering an area of 30 meters by 30 meters around it and outputs the coordinates of the region to S3.

[0068] S3. In response to the output of the preliminary candidate region, at least one non-radar sensor is directed to observe the preliminary candidate region to obtain high-precision multi-modal observation data within the preliminary candidate region.

[0069] In S3, the non-radar sensor generally refers to an imaging device working in the optical band, which needs to meet the requirements of being controlled by a precision servo system for rapid pointing and having optical zoom capability. In a specific embodiment of the present application, the non-radar sensor assembly can integrate a visible light camera and a long-wave infrared thermal imaging sensor. These two sensors can provide complementary target information. The former is used to obtain the shape, contour and color features of the target, and the latter is used to obtain the infrared thermal radiation features of the target.

[0070] To realize the directional observation of the preliminary candidate region, the non-radar sensor can be installed on a servo turret capable of rotating in azimuth and elevation, and preferably the optical axis of the visible light camera and the infrared sensor are parallel or coaxial. When the system receives the geographic coordinates of the preliminary candidate region output by S2, the control system first calculates the azimuth angle and the elevation angle that the servo turret needs to rotate, and then drives the motor to control the turret to rotate quickly and smoothly until the optical axis of the sensor is accurately pointed to the center of the candidate region. In addition to directly driving the turret, other possible guiding methods include using a high dynamic range mirror system to guide the field of view of the sensor by changing the angle of the mirror to achieve faster pointing speed.

[0071] The high-precision multi-modal observation data obtained by the above directional observation has several significant characteristics. First of all, high precision, which is reflected in its spatial resolution. Through the magnification effect of the optical system, the image data obtained can clearly show the fine details of the target, which is not possessed by low-resolution radar data. Secondly, multi-modal, which means that the data set contains information from two different physical dimensions (modalities) of visible light and infrared. Visible light data reflects the physical shape of the target, while infrared data reflects the temperature difference between the target and the background environment, and the combination of the two can provide more abundant classification criteria than single modal.

[0072] Specifically, in an embodiment, S3 includes the following steps S31-S34.

[0073] S31. Control the optical axis of the at least one non-radar sensor to point to the preliminary candidate region.

[0074] S32. Capture a wide field of view image of the preliminary candidate region, and identify the pixel coordinates of the moving target in the wide field of view image.

[0075] S33. Based on the pixel coordinates, adjust the optical focal length of the non-radar sensor to optically magnify the moving target, and establish stable tracking of the moving target.

[0076] S34. In the stable tracking state, synchronously collect visible light image data and infrared thermal imaging data of the moving target to obtain the high-precision multi-modal observation data.

[0077] When the non-radar sensor adjusts the optical focal length based on the pixel coordinates to optically magnify the moving target, the system starts a closed-loop feedback control process to establish stable tracking. In this process, the image processing module of the system will analyze each subsequent frame of image in real time, and calculate the new pixel coordinates of the target in the current field of view. The deviation between the new coordinates and the reference coordinates of the image center is the tracking error. This error signal is fed back to the servo control system, which calculates the fine adjustment correction amount required for the azimuth axis and the elevation axis of the turntable, and drives the motor to execute the correction, so as to reposition the target at the image center. This process is executed at a very high frequency, ensuring that the field of view of the sensor can always follow the target movement, forming stable tracking. The advantage of this method is that through optical zoom, the system obtains more pixels and clearer features of the image covering the target, providing high-quality visual evidence for subsequent accurate identification. At the same time, after zooming, the field of view is narrowed, and the system no longer needs to process a large range of irrelevant background information, reducing the computational load of subsequent image analysis algorithms and reducing the acquisition of redundant information.

[0078] In the state of stable tracking, the system fuses the data from the visible light camera and the infrared thermal imaging sensor into high-precision multi-modal observation data. Here, fusion refers to synchronization and pairing at the data acquisition level. The data acquisition module of the system will ensure that at any time point, image frames are captured synchronously from the two sensors. Each visible light image will be combined with the infrared thermal imaging image corresponding to it in the time stamp to form a data pair. This series of continuous data pairs with synchronized time stamps constitutes the high-precision multi-modal observation data delivered to the subsequent processing steps. The characteristics of this data set are that the two modal information inside is strictly consistent in time, ensuring that the subsequent fusion analysis processes the different physical attributes of the target at the same instant.

[0079] Continuing the scenario of the seagull from the previous embodiment as an example. The system has generated a 30m by 30m preliminary candidate area for the seagull. The servo controller of the system calculates the corresponding azimuth and elevation angles, and drives the photoelectric turntable so that the optical axes of the visible light camera and the infrared sensor mounted thereon are jointly pointed to the center of the candidate area. The visible light camera captures a frame of image of the area in the wide field of view mode, and the image processing algorithm quickly identifies the pixel coordinates of the seagull in the frame of image. Based on the pixel coordinates, the optical lens group of the system adjusts the focal length to optically magnify the seagull until it clearly occupies the main part of the field of view. At the same time, the tracking algorithm is started, and according to the position changes of the seagull in the subsequent video frames, the turntable attitude is continuously fine-tuned so that the seagull always remains in the center of the field of view, thereby establishing stable tracking. In the process of stably tracking the seagull, the data acquisition module of the system synchronously records the high-definition color images from the visible light camera and the thermal imaging images from the infrared sensor at a rate of 30 frames per second, and packs each pair of images with the same timestamp as a high-precision multi-modal observation data, which is continuously output to the subsequent steps.

[0080] S4. Detect and acquire the environmental characteristics of the monitoring airspace, and determine the fusion weight parameters of the multi-modal fusion model corresponding to the environmental characteristics of the monitoring airspace.

[0081] In a multi-sensor fusion system, the credibility of the data provided by different types of sensors is not constant, but is closely related to the real-time environmental characteristics of the system. Specifically, the data credibility of the visible light camera reaches the highest in the daytime, under sufficient light and high visibility conditions, at which time it can provide the highest resolution and the most detailed morphological feature information. However, in low-light environments such as at night or dusk, or in severe weather such as heavy fog or heavy rain, the imaging quality will decrease significantly, and the data credibility will also decrease. The data credibility of the long-wave infrared thermal imaging sensor mainly depends on the temperature difference between the target and the background. Usually at night, the temperature difference between warm-blooded birds and the rapidly cooled background environment is obvious, so the credibility of the infrared data is high. However, under certain specific conditions, such as noon when the air temperature is close to the body temperature of the birds or the surface of an object soaked in rainwater, the thermal feature contrast will weaken, and the data credibility will also be affected. The detection capability of the radar sensor is basically not affected by the light conditions, and it has good penetration to fog, haze, etc., so it usually has high credibility in the detection of the presence or absence of moving targets, but it provides limited effective features in fine classification and identification, and it may also be disturbed by meteorological clutter in extreme precipitation weather.

[0082] Specifically, in a certain embodiment, the S4 includes the following steps S41-S43.

[0083] S41. Real-time quantification of the environmental characteristics of the monitoring airspace is performed to obtain a set of continuously changing physical parameters representing the current environmental state; the physical parameters at least include ambient light intensity, atmospheric visibility, and precipitation rate.

[0084] S42. For each data modality of the pre-trained multi-modal fusion model, based on the set of continuously changing physical parameters, a non-normalized reference weight is determined for each data modality by calculating a preset weight function corresponding to the data modality.

[0085] S43. All non-normalized reference weights are normalized to generate the final fusion weight parameter.

[0086] For example, assume that there are two typical and completely different environmental scenarios. The first scenario is a daytime dense fog weather. The system-integrated environmental sensors real-time quantification of a set of physical parameters, such as: the ambient light intensity is 15000 lux, indicating daytime; the atmospheric visibility is 400 meters, indicating the presence of dense fog; the precipitation rate is 0 millimeters per hour. The system obtains a set of continuous parameters representing the current environmental state. This set of parameters is respectively input into the weight function corresponding to each of the radar, visible light, and infrared data modalities. The weight function of the radar is not sensitive to the dense fog which is good at penetrating, and outputs a higher non-normalized reference weight, such as 0.9. The weight function of the visible light camera is extremely sensitive to the atmospheric visibility parameter, and the low visibility of 400 meters will result in it outputting a very low reference weight, such as 0.1. The weight function of the infrared sensor is affected by water vapor to some extent, but the effect is better than that of visible light, so it outputs a medium reference weight, such as 0.4. In S43, the system normalizes the set of reference weights (0.9, 0.1, 0.4) to calculate the final fusion weight parameter, in which the radar weight accounts for about 64%, the infrared weight accounts for about 29%, and the visible light weight accounts for only 7%. The result shows that in this scenario, the subsequent fusion decision will mainly rely on radar data, supplemented by infrared data, and basically ignore the failed visible light data.

[0087] The second scenario is a clear night. The physical parameters collected by the system become: the ambient light intensity is 1 lux, indicating a night; the atmospheric visibility is 10000 meters, indicating a clear weather; the precipitation rate is 0 mm per hour. The weight functions of each sensor are recalculated based on this new set of parameters. The weight function of the radar is not affected and still outputs a high baseline weight, such as 0.8. The weight function of the visible light camera outputs a baseline weight close to zero due to the extremely low light intensity, such as 0.05. The weight function of the infrared sensor, on the other hand, outputs a high baseline weight according to the prior model that the temperature difference between the target and the background is usually large at night, such as 1.0. The system normalizes the new set of baseline weights (0.8, 0.05, 1.0) to obtain the final fusion weight parameters, in which the infrared weight accounts for about 54%, the radar weight accounts for about 43%, and the visible light weight accounts for only 3%. The result shows that in this scenario, the system will intelligently switch to mainly relying on the data of the infrared sensor for recognition, and use the radar data as a key auxiliary judgment basis.

[0088] S5. Fusing the preliminary track data, the high-precision multi-modal observation data, and the fusion weight parameters, inputting the multi-modal fusion model for multi-modal recognition to determine the final target object.

[0089] The fusion weight parameter does not change the inherent network structure of the multi-modal fusion model, but serves as a dynamic modulator to adjust the information in the middle link of the data processing flow. Specifically, the weight parameter is used as a multiplier to act on the features extracted from the data of various sensors. A higher weight value will amplify the magnitude of the corresponding modal features, thereby enhancing their influence in the subsequent fusion decision; on the contrary, a lower weight value will weaken the influence of the corresponding modal features.

[0090] A typical multi-modal fusion model usually contains multiple parallel feature extraction branches, a fusion module, and a classification output head. Taking the present application as an example, the model can contain a one-dimensional time series processing branch for processing the preliminary track data, such as using a Gated Recurrent Unit (GRU) or a Long Short-Term Memory (LSTM) network to extract the kinematic features of the target. At the same time, the model also contains a two-dimensional image processing branch for processing the high-precision multi-modal observation data, such as using a Convolutional Neural Network (CNN) such as the ResNet architecture to extract the morphological contour features in the visible light image and the thermal distribution features in the infrared thermal imaging image. The feature vectors extracted by each branch are finally sent to the fusion module, which can integrate multi-source features through feature concatenation or more complex cross-modal attention mechanisms, etc.

[0091] In a specific internal data processing flow, the preliminary track data and the high-precision multi-modal observation data are input into the respective corresponding feature extraction branches of the model in parallel. After the branches output independent modal feature vectors, the system applies the fusion weight parameters determined by the previous steps to weight and adjust the feature vectors. The weighted feature vectors are then sent to the fusion module to integrate into a single comprehensive feature representation containing all modal key information. Finally, the comprehensive feature representation is passed to the classification output head, which calculates the probability distribution of the target belonging to birds, drones or other categories through structures such as fully connected layers and Softmax functions, thereby determining the final target object.

[0092] Specifically, in an embodiment, S5 includes steps S51-S56.

[0093] S51. Time stamp alignment processing is performed on the filtered preliminary track data associated with the high-precision multi-modal observation data to generate a set of strictly synchronized time-dimension data sets to be processed.

[0094] S52. Based on the data sets to be processed, independent modal feature vectors are extracted for radar, visible light and infrared thermal imaging data modalities, respectively.

[0095] S53. The fusion weight parameters are applied to weight the independent modal feature vectors to generate a set of weighted modal feature vectors for representing the priority of each modal feature in the current environment.

[0096] S54. The weighted modal feature vectors are input into a pre-trained fusion network, and deep fusion is performed through the cross-modal attention mechanism in the fusion network to generate a comprehensive feature representation.

[0097] S55. The comprehensive feature representation is input into a classifier for probability classification to output a preliminary identification result containing the target category and its corresponding confidence.

[0098] S56. The preliminary identification result is compared with a pre-set physical constraint database for physical constraint compliance verification, and the preliminary identification result is determined as the final target object only when the verification passes.

[0099] Continuing the scenario of the preceding embodiment regarding the seagull, and taking the example of a clear, cloudless night. The system's data processing module receives asynchronous data streams from different sensors: the radar module outputs preliminary track data of the seagull at a frequency of 12 Hz, while the visible light and infrared cameras output high-precision observation data at a frequency of 30 Hz. The system uses interpolation or extrapolation algorithms to encrypt the 12 Hz track data in the time dimension, so that it is strictly aligned with the 30 Hz image data at each timestamp, thereby generating a set of to-be-processed data sets containing the position, speed, visible light image, and infrared image at the same time.

[0100] The data set is sent to different feature extraction branches of the multi-modal fusion model. Among them, the aligned track data is sent to a recurrent neural network branch to extract a track feature vector representing its low speed and medium maneuverability. The visible light image, which is dark in the night environment, is sent to a branch of the convolutional neural network to extract an observation feature vector with little information. The infrared thermal imaging image, which clearly shows the warm contour of the seagull, is sent to another branch of the convolutional neural network to extract an observation feature vector with rich information and significant features.

[0101] The system applies the fusion weight parameters generated under this clear night condition, such as a radar weight of 0.43, a visible light weight of 0.03, and an infrared weight of 0.54, to weight the three modal feature vectors extracted in S52. This allows the effective features from infrared and radar to be retained and emphasized, while the features from the almost ineffective visible light image are greatly weakened.

[0102] The set of weighted modal feature vectors is input into the fusion network. The cross-modal attention mechanism within the network can learn and discover the strong correlation between the "flight" behavior represented by the track feature vector and the "warm-blooded bird" morphology represented by the infrared feature vector during feature fusion, thereby giving them higher fusion weights, and finally generating a highly condensed and clearly directed comprehensive feature representation.

[0103] The comprehensive feature representation is input into the end classifier. The classifier outputs a probability distribution result after calculation, such as {bird: 98%, drone: 1%, background noise: 1%}. Based on this, the system obtains a preliminary recognition result that the target class is a bird, with a corresponding confidence of 98%.

[0104] The system performs a final compliance check of the preliminary identification result against a physical constraint database. The database contains physical characteristic constraints of birds, such as the flying speed is usually less than 100 kilometers per hour. The system compares the current flying speed of the seagull, i.e. 45 kilometers per hour, attached to the preliminary identification result, against the constraint. Since 45 kilometers per hour is significantly less than the upper limit of 100 kilometers per hour, it complies with the physical constraint, and the check passes. At this point, the system finally determines the preliminary identification result as a valid target, i.e. a final target object, and passes its data to a subsequent disposition decision step.

[0105] S6. Generating a disposition instruction based on the final target object and preset security policy data.

[0106] The preset security policy data is a set of rules and data that are used to constrain and regulate the final disposition behavior of the system, which constitutes the highest safety criterion that must be followed when the system makes decisions. The security policy data can specifically include multiple components. One part is static absolute safety area geographic information data, which pre-defines the key protected areas within the airport, such as the terminal, the apron, the fuel depot, the navigation facilities, and the permanent personnel work area, etc., in the form of a digital map, and any disposition path must not enter or pass through these areas. Another part is a dynamic set of rules for engagement, which defines under what conditions the system can or is prohibited to take a specific type of disposition action, for example, it can be specified that only when the target threat level reaches a certain threshold and is in a non-critical airspace, the physical disposition can be started; or it is specified that in any case, a high-power clearance instruction must be authorized by a human before it can be executed.

[0107] Based on the comprehensive analysis of the final target object and the above-mentioned security policy data, the disposition instruction generated by the system can be divided into several different levels and types. One is a monitoring instruction, i.e. the system does not intervene the target, but keeps a stable tracking of it and continuously reports its position and state to the general control center for situation awareness. Another is a low-power repelling instruction, i.e. the disposition unit uses a non-lethal and low-energy way to drive the target away, for example, using a low-power laser beam to irradiate in the area near the target to visually stimulate it to move away from the sensitive airspace. The last one is a high-power clearance instruction, i.e. the disposition unit uses a high-energy way to physically attack the target to directly eliminate the threat, which is usually generated only when the target constitutes the highest level of threat and meets all the safety checks and human authorization.

[0108] Specifically, in an embodiment, the S6 includes the following steps S61-S67.

[0109] S61. Based on the flight trajectory of the final target object, the category attribute, and the real-time operation state data of the airport, real-time threat level assessment is performed on the final target object to obtain a threat score.

[0110] S62. A disposal path from the disposal unit at the preset location to the final target object is predicted and generated, and the disposal path is compared with the absolute safety area defined in the preset safety policy data to perform static safety verification.

[0111] S63. When and only when the static safety verification passes, the disposal path is compared with the real-time updated airspace and ground traffic data to perform dynamic conflict analysis, and a dynamic safety permission signal is output.

[0112] S64. Based on the threat score, the result of the static safety verification, and the dynamic safety permission signal, the best disposal strategy is decided from a preset disposal strategy library.

[0113] S65. When the best disposal strategy requires physical attack, the disposal parameters for guiding the disposal unit are calculated, including at least azimuth angle, elevation angle, and target lead time.

[0114] S66. For disposal strategies requiring human intervention, the best disposal strategy and the disposal parameters are packaged into an authorization request and submitted to the human-computer interaction interface, and an affirmative authorization instruction is received.

[0115] S67. Based on the affirmative authorization instruction or disposal strategy decision without human intervention, the disposal instruction containing the disposal parameters is generated and issued to the disposal unit.

[0116] Static safety verification is a geometric path verification based on preset geographic information. In this verification, the absolute safety area is predefined as a three-dimensional spatial geometric body within the airport to mark the key area that any physical attack path is strictly prohibited from crossing or entering, such as terminal, oil depot, navigation equipment, and known personnel resident work area. The process of static safety verification is to judge the intersection or proximity of the disposal path (which can be modeled as a line segment in space) from the disposal unit to the final target object with all preset absolute safety area geometric bodies. Only when the disposal path does not interfere with all absolute safety areas in space, the verification can pass.

[0117] Dynamic conflict analysis is a predictive spatio-temporal path verification based on real-time traffic data. The analysis process requires the system to access and decode real-time air space surveillance data such as ADS-B signals and ground surveillance data such as surface surveillance radar signals. The principle of dynamic conflict analysis is that the system not only considers the current state of the handling path, but also extrapolates it in the time dimension, while predicting the motion trajectories of all legal traffic entities such as aircraft and vehicles in the vicinity of the handling path within a short time window in the future. By comparing the four-dimensional spatio-temporal trajectories of the handling path and these traffic entities, it is determined whether there is a potential collision or risk of dangerous approach.

[0118] The pre-set handling strategy library is a rule module with built-in decision logic, which stores the mapping relationship between different handling strategies and trigger conditions. The input of the decision logic is the threat score and the safety verification result, and the output is the best handling strategy. For example, it may contain the rule: if the threat score is higher than 0.9 and both safety checks pass, select the high-power removal strategy; if the threat score is between 0.5 and 0.9 and the safety check passes, select the low-power repulsion strategy; if any safety check fails, no matter how high the threat score is, the monitoring strategy is forced to be selected. Physical attack refers to a way of making a target lose its flight capability by emitting energy or physical contact. In this embodiment, the handling unit can be a set of high-energy lasers installed on a high-precision servo turntable, and the physical attack it performs is to emit a focused high-power laser beam.

[0119] Continuing the example of the previous embodiment about the night seagull, the system confirms that the final target object is a seagull, whose flight trajectory is crossing the runway extension line where a flight will take off in 2 minutes. Combined with the real-time running state data of the airport, the system performs threat assessment on the seagull and gives a threat score as high as 0.95. The system predicts and generates a handling path from the laser handling unit location to the current location of the seagull, and compares the path with the absolute safety area defined in the safety strategy data (such as the air traffic control facility beside the runway), and the verification result is no conflict, the static safety check passes. The system further compares the handling path with real-time traffic data to confirm that there is no activity trajectory of any aircraft or ground vehicle on the path, and the dynamic conflict analysis passes, outputting a dynamic safety permission signal.

[0120] The system decides high-power removal as the best disposal strategy from the disposal strategy library based on the threat score of 0.95 and the result of passing the security check. Since the strategy needs to perform physical attack, the system calculates the accurate disposal parameters required by the guide laser disposal unit according to the current speed and position of the seagull, including the target advance. Since high-power removal belongs to the strategy that needs human intervention, the system packages the target information, threat assessment, security check result and disposal parameters into an authorization request and presents it on the human-computer interaction interface of the airport operation control center. The operation control center operator issues a positive authorization instruction after reviewing the information and confirming that there is no risk. The system finally generates a disposal instruction containing accurate disposal parameters after receiving the instruction and issues it to the laser disposal unit for execution.

[0121] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example. In actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0123] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit it. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features. The modification or replacement does not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An airport all-weather bird detection method based on multi-modal adaptive fusion, characterized in that, The method comprises the following steps: S1. obtaining preliminary track data in a monitoring airspace based on radar detection; wherein the preliminary track data at least represents initial position and velocity information of a moving target; S2. inputting the preliminary track data into a super-lightweight model to generate a preliminary candidate region; wherein the super-lightweight model is obtained by pre-training based on radar data; and the preliminary candidate region is a sub-region in the monitoring airspace where a bird target is suspected to exist; S3. in response to the output of the preliminary candidate region, guiding at least one non-radar sensor to perform directional observation on the preliminary candidate region to obtain high-precision multi-modal observation data in the preliminary candidate region; S4. detecting and obtaining environmental features of the monitoring airspace, and determining fusion weight parameters of a multi-modal fusion model corresponding to the environmental features of the monitoring airspace; S5. fusing the preliminary track data, the high-precision multi-modal observation data and the fusion weight parameters, and inputting the multi-modal fusion model for multi-modal recognition to determine a final target object; S6. generating a handling instruction based on the final target object and preset safety policy data.

2. The all-weather bird detection method for airport based on multi-modal adaptive fusion according to claim 1, characterized in that, The S1 comprises the following steps: S11. receiving radar raw echo signals of the monitoring airspace to obtain an original data set containing target signals and clutter signals; S12. applying a dynamically updated clutter map to process the original data set to suppress the clutter signals in the original data set, wherein the clutter signals include ground clutter or weather clutter; S13. performing constant false alarm rate detection on the data set after suppressing the clutter to extract independent radar tracks; S14. logically associating multiple groups of radar tracks appearing in continuous multiple frames, and creating a temporary candidate track archive when a group of radar tracks meets preset track initiation logic; S15. data associating newly extracted radar tracks in subsequent scanning frames with existing candidate track archives, and updating and predicting the state vector of the successfully associated candidate track archives using a Kalman filter; S16. confirming and outputting the candidate track archive as the preliminary track data when the stable state of the candidate track archive meets the preset confirmation condition.

3. The all-weather bird detection method for airport based on multi-modal adaptive fusion according to claim 1, characterized in that, The S2 comprises the following steps: S21. dynamically determining a current danger region in the monitoring airspace defined by the real-time position of personnel or vulnerable targets; S22. comparing the position of the moving target represented by the preliminary track data with the current danger region in terms of geographical position, and if the moving target is located in the current danger region, eliminating the preliminary track data and aborting subsequent processing; S23. if the moving target is located outside the current danger region, analyzing the preliminary track data to extract flight motion features, and classifying and identifying through the super-lightweight model to obtain target category attribution; S24. when the target category attribution is a preset bird target, generating the preliminary candidate region based on the current position information of the moving target.

4. The all-weather bird detection method for airport based on multi-modal adaptive fusion according to claim 1, characterized in that, The S3 comprises the following steps: S31. controlling the optical axis of the at least one non-radar sensor to point to the preliminary candidate region; S32. Capture a wide field image of the preliminary candidate region, and identify pixel coordinates of the moving target within the wide field image; S33. Based on the pixel coordinates, adjust the optical focal length of the non-radar sensor to optically zoom in on the moving target, and establish a stable tracking of the moving target; S34. In the stable tracking state, synchronously collect visible light image data and infrared thermal imaging data of the moving target to obtain the high-precision multi-modal observation data.

5. The all-weather bird detection method for airport based on multi-modal adaptive fusion according to claim 1, characterized in that, The S4 includes the following steps: S41. Real-time quantification of the environmental characteristics of the monitored airspace to obtain a set of continuously changing physical parameters representing the current environmental state; the physical parameters at least include ambient illumination, atmospheric visibility, and precipitation rate; S42. For each data modality of the pre-trained multi-modal fusion model, based on the set of continuously changing physical parameters, calculate through a preset weight function corresponding to the data modality to determine a non-normalized reference weight for each data modality, respectively; S43. Normalize all non-normalized reference weights to generate the final fusion weight parameter.

6. The all-weather bird detection method for airport based on multi-modal adaptive fusion according to claim 1, characterized in that, The S5 includes the following steps: S51. Time stamp alignment processing of the filtered preliminary track data associated with the high-precision multi-modal observation data to generate a set of strictly synchronized time-dimension data sets to be processed; S52. Based on the to-be-processed data set, extract independent modal feature vectors for radar, visible light, and infrared thermal imaging data modalities, respectively; S53. Apply the fusion weight parameter to weight the independent modal feature vectors to generate a set of weighted modal feature vectors for representing the priority of each modal feature in the current environment; S54. Input the weighted modal feature vectors into the pre-trained fusion network and perform deep fusion through the cross-modal attention mechanism within the fusion network to generate a comprehensive feature representation; S55. Input the comprehensive feature representation into a classifier for probability classification to output a preliminary identification result containing the target category and its corresponding confidence; S56. Compare the preliminary identification result with the preset physical constraint database to perform physical constraint compliance verification, and only when the verification passes, determine the preliminary identification result as the final target object.

7. The all-weather bird detection method for airport based on multi-modal adaptive fusion according to claim 1, characterized in that, The S6 includes the following steps: S61. Based on the flight trajectory, category attribute, and real-time operation state data of the airport of the final target object, perform real-time threat level assessment on the final target object to obtain a threat score; S62. Predict and generate a disposal path from a preset disposal unit at a location to the final target object, and compare the disposal path with the absolute safety area defined in the preset safety policy data to perform static safety verification; S63. When and only when the static safety verification passes, compare the disposal path with the real-time updated airspace and ground traffic data to perform dynamic conflict analysis and output a dynamic safety permission signal; S64. Based on the threat score, the result of the static security check and the dynamic security permission signal, the best treatment strategy is decided from a preset treatment strategy library; S65. When the best treatment strategy requires physical attack, the treatment parameters for guiding the treatment unit are calculated, including at least azimuth angle, elevation angle and target advance; S66. For the treatment strategy requiring human intervention, the best treatment strategy and the treatment parameters are packaged into an authorization request and submitted to the human-computer interaction interface, and the positive authorization instruction is received; S67. Based on the positive authorization instruction or the treatment strategy decision without human intervention, the treatment instruction containing the treatment parameters is generated and sent to the treatment unit.

Images