Low-altitude target image detection and recognition early warning system based on computer vision

Abstract

This application relates to the fields of computer vision and image recognition technology, specifically disclosing a low-altitude target image detection, recognition, and early warning system based on computer vision. The system includes: multi-source heterogeneous visual acquisition, environment adaptive enhancement, spatiotemporal multi-scale feature extraction, dynamic target detection and localization, attribute discrimination and classification, motion state estimation, depth trajectory evolution prediction, threat assessment and early warning unit, and edge computing resource scheduling center. By fusing visible light and long-wave infrared modes and modeling nonlinear motion laws, it solves the problems of low recognition rate and difficulty in trajectory prediction for small targets under complex interference such as strong light and dense fog. This solution achieves accurate perception and hierarchical automated early warning of low-altitude risks, possesses high concurrency processing capabilities, significantly reduces false alarm rate and missed alarm rate, and ensures the safety of low-altitude airspace.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image recognition technology, specifically relating to a low-altitude target image detection, recognition and early warning system based on computer vision. Background Technology

[0002] With the booming development of the low-altitude economy and the widespread application of drone technology, the safety monitoring and refined management of low-altitude airspace have become an important part of public safety and airspace defense. Computer vision, as a core branch of artificial intelligence, provides crucial technical support for target perception, behavior understanding, and risk assessment in complex airspace environments through the capture, processing, and in-depth analysis of multi-dimensional image signals.

[0003] Among them, low-altitude target detection and recognition technology based on computer vision aims to accurately capture and determine the attributes of small, high-speed, or hovering low-altitude flying objects under complex background interference. This technology typically integrates high-resolution imaging units and high-performance inference algorithms, aiming to achieve automated monitoring and graded early warning of potential airspace threats by extracting the texture, contour, and motion features of targets in real time.

[0004] Existing technologies still face multiple challenges in complex low-altitude environments. These challenges primarily manifest in insufficient robustness of algorithms to strong light, dense fog, or complex terrain backgrounds, making it difficult to balance false alarm and missed alarm rates. Their feature extraction capabilities for distant, small targets are weak, easily leading to target loss or misidentification as uncontrolled objects such as birds. Furthermore, existing early warning systems suffer from limited real-time processing performance in multi-target concurrent scenarios. Uneven allocation of computing resources results in high recognition latency, and the lack of a deep prediction mechanism for the nonlinear flight trajectories of low-altitude targets makes it difficult to meet the comprehensive air defense early warning requirements of high sensitivity and timeliness. Summary of the Invention

[0005] The purpose of this invention is to provide a low-altitude target image detection, recognition and early warning system based on computer vision, which aims to solve the shortcomings of existing technologies in low recognition rate, high false alarm rate, weak real-time multi-target processing performance and lack of effective nonlinear trajectory prediction capability for low-altitude small flying targets under strong light, dense fog and complex ground interference. In this way, it can achieve accurate perception and all-round automated early warning of low-altitude airspace risks.

[0006] To achieve the above objectives, the technical solution provided by the present invention includes: The multi-source heterogeneous vision acquisition system is used to deploy sensing terminals at the perimeter of a controlled airspace. This system simultaneously acquires raw image streams covering the visible light band and the long-wave infrared band, and uses high-precision calibration parameters to spatially align and timestamp images of different modalities to ensure real-time correlation of multi-dimensional data.

[0007] The environment-adaptive enhancement processing unit receives the raw image stream and invokes preset image enhancement algorithms based on real-time detected illumination intensity and visibility parameters. In strong light environments, this unit performs adaptive histogram trimming and dynamic brightness suppression; in dense fog environments, it executes a physically-based dehazing algorithm, estimating atmospheric transmittance maps and restoring scene radiance to improve the contrast and edge sharpness of targets in the image.

[0008] The spatiotemporal multi-scale feature extraction unit is built on a deep neural network architecture. It utilizes an improved feature pyramid structure to perform layer-by-layer downsampling and feature mapping on the enhanced image. This unit extracts fine-grained information such as the contour and texture of the target in the low-level feature map, and extracts the semantic and category attributes of the target in the high-level feature map. It also introduces a temporal attention mechanism to perform differential fusion of the features of the current frame with the features of adjacent historical frames to enhance the perceptual weight of small moving targets.

[0009] The dynamic target detection and localization unit performs feature fusion processing on the extracted multi-scale features. By predicting the probability heatmap of the target's possible existence and the bounding box regression parameters, it marks the position and scale of the potential target in the image coordinate system in real time. This unit adopts an anchor-free detection mechanism, avoiding the problem of poor adaptability of traditional fixed-ratio anchor frames to low-altitude flying objects with varying aspect ratios, such as UAVs.

[0010] The refined attribute discrimination and classification unit resamples the detected candidate target regions to obtain high-resolution sub-image features. Using a trained deep classifier, it distinguishes targets into multi-rotor drones, fixed-wing aircraft, helicopters, birds, or suspended obstructions. This unit also extracts the target's physical attributes, including estimated physical size, direction of motion, and appearance texture features, and encapsulates these attributes into a target feature vector.

[0011] The nonlinear motion state estimation unit is used to establish an independent motion tracking chain for each identified target. This unit uses the Kalman filter algorithm to smooth the target's centroid coordinates and calculates the target's real-time displacement vector, acceleration, and angular velocity.

[0012] The deep trajectory evolution prediction unit, based on a long short-term memory network or a gated recurrent neural network, takes the target's motion state sequence as input and models the nonlinear flight law of the target in three-dimensional space. This unit outputs a set of predicted trajectory points for the target over the next 3 to 5 seconds and provides the confidence interval of the prediction results.

[0013] The threat classification assessment and early warning execution unit is used to perform real-time comparisons between the target's classification attributes, predicted trajectory, and pre-defined sensitive area geofences. This unit calculates a comprehensive threat index by calculating the target's estimated arrival time in the core airspace, the physical distance to the nearest controlled point, and the target's potential payload risk, and triggers corresponding level 1 to 4 early warning commands based on the threat index.

[0014] The edge computing resource scheduling center monitors the computing load of each processing unit within the system. In multi-target concurrent scenarios, this center dynamically adjusts the power consumption strategy and computing resource allocation weights of heterogeneous processors, prioritizing real-time inference tasks for high-threat targets, and optimizes the transmission efficiency of video streams between different hardware modules through zero-copy memory technology.

[0015] Furthermore, in the multi-source heterogeneous visual acquisition system, the resolution of the visible light camera is set to no less than 3840 x 2160 pixels, and the frame rate is set to no less than 60 frames per second to ensure the accuracy of capturing the motion of high-speed passing targets. The infrared imaging mode uses a focal plane array detector, with an operating wavelength range covering 8 micrometers to 14 micrometers, for identifying the heat dissipation signals of target engines or electronic equipment through thermal features at night or in low visibility conditions.

[0016] Furthermore, the environment adaptive enhancement processing unit employs a dynamic suppression method based on background modeling when dealing with complex background interference. By maintaining a global background update model, static ground features, trees, and building backgrounds are modeled at the pixel level. Background subtraction is performed on the input image before the detection process, thereby highlighting the pixel changes of moving targets and significantly reducing false alarms caused by environmental disturbances.

[0017] Furthermore, the spatiotemporal multi-scale feature extraction unit integrates a spatial channel attention module. This module automatically assigns higher weights to channels containing target feature information by calculating the correlation between channels of the feature map, while suppressing channels containing background noise. This mechanism enables the system to maintain a stable feature activation response even in long-distance detection scenarios, where the target occupies less than 10 pixels in the image.

[0018] Furthermore, the dynamic target detection and localization unit incorporates an intersection-over-union (IoU) equalization mechanism in its loss function design. This mechanism is specifically optimized for bounding box regression of small targets. By increasing the proportion of small deviations in the total loss, it prompts the neural network to iterate and update the localization accuracy of distant targets more frequently, ensuring that each target can obtain an accurate boundary description when multiple targets are detected concurrently.

[0019] Furthermore, the refined attribute discrimination and classification unit incorporates motion frequency domain feature analysis. The system extracts the frequency features of the target's morphological changes by performing a Fast Fourier Transform on the target's contour changes over 10 to 20 consecutive frames. If the frequency features conform to high-frequency vibration patterns, it is identified as a rotorcraft drone; if the frequency features conform to low-frequency fluctuation patterns, it is further identified as a bird, thereby reducing the misidentification rate between species.

[0020] Furthermore, the nonlinear motion state estimation unit employs a multi-model adaptive estimation technique. The system switches in real-time between a constant velocity model, a constant acceleration model, and a cooperative turning model based on the target's maneuverability. When the system detects that the target is performing highly agile obstacle avoidance maneuvers or experiencing sharp climbs, it automatically increases the weight of the cooperative turning model to ensure the continuity and smoothness of the tracking path.

[0021] Furthermore, the deep trajectory evolution prediction unit utilizes an attention encoder architecture. This architecture not only focuses on the target's own historical path but also analyzes the interrelationships between multiple targets through spatial topology network analysis. In formation flight or group intrusion scenarios, this unit can identify cooperative movement patterns among targets and predict the diffusion or aggregation trends of group targets, providing more macroscopic situational analysis data for the early warning system.

[0022] Furthermore, the threat classification assessment and early warning execution unit establishes a multi-dimensional threat matrix. The input variables for this matrix include: the radial distance between the target and the protected target, the radial velocity component of the target, the target's category risk coefficient, and whether the target is within the no-fly zone coordinates. The system calculates a threat score using a weighted summation algorithm. When the score exceeds the first threshold, a visual alarm is triggered; when the score exceeds the second threshold, a coordinated audio-visual early warning is triggered and automatically reported to the air traffic control platform; when the score exceeds the third threshold, the system automatically guides jamming equipment or countermeasures links towards the target's location.

[0023] Furthermore, the edge computing resource scheduling center employs task-level parallel processing logic. The system divides video frames into multiple spatial blocks, allocating them to different computing cores in heterogeneous processors for simultaneous detection and inference. Regarding memory management, the system uses a circular buffer mechanism to cyclically store nearly 500 frames of raw image data for retrospective analysis and evidence preservation in the event of a security incident.

[0024] Furthermore, the system involved in this invention also includes a self-evolutionary learning module. This module correlates the identification results with the feedback information after the warning. If a certain identification is manually confirmed as a false alarm, the system will automatically extract the sample and add it to the negative sample library, and optimize the classification model online through incremental learning technology to achieve continuous iterative improvement of the system's perception accuracy.

[0025] In this invention, the system supports dynamic switching modes for parameter configurations in different airspace environments. In complex urban environments, the system enhances the gain of the ground clutter suppression algorithm; in open suburban airspace, the system automatically increases the sampling frequency of the telephoto lens to extend the detection range. All control commands are transmitted via a high-speed serial bus, ensuring that the delay between command issuance and execution is controlled within 10 milliseconds.

[0026] All software modules involved in this invention run on a ruggedized embedded computing platform, which possesses physical characteristics of shock resistance, mold resistance, and wide temperature range, enabling it to adapt to harsh working environments ranging from -40 degrees Celsius to 85 degrees Celsius. The system is powered by dual redundant power supplies and automatically switches to the backup processing unit when an anomaly is detected in the main computing unit, ensuring continuous and uninterrupted execution of monitoring tasks.

[0027] When processing concurrent data streams from multiple targets, this invention utilizes a distributed communication architecture to broadcast early warning information to multiple mobile terminals or command centers. Each data packet is encrypted and includes high-precision timing information to ensure spatiotemporal consistency during remote collaborative defense.

[0028] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This invention addresses the limitations of environmental interference on detection accuracy at the physical perception layer by integrating a multi-source heterogeneous visual acquisition system with an environment-adaptive enhancement processing unit. Utilizing the complementary characteristics of long-wave infrared and visible light, combined with an atmospheric transmittance compensation algorithm, the system maintains stable target perception capabilities even in darkness, bright light, and dense fog with visibility below level 5, significantly reducing the false alarm rate caused by environmental factors.

[0029] This invention introduces a spatiotemporal multi-scale feature extraction and attention mechanism, effectively overcoming the challenge of sparse features for distant, small targets through the fusion of temporal information. Compared to traditional static frame-based detection techniques, this solution can extract the motion continuity features of targets in the temporal dimension, achieving a stable detection rate of over 95% for small UAV targets located more than 1 kilometer away with an image pixel ratio of less than 0.1%, greatly enhancing the system's defense depth.

[0030] This invention achieves a technological leap from passive identification to active prediction through the collaborative work of a deep trajectory evolution prediction unit and a nonlinear motion state estimation unit. The prediction model based on a long short-term memory network can capture the complex and ever-changing flight intentions of low-altitude flying objects, controlling the prediction deviation for nonlinear, highly maneuverable trajectories to within 5%. This provides a valuable lead time for threat classification and early warning, enabling security measures to shift from end-point interception to the early warning and intervention stage.

[0031] This invention achieves optimal allocation of computing resources through an edge computing resource scheduling center, solving the real-time bottleneck in multi-target concurrent scenarios. The system can simultaneously handle the tracking and identification of more than 50 independent targets, with an end-to-end processing latency of less than 50 milliseconds. This high-concurrency processing capability, combined with a multi-dimensional threat assessment matrix, ensures that even in the face of complex, group-based low-altitude threats, the system can still quickly identify core risks and provide definitive early warning feedback.

[0032] This invention constructs a comprehensive filtering mechanism based on wing vibration frequency domain analysis and dynamic background modeling, fundamentally reducing false alarms caused by birds, floating objects, or ground swaying. Through in-depth analysis of the target's physical properties and kinematic laws, the false alarm rate is reduced to less than once per 24-hour operating cycle. This extremely high identification certainty provides technical support for unmanned operation, significantly reduces the cost of manual verification, and has significant economic benefits and social security value. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of spatiotemporal multi-scale feature extraction and target detection and localization in this invention; Figure 3 This is a logical flowchart of the multi-source heterogeneous visual acquisition and environment adaptive enhancement processing in this invention; Figure 4 This is a logical flowchart of the nonlinear motion state estimation and depth trajectory evolution prediction in this invention. Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow of edge computing resource scheduling and threat classification assessment in this invention. Detailed Implementation

[0034] Example 1 Please refer to the attached document. Figure 1 This embodiment discloses a low-altitude target image detection and recognition early warning system based on computer vision. The system is built on highly integrated embedded computing hardware and multimodal sensing architecture. It aims to achieve precise control of small, high-speed and complex low-altitude flying targets in the controlled airspace through comprehensive visual perception, in-depth spatiotemporal feature mining and forward-looking trajectory prediction.

[0035] Please refer to the attached document. Figure 3The multi-source heterogeneous visual acquisition system, serving as the sensing source of the entire system, is deployed at the sensing terminal location at the perimeter of the controlled airspace. This system is not a single optical acquisition device, but rather a comprehensive sensing platform integrating visible light imaging and infrared imaging components. The visible light imaging component has a resolution of 3840 x 2160 pixels and is equipped with a large-size complementary metal-oxide-semiconductor (CMOS) photosensitive element, capable of capturing video streams with extremely high dynamic range at a sampling frequency of 60 frames per second. This combination of high resolution and high frame rate ensures that when a target sweeps across the field of view at a speed exceeding 30 meters per second, the motion blur of the target in a single frame is controlled within 3 pixels. Simultaneously, the infrared imaging component employs a focal plane array detector, with its operating wavelength strictly limited to the long-wave infrared range of 8 to 14 micrometers. The choice of this wavelength band reflects profound engineering considerations. The engines, battery packs, and electronic control modules of low-altitude aircraft generate significant thermal radiation during operation. This radiation has extremely high emissivity in the long-wave infrared band, enabling the system to extract a clear outline of the target through thermal features in scenarios where visible light is ineffective, such as at night, in backlight, or in dense fog. To achieve multi-source data fusion, the multi-source heterogeneous vision acquisition system integrates a clock synchronization controller. This controller receives nanosecond-level timing signals from the Global Positioning System (GPS) and assigns an absolute timestamp to each frame of visible light and infrared images, ensuring a time synchronization error of less than 0.5 milliseconds. Simultaneously, the system utilizes intrinsic and extrinsic parameter matrices obtained from offline calibration to perform perspective transformation and affine alignment on the two images, enabling pixel-level overlap of the same physical point in image coordinate systems of different modalities.

[0036] The environment-adaptive enhancement processing unit is connected to the output of the multi-source heterogeneous vision acquisition system. This unit incorporates an environmental parameter evaluation engine that can statistically analyze the average brightness, contrast, and global gradient distribution of the current image in real time. In bright light environments, when the environmental parameter evaluation engine detects a large overexposed area in the image, the environment-adaptive enhancement processing unit automatically activates an adaptive histogram trimming algorithm. By limiting the peak value of the histogram distribution, it prevents detail loss during brightness stretching and applies a dynamic brightness suppression operator to compress the grayscale values ​​of bright backgrounds such as the sky to a preset safe range, thereby highlighting the edges of objects in dark areas. In dense fog or low-visibility weather, this unit executes a physically-based atmospheric compensation defogging algorithm.

[0037] The mathematical and physical logic of the atmospheric compensation defogging algorithm is described in this embodiment using the following physical model: in, Represents the pixel values ​​of the observed image acquired by a multi-source heterogeneous vision acquisition system. The scene radiance representing the true radiation intensity of the target to be restored. Represents the global atmospheric light composition. This represents the atmospheric transmittance map. The environment-adaptive enhancement processing unit initially estimates the atmospheric transmittance map based on the prior laws of the dark channel, and then uses guided filtering to perform edge-preserving denoising on the map, obtaining a refined transmittance distribution. Through the inverse operation of the above formula, the system can subtract clutter interference caused by atmospheric scattering at the pixel level, significantly improving the contrast and edge sharpness of targets in the image. In addition, for complex background interference, the environment-adaptive enhancement processing unit also maintains a global background update model. This model models the pixel-level mean and variance of buildings, vegetation, and ground debris below the horizon. Before the detection process begins, this unit uses background subtraction technology to suppress static pixels in the input image that match the background model by more than 95%, allowing computational resources to be concentrated on processing pixel clusters with moving features.

[0038] Please refer to the attached document. Figure 2 The spatiotemporal multi-scale feature extraction unit is the core deep learning module of the entire recognition system. This unit is built upon an improved residual network and incorporates a feature pyramid structure. When the enhanced image enters this unit, it first undergoes a series of downsampling operators with a kernel size of 3x3 and a stride of 2, forming multiple feature levels from low to high resolution. In the lower-level feature maps, the unit retains fine-grained information about the target, such as the edge features of the rotor or the geometric cues of the fuselage. In the higher-level feature maps, by increasing the receptive field, the unit extracts the semantic attributes of the target, such as the spatial relationship between the target and surrounding clouds or buildings. To address the issue of distant targets having a very low pixel count in images, the spatiotemporal multi-scale feature extraction unit integrates a spatial channel attention module. This module performs global average pooling on each feature channel and calculates the correlation weight coefficients between channels. If a channel has a high feature response to the target, the system automatically increases the gain weight of that channel, while suppressing channels containing noise or invalid background. More importantly, this unit introduces a temporal attention mechanism, which not only processes the current frame but also retrieves historical features from the previous five frames through a recurrent convolutional structure to perform spatiotemporal differential fusion. This mechanism enables the system to quickly perceive the positional shift of small targets over continuous time. Even if a target appears as an isolated noise point in a static frame, it can be significantly activated after spatiotemporal fusion due to its motion continuity.

[0039] The dynamic target detection and localization unit receives multi-scale feature maps from the feature extraction unit. This unit abandons the traditional fixed anchor box detection logic and adopts an anchor box-less center point regression mechanism. During processing, the system generates a probability heatmap, where each pixel value represents the probability that the point belongs to the target's center point. By finding local maxima in the heatmap, the system can determine the precise location of the potential target in the image coordinate system. Simultaneously, the unit regresses the target's bounding box parameters, including width, height, and center offset, in parallel at each potential point. To improve the robustness of localization, the unit introduces an intersection-union (IoU) equalization mechanism in the loss function design. This mechanism increases the gradient proportion of small deviations in the total loss function, forcing the neural network to iterate parameters more frequently for distant, small-scale targets, ensuring that the target localization error is controlled within 2 pixels.

[0040] The refined attribute discrimination and classification unit crops and resamples the candidate regions output by the dynamic target detection and localization unit, normalizing the target region into a fixed-size high-resolution sub-image. This unit incorporates a deep classification network, whose training set covers over 1 million low-altitude flying object samples, including various types of multi-rotor drones, fixed-wing aircraft, helicopters, and different species of birds. To further reduce the false alarm rate, the refined attribute discrimination and classification unit introduces motion frequency domain feature analysis. The system extracts the target's contour change sequence over 15 consecutive frames and performs a Fast Fourier Transform.

[0041] Regarding the calculation logic for threat classification assessment, the system adopts the following multi-dimensional weighted evaluation model: in, Represents the overall threat index. Represents the reciprocal of the radial distance between the target and the controlled point. The radial velocity component representing the target, This represents the inherent risk coefficient based on the target classification results; for example, the coefficient for large fixed-wing drones is higher than that for micro birds. This represents the geofencing status of the target. The value is 1 if the target is in a no-fly zone, and 0 otherwise. , , , These are preset weights, which are dynamically adjusted based on the security level of the controlled airspace.

[0042] Please refer to the attached document. Figure 4The nonlinear motion state estimation unit establishes an independent tracking chain for each identified target. This unit not only records the target's coordinates but also uses an extended Kalman filter algorithm to perform depth estimation of the target's motion state. The estimated state variables include the target's real-time three-dimensional displacement vector, acceleration vector, and angular velocity. To cope with the potentially violent maneuvers that UAVs may perform during obstacle avoidance or reconnaissance, the nonlinear motion state estimation unit employs a multi-model adaptive estimation technique. The system simultaneously runs a constant velocity model, a constant acceleration model, and a cooperative turning model. When the residual analyzer detects that the deviation between the current tracking trajectory and the prediction of the constant velocity model exceeds a threshold, the system automatically increases the weight of the cooperative turning model, thereby ensuring that the tracking trajectory remains smooth and the target is not lost when making large-angle turns.

[0043] The deep trajectory evolution prediction unit, based on a long short-term memory network, takes the historical state sequence output by the nonlinear motion state estimation unit as input. Internally, this unit incorporates multiple recursive units capable of memorizing the target's flight intentions over the past 5 seconds. Through an attention encoder architecture, this unit not only focuses on the target's own historical path but also analyzes the cooperative motion patterns among multiple targets using a spatial topology network. In formation flight scenarios, this unit can identify the consistent spacing between multiple drones, thereby predicting the diffusion path or aggregation point of the entire group within the next 5 seconds. The prediction results include not only a set of future 3D coordinates but also a confidence interval based on a Gaussian distribution, providing a quantitative reference for subsequent defense decisions.

[0044] Please refer to the attached document. Figure 5 The threat classification assessment and early warning execution unit summarizes all the above analysis results. This unit establishes a multi-dimensional threat matrix, comparing the target's category attributes, predicted trajectory, estimated arrival time, and preset sensitive area geofences in real time. Based on the calculated comprehensive threat index, the system triggers a graded early warning process. Level 1 warning is indicated by a yellow box marking the target on the monitoring screen and an intermittent buzzing alert. Level 2 warning is indicated by a flashing red light from the visual alarm system, and the target's coordinates, type, and speed information are automatically reported to the superior air traffic control platform. Level 3 warning automatically triggers a coordinated audio-visual alarm and guides the photoelectric turntable for high-magnification continuous tracking. Level 4 warning, the highest level, automatically calculates the guidance parameters for the countermeasure link and directs jamming equipment to the target's location.

[0045] The edge computing resource scheduling center is the nerve center of the system, running on a ruggedized embedded computing platform. This platform possesses physical characteristics such as shock resistance, mold resistance, and wide temperature range, ensuring stable operation in environments ranging from -40 degrees Celsius to 85 degrees Celsius. The edge computing resource scheduling center employs task-level parallel processing logic, splitting video frames into multiple spatial blocks and allocating them to different computing cores in heterogeneous processors for simultaneous detection and inference. In terms of memory management, the system utilizes zero-copy memory technology; when video data flows between modules of acquisition, enhancement, and feature extraction, no physical memory movement is required, achieving efficient interaction only by passing memory pointers. Simultaneously, the system maintains a circular buffer, cyclically storing nearly 500 frames of raw image data. In the event of a security incident, the system can immediately rewind and export the raw high-definition video of the 8 seconds preceding the incident as evidence.

[0046] The system also includes a self-evolutionary learning module. This module establishes a closed-loop feedback mechanism, linking manual verification results with the system's automatic identification results. If a record of misidentifying a bird as a drone is marked by the operator, the self-evolutionary learning module will automatically extract the feature vector of that sample and add it to the negative sample library. The system utilizes incremental learning technology to fine-tune the classifier in the background during idle time, thereby achieving continuous iteration of perception accuracy during actual operation. When handling multi-target concurrent scenarios, the distributed communication architecture ensures that early warning information can be synchronized to multiple mobile terminals with a latency of less than 10 milliseconds. Each sent data packet is encrypted using commercial cryptographic algorithms and includes a high-precision time watermark, ensuring the spatiotemporal consistency of the entire defense system.

[0047] Example 2 Based on Example 1, this example optimizes the configuration of a computer vision-based low-altitude target image detection and recognition early warning system for special industrial scenarios such as ports and chemical plants with high-density metal structures, complex electromagnetic environments, and extremely strong background clutter.

[0048] Please refer to the attached document. Figure 3In the complex terrain of a port, the environment adaptive enhancement processing unit strengthens the weight of the dynamic background suppression algorithm. Because the metal surfaces of cranes, containers, and yard equipment produce strong specular reflections, these reflected lights often appear as high-frequency flickering in the video stream, easily causing false alarms. Therefore, the environment adaptive enhancement processing unit introduces an improved multi-Gaussian model algorithm during background modeling. This algorithm can establish 3 to 5 Gaussian distributions for each pixel, representing the sky background, moving crane booms, reflected light spots, and static buildings, respectively. By dynamically adjusting the pixel distribution weights, the system can effectively identify periodic light spot flickering as background noise, thus avoiding interference with normal production operations. Simultaneously, considering the tendency for sea fog to form on the port's sea surface, the number of iterations in the physical model defogging algorithm is set to 1.5 times the original number to ensure that the physical outline of the target can still be restored even in environments with extremely high water vapor concentrations.

[0049] Please refer to the attached document. Figure 2 In the spatiotemporal multi-scale feature extraction unit, an adaptive receptive field adjustment mechanism is introduced to address the concurrent multi-target requirements in industrial scenarios. In the image center region, typically the path of a target, the system increases the receptive field of neurons through dilated convolution operators to improve the semantic recognition accuracy of large, low-flying objects. At the image edges, dense stride convolutions are used to enhance the sensitivity to capture small-scale targets just entering the field of view. The spatial channel attention module plays a crucial role here, identifying the unique gantry crane contour features in port scenes and suppressing them as known patterns, thus allowing the system to focus more on moving, unusual targets.

[0050] In this embodiment, the dynamic target detection and localization unit employs a multi-task decoupled head architecture. This architecture processes target category classification, center point regression, and target physical attribute extraction in three parallel sub-branches. This design avoids parameter interference between different tasks. Especially when dealing with UAVs and their carried payloads, the system can simultaneously determine the center positions of the aircraft body and the payload, and judge whether there is a risk of dropping hazardous materials based on the relative displacement between them.

[0051] In this embodiment, the refined attribute discrimination and classification unit further integrates a voiceprint-assisted verification mechanism. Although primarily relying on visual information, the system simultaneously analyzes the micro-motion patterns of the target area during the recognition process. For multi-rotor drones, the rotors generate minute thermal oscillations in the infrared image during rotation, and the frequency of these oscillations is directly related to the motor speed. By analyzing the pulse response of the target's thermal infrared features on the time axis, the system can identify the operating state of its motor and strictly distinguish it from the biological movement patterns of birds.

[0052] Please refer to the attached document. Figure 4 To address the challenges of high wind speeds and complex airflow at ports, the nonlinear motion state estimation unit incorporates an external disturbance term estimation into the Kalman filter model. The system acquires real-time wind data from the port's meteorological station, using this data as control increments input into the kinematic equations. When a target deviates from its course in strong crosswinds, the system can predict its corrected trajectory, thus improving target tracking stability by over 20% even under complex airflow disturbances.

[0053] In this embodiment, the deep trajectory evolution prediction unit incorporates knowledge graph-based intent recognition. The system pre-stores the coordinates of all legal flight paths and sensitive hazard sources within the controlled airspace. When a target trajectory exhibits abnormal behavior patterns, such as circling around an oil storage tank or hovering for extended periods at the boundary of a no-fly zone, the prediction unit uses a recurrent neural network to capture these nonlinear patterns and marks potential intrusion intent on the predicted trajectory. This intent-based prediction allows the early warning system to detect threats more than 3 seconds earlier than predictions based solely on physical displacement.

[0054] Please refer to the attached document. Figure 5 The threat classification assessment and early warning execution unit implements a stricter threshold strategy in port scenarios. The system links the threat index calculation matrix with the port's security level in real time. During peak port loading and unloading operations, the first and second thresholds are lowered accordingly to increase vigilance. The early warning execution unit not only outputs audible and visual alarms but also links with the port's broadcasting system to issue warning instructions to staff in relevant areas and automatically pushes real-time video streams of the target to handheld inspection terminals, achieving multi-dimensional reach of warning information.

[0055] The edge computing resource scheduling center employs dynamic frequency adjustment technology when handling up to 100 concurrent targets. Based on the temperature and load of each processing core, the scheduling center adjusts the computing frequency in real time. When a high-threat target is detected, the system instantly migrates the highest priority computing tasks to the most powerful processing core and uses the high-speed bus to allocate exclusive bandwidth to the target's image data. Furthermore, the capacity of the circular buffer is expanded in this embodiment, enabling the storage of up to 15 minutes of compressed video, providing ample data support for complex incident investigations.

[0056] The self-evolutionary learning module focuses on suppressing false alarms in industrial scenarios. Due to the large number of birds in industrial areas, the system generates numerous suspected target records daily. Utilizing an active learning strategy, the module automatically selects 100 samples with critical confidence levels each day, prompting manual online annotation. After 30 days of operation, the system's false bird identification rate decreased from an initial 2% to below 0.05%. All control command transmissions undergo redundant verification to ensure accuracy in command execution even under strong electromagnetic interference.

[0057] Example 3 Building upon Example 1, this example discloses a specific implementation of the system in monitoring scenarios in open rural areas and along border lines. In such scenarios, the extension of detection range and the reliability of all-weather operation are the core considerations for the system.

[0058] Please refer to the attached document. Figure 3 In this embodiment, the multi-source heterogeneous visual acquisition system is equipped with a high-magnification continuous zoom telephoto lens. The visible light imaging component integrates optical image stabilization, effectively compensating for minor vibrations of the lens base in strong winds, ensuring clear imaging of targets 2 kilometers away. The infrared imaging component adds an optional cooled detector, increasing its detection sensitivity by an order of magnitude, enabling it to accurately detect temperature differences as small as 0.02 degrees Celsius between a distant aircraft and the background, even in rainy or foggy weather with minimal temperature variations. The clock synchronization controller utilizes a more advanced time synchronization protocol in this scenario, connecting to a reference clock server via a fiber optic link, improving the time alignment accuracy between multiple sensing terminals to the 10-microsecond level. This lays the foundation for cross-site multi-target handover tracking.

[0059] The environment-adaptive enhancement processing unit employs contrast-limited adaptive histogram equalization technology for large areas of sky background in suburban scenes. This technology can adjust the contrast of local blocks in the image, enhancing faint targets hidden in the clouds while avoiding noise amplification caused by excessive enhancement of the sky background. During low-light periods such as dusk or dawn, the unit automatically calls multi-frame noise reduction algorithms, using joint filtering in the spatial and temporal domains to significantly reduce thermal noise generated by the CMOS image sensor while preserving the target's motion trajectory.

[0060] Please refer to the attached document. Figure 2To address the issue of target minimization caused by long-distance detection, the spatiotemporal multi-scale feature extraction unit adds a super-resolution reconstruction module at the top of the feature pyramid. This module utilizes deep residual learning to attempt to infer the detailed contours of the target based on low-resolution feature layers, thereby achieving virtual completion of tiny targets at the feature map level and greatly enhancing the system's ability to perceive targets smaller than 5 pixels. The temporal attention mechanism is configured in long-window mode in this scenario, extracting stable motion vectors of the target at long distances by associating features from the past 30 frames, effectively filtering out visual jitter caused by air disturbances.

[0061] In this embodiment, the dynamic target detection and localization unit introduces a heatmap-guided search mechanism. The system first identifies potential heat source regions in the low-resolution infrared stream and uses these regions as prior information to guide a refined search within the corresponding spatial range using the high-resolution visible light stream. This dual-modal complementary strategy allows the system to maintain high search efficiency while not missing any potentially weak signals. Localization accuracy is further improved in this scenario through weighted median filtering of multi-frame detection results, ensuring that the localization box can be quickly and accurately locked when the target is occluded and reappears.

[0062] The refined attribute discrimination and classification unit has undergone in-depth feature library optimization for common drone types used in border surveillance. The system not only identifies the type of aircraft but also attempts to infer the material properties of the target by analyzing its reflectivity distribution. For example, the highlight features of a carbon fiber or metal aircraft under specific lighting conditions are used as a key classification attribute. The classifier integrates a hierarchical classification architecture, first determining whether it is a flying object, then whether it is a drone, and finally refining the classification to the specific number of rotors or wing type.

[0063] Please refer to the attached document. Figure 4 In this embodiment, the nonlinear motion state estimation unit focuses on enhancing three-dimensional spatial positioning capabilities. By deploying multiple sensing terminals, the system utilizes binocular or multi-view vision principles to calculate the target's slant range, azimuth, and elevation angle in real time, constructing the target's real-time trajectory in a three-dimensional geographic coordinate system. The state space of the Kalman filter is expanded to include the prediction of flight attitude angles, enabling the system to more accurately predict the target's next position when performing complex maneuvers such as dives or climbs.

[0064] In this scenario, the deep trajectory evolution prediction unit performs long-range path analysis. Due to the limited obstacles in the wilderness, the target's flight path typically follows the shortest path or a specific tactical route. Utilizing a prediction model based on the Transformer architecture, the prediction unit can capture long-term dependencies lasting over 10 seconds. Based on the target's current heading, the system automatically correlates with the terrain database to predict whether it is likely to use mountains or forests for cover during a low-altitude penetration.

[0065] Please refer to the attached document. Figure 5 The threat classification assessment and early warning execution unit introduces regional correlation logic in border scenarios. The system divides the monitored airspace into a detection zone, a warning zone, and a core zone. A Level 1 warning is triggered when a target enters the detection zone; a Level 3 warning is triggered when a target enters the warning zone and its predicted trajectory points towards the core zone. The early warning execution unit issues warnings to the target's location via a long-range, high-power loudspeaker, while simultaneously transmitting the target's precise geographic coordinates to the patrol team via a microwave link, achieving a response speed of detection-based early warning and early warning-based location.

[0066] In this scenario, the edge computing resource scheduling center employs an energy-efficient allocation strategy. Since some sensing terminals may be powered by solar panels and battery packs, the scheduling center dynamically adjusts the complexity of computing tasks based on the remaining battery power. When power is sufficient, all algorithm modules are activated to ensure maximum accuracy; in low-power mode, core detection tasks are prioritized, and the frame rate of video compression encoding is reduced to extend the system's continuous runtime. Zero-copy memory technology ensures smooth processing of high-definition video streams even on low-power processors.

[0067] In this scenario, the self-evolutionary learning module utilizes a cloud-based joint training mechanism. Each sensing terminal periodically uploads representative challenging samples to the command center server via an encrypted link. The server then performs large-scale distributed training before distributing the optimized weight file to each terminal. This distributed sensing, centralized learning, and localized execution model enables the system to exhibit extremely high self-evolution speed when facing new low-altitude threats. The entire system's hardware and software architecture has undergone rigorous hardening to ensure long-term, maintenance-free operation in harsh outdoor environments.

[0068] The system's heterogeneous processors interact via a high-speed serial bus, and all video encoding and decoding processes comply with international standards and have undergone specialized real-time optimization. Even in extreme situations with multiple concurrent targets and complex backgrounds, the total latency from target entry into view to issuing a warning command remains below 50 milliseconds, significantly improving the defense system's response window.

[0069] The system disclosed in this embodiment, through deep fusion of multi-source heterogeneous vision, combined with environmental adaptive processing and depth trajectory prediction, constructs a complete closed loop from the physical perception layer to the cognitive decision-making layer, which significantly improves the ability to control risks in low-altitude airspace and has extremely high engineering application value.

[0070] Example 4 Building upon Example 1, this example discloses a specific application scheme for the system in densely populated urban areas with high-rise buildings. In this scenario, the main challenges faced by the system are the complex changes in light and shadow caused by tall buildings, visual interference due to multipath effects, and legal constraints on privacy protection.

[0071] Please refer to the attached document. Figure 3 The multi-source heterogeneous visual acquisition system employs image sensors with an extremely wide dynamic range in urban environments. Since the light intensity difference between shaded areas and directly exposed areas can reach 100 decibels, the system uses multi-exposure fusion technology to acquire multiple images with different exposure times within a single frame, and then synthesizes them into a high-quality image with clear details in dark areas and unexposed highlights. In this scenario, the infrared imaging component is not only used to detect target heat but also to assist the environmental adaptive enhancement processing unit in performing refined background modeling by monitoring the thermal radiation characteristics of buildings.

[0072] The environment-adaptive enhancement processing unit introduces a deep learning-based semantic segmentation preprocessing module to address the cluttered backgrounds of urban streets. This module can identify building edges, windows, billboards, and pedestrian groups in images in real time. During enhancement, the system performs pixel-level anchoring of these non-spatial regions, ensuring that while improving image contrast, it avoids creating false edge interference due to excessive enhancement of background textures. The dehazing algorithm automatically adjusts parameters in urban smog environments, extracting local dark channel statistical features to achieve more targeted transmittance compensation, ensuring that the system's recognition accuracy within a 500-meter range is unaffected by light smog.

[0073] Please refer to the attached document. Figure 2 The spatiotemporal multi-scale feature extraction unit enhances the ability to handle occlusion in urban applications. When a drone is frequently obstructed by buildings while navigating between them, the temporal attention mechanism automatically switches to long-term memory mode. The system utilizes a recurrent neural network to retain the target's velocity, acceleration, and attitude information before it is obstructed. Even if the target completely disappears from view for up to 2 seconds, the unit can pre-assign feature extraction weights at the predicted target reappearance location based on the inertia of historical features, achieving continuous feature association across buildings.

[0074] In this embodiment, the dynamic target detection and localization unit employs a decoupling strategy using multi-scale detection heads. For targets flying at high altitudes, a large-scale detection head captures the global contour; for small rotorcraft gliding low between buildings, a high-resolution, small-scale detection head is used. The frameless detection mechanism ensures the system can flexibly handle the geometric deformation of the UAV at different pitch angles. A horizon reference correction is added to the localization logic, automatically correcting subtle displacements of the sensing terminal caused by environmental vibrations by identifying vertical and horizontal lines in the image, ensuring the target's calibration accuracy in the geographic coordinate system.

[0075] The refined attribute discrimination and classification unit has been specially enhanced with fingerprint recognition capabilities for common civilian drone models in urban environments. By extracting the target's appearance color ratio, number of rotors, and landing gear configuration, the system can quickly match known models in the database and associate them with their physical performance parameters, such as maximum climb rate and cruise speed. Furthermore, this unit integrates a privacy masking function, automatically blurring sensitive areas such as windows and faces outside the airspace at the image output end, ensuring compliance with privacy regulations while performing early warning and monitoring.

[0076] Please refer to the attached document. Figure 4 The nonlinear motion state estimation unit introduces a real-time correction model based on a simplified computational fluid dynamics approach to address the variable airflow environment in cities. The system combines a city building distribution map to calculate the potential trajectory deviation caused by the funneling effect between buildings, and incorporates this as a compensation parameter into the Kalman filter. This enables the system to provide stable centroid coordinate estimations even when the target experiences momentary attitude instability due to gusts of wind, improving the tracking success rate by more than 15% compared to traditional methods.

[0077] In this embodiment, the deep trajectory evolution prediction unit utilizes a path planning algorithm based on graph convolutional networks. The system abstracts the urban airspace as a topological graph containing obstacle nodes and feasible path edges. The prediction unit analyzes the target's current trajectory to determine whether it is flying along the street centerline or exhibiting a trend towards a specific high-level target. This trajectory evolution prediction based on topological constraints can significantly compress the prediction solution space and improve the accuracy and real-time performance of the prediction.

[0078] Please refer to the attached document. Figure 5The threat classification assessment and early warning execution unit has established a more complex weighted model in urban scenarios. The distance between the target and densely populated areas, the distance to critical power facilities, and whether it is located at a core waypoint are all included in the threat index calculation. When a Level 4 warning is triggered, the early warning execution unit will coordinate with the city's low-altitude management system to automatically send avoidance instructions to all legal aircraft in the area. At the same time, the system pushes the warning information to nearby patrol command vehicles through the high-speed interface of the edge computing resource scheduling center, realizing integrated ground-air coordinated interception.

[0079] In high-concurrency urban environments, the edge computing resource scheduling center utilizes task slicing technology. Each processor core is responsible for processing video streams from different sectors, and seamless target transfer tracking is achieved between cores through shared video memory. When a target moves from the field of view of sensing terminal A into the field of view of sensing terminal B, the scheduling center directly transfers the target's state vector and feature library index using zero-copy technology, ensuring the continuity of cross-terminal tracking. The system is equipped with dual redundant power supplies and an automatic fault switching mechanism, guaranteeing uninterrupted monitoring tasks even under complex urban electromagnetic interference or partial power outages.

[0080] The self-evolutionary learning module focuses on learning recognition patterns under complex lighting conditions during urban operations. By automatically classifying false alarm samples from different time periods such as morning, noon, and dusk, the system generates a time-adaptive classifier weight set. Under different lighting angles, the system automatically loads corresponding model parameters to ensure consistent recognition accuracy throughout the day. All these complex processing steps are completed within milliseconds, ensuring that the system can provide accurate early warning responses at the first moment of a low-altitude safety incident.

[0081] In summary, this invention constructs a complete low-altitude target detection and early warning system through multi-source heterogeneous sensing, environmental adaptive enhancement, spatiotemporal feature mining, and intelligent threat assessment. It can effectively address security challenges in various complex scenarios and has significant technical advantages and broad application prospects.

Claims

1. A low-altitude target image detection, recognition, and early warning system based on computer vision, characterized in that, include: A multi-source heterogeneous vision acquisition system is used to simultaneously acquire raw image streams covering the visible light band and the long-wave infrared band, and to use high-precision calibration parameters to spatially align and timestamp images of different modalities. An environment adaptive enhancement processing unit is used to receive the original image stream and call a preset image enhancement algorithm based on the real-time detected light intensity and visibility parameters. In a strong light environment, it performs adaptive histogram trimming and dynamic brightness suppression, and in a dense fog environment, it performs a defogging algorithm based on a physical model. The spatiotemporal multi-scale feature extraction unit is built on a deep neural network architecture. It uses an improved feature pyramid structure to perform layer-by-layer downsampling and feature mapping on the enhanced image, and introduces a temporal attention mechanism to perform differential fusion of the current frame features and the features of adjacent historical frames. The dynamic target detection and localization unit is used to perform feature fusion processing on the extracted multi-scale features. By predicting the probability heatmap of the possible existence of the target and the bounding box regression parameters, the position and scale of the potential target are marked in real time in the image coordinate system, and an anchorless detection mechanism is adopted. The refined attribute discrimination and classification unit is used to resample the detected candidate target regions to obtain high-resolution sub-image features. The deep classifier distinguishes the targets into multi-rotor drones, fixed-wing aircraft, helicopters, birds or suspended interference objects, and extracts the physical attributes of the targets and encapsulates them into target feature vectors. The nonlinear motion state estimation unit is used to establish an independent motion tracking chain for each identified target, and uses the Kalman filter algorithm to smooth the target's centroid coordinates and calculate the target's real-time displacement vector, acceleration and angular velocity. The deep trajectory evolution prediction unit is used to take the target's motion state sequence as input based on the long short-term memory network, model the nonlinear flight law of the target in three-dimensional space, and output the predicted trajectory point set and confidence interval of the target in the future time period. The threat classification assessment and early warning execution unit is used to compare the target's classification attributes, predicted trajectory and preset sensitive area geofence in real time, calculate the comprehensive threat index and trigger the corresponding classification early warning command based on the threat index; The edge computing resource scheduling center is used to monitor the computing load of each processing unit within the system and dynamically adjust the power consumption strategy and computing resource allocation weight of heterogeneous processors in multi-objective concurrent scenarios. The self-evolutionary learning module is used to associate the recognition results with the feedback information after the warning. If a recognition is manually confirmed as a false alarm, the sample is automatically extracted and added to the negative sample library. The classification model is optimized online through incremental learning technology.

2. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The environment adaptive enhancement processing unit also maintains a global background update model, which performs pixel-level mean and variance modeling for static ground features, trees, and building backgrounds.

3. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The spatiotemporal multi-scale feature extraction unit extracts the contour and fine-grained texture information of the target in the low-level feature map, and extracts the semantic and category attribute information of the target in the high-level feature map.

4. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The dynamic target detection and localization unit determines the precise location of potential targets in the image coordinate system by finding local maxima points in the heatmap, and regresses the bounding box parameters of the target in parallel at each potential point.

5. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The dynamic target detection and localization unit introduces an intersection-union ratio equalization mechanism in the loss function design, and performs special optimization for the bounding box regression of small targets.

6. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The refined attribute discrimination and classification unit extracts frequency features of morphological changes by performing a fast Fourier transform on the contour changes of the target in consecutive frames to help determine the target type.

7. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The nonlinear motion state estimation unit switches between constant speed model, constant acceleration model and cooperative turning model in real time according to the target's maneuverability.

8. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The deep trajectory evolution prediction unit analyzes the mutual influence relationships between multiple targets through spatial topology network analysis, and identifies the cooperative motion patterns between targets in formation flight or group invasion scenarios.

9. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The threat classification assessment and early warning execution unit establishes a multi-dimensional threat matrix. The input variables include the radial distance between the target and the protected target, the radial velocity component of the target, the target's category risk coefficient, and whether the target is within the coordinates of the no-fly zone.

10. The low-altitude target image detection and recognition early warning system based on computer vision according to claim 1, characterized in that, The edge computing resource scheduling center optimizes the transmission efficiency of video streams between different hardware modules through zero-copy memory technology.

Images