Multi-dimensional anti-bird intelligent identification method and system based on thermal imaging

Abstract

The invention discloses a multi-dimensional anti-bird intelligent recognition method and system based on thermal imaging, and relates to the technical field of intelligent monitoring and ecological protection. Multi-modal data is collected through thermal imaging, visible light, millimeter wave radar and a voiceprint sensor, after PTP protocol synchronization and Kalman filtering preprocessing, 3-5-second tracks of birds are predicted by using an LSTM network, and the anti-bird intelligent recognition method and system based on the thermal imaging are obtained. And the cross-modal features are fused through a Transform architecture, so that 95% of classification accuracy is realized. A bird repelling strategy is generated in real time through edge calculation, and the model is updated through cloud federal learning. The system integrates an oil-electric hybrid unmanned aerial vehicle and a ground device, supports dynamic path planning based on a thermodynamic diagram and differentiated repelling of directional sound waves, laser stroboflash and the like, has the night recognition accuracy rate of 92% and the bird repelling response time of 0.8 second, and is suitable for scenes of electric power, airports and the like. Through multi-dimensional perception, dynamic modeling and eco-friendly expelling, the problems of poor environmental adaptability, single strategy and the like of a traditional scheme are solved, and the anti-bird efficiency and the ecological safety are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent monitoring and ecological protection, and in particular relates to a multi-dimensional bird prevention intelligent identification method and system based on thermal imaging. Background Art

[0002] Traditional bird deterrent technologies rely on single sensors (such as visible light cameras or ultrasonic devices), resulting in poor environmental adaptability, weak nighttime recognition capabilities, and a single bird deterrent strategy. For example, the recognition accuracy of visible light cameras drops significantly in low light or rainy or foggy weather, while traditional sonic bird deterrents can easily cause birds to adapt. Existing solutions lack the ability to integrate multimodal data and dynamically adjust strategies, making it difficult to cope with the diversity and behavioral changes of birds in complex scenarios. This results in high false alarm rates, low deterrent efficiency, and potential negative impacts on the ecological environment. Summary of the Invention

[0003] The present invention aims to provide a multi-dimensional bird prevention intelligent identification method and system based on thermal imaging to solve the problems mentioned in the above background technology.

[0004] In order to achieve the purpose of the present invention, the present invention discloses a multi-dimensional bird prevention intelligent identification method based on thermal imaging, comprising the following steps:

[0005] Step 1: Multimodal data collection: Use thermal imaging sensors to obtain bird body temperature distribution data, visible light cameras to collect bird texture features, millimeter wave radar to track the three-dimensional trajectory of birds, and voiceprint sensors to collect bird sound signals;

[0006] Step 2: Spatiotemporal synchronization preprocessing: Time synchronization of multi-sensor data based on the PTP protocol, combined with the Kalman filter algorithm to eliminate inter-sensor delay errors, dynamic range optimization and noise reduction processing of thermal imaging data, detail enhancement and target detection of visible light images, point cloud data generation for millimeter-wave radar data, and Mel-frequency cepstral coefficients extraction for voiceprint signals;

[0007] Step 3: Dynamic thermal imaging modeling: Build a spatiotemporal thermal distribution model of bird flight, use the LSTM network to predict the flight trajectory for the next 3-5 seconds, and use the Kalman filter algorithm to correct the predicted trajectory. When the trajectory error is less than 5 pixels, trigger the bird repelling response.

[0008] Step 4: Multimodal feature fusion: Encode the thermal imaging temperature features, voiceprint frequency features, and visible light texture features into a token sequence, implement cross-modal feature interaction through the Transformer architecture, and generate a fused feature vector.

[0009] Step 5: Edge-cloud collaborative decision-making: The edge computing device processes multimodal data in real time and generates a preliminary bird-repelling strategy. The cloud server aggregates the encrypted model gradients uploaded by the edge nodes based on the federated learning framework and updates the global recognition model.

[0010] Step 6: Execute intelligent bird repellent strategies; based on the bird species, number, and predicted trajectory, dynamically select strategies such as directional acoustic repellent, laser stroboscopic deterrence, or natural enemy heat signal simulation to form differentiated bird repellent solutions.

[0011] Furthermore, in step 2, the dynamic range optimization of thermal imaging data uses AGC4.0 technology, and the noise reduction processing uses the 3DDNR algorithm; the visible light image detail enhancement uses DDE technology, and the target detection uses the YOLOv8 algorithm.

[0012] Furthermore, in step 3, the hidden layer dimension of the LSTM network is 64, the input sequence length is 4 frames, and the prediction step corresponds to a time of 3-5 seconds; the state transition matrix of the Kalman filter algorithm includes position, velocity, and temperature change rate parameters.

[0013] Furthermore, in step 4, the Transformer architecture contains 6 encoder layers, each encoder layer adopts an 8-head self-attention mechanism with a single-head dimension of 32, and weighted fusion of cross-modal features is achieved through the multi-head attention mechanism.

[0014] Furthermore, in step 5, the edge nodes of the federated learning framework add Laplace noise to the local gradient to achieve differential privacy protection, with a privacy budget parameter ε≤1 and a model update cycle ≤24 hours.

[0015] In order to achieve the purpose of the present invention, the present invention also discloses a multi-dimensional bird-proofing intelligent identification system based on thermal imaging, including a multimodal perception module, an edge computing unit, a cloud server and an intelligent bird-repelling execution module; the multimodal perception module integrates a thermal imaging sensor, a visible light camera, a millimeter-wave radar and a voiceprint sensor, and is configured with a PTP protocol synchronization circuit to support operation in a wide temperature environment of -40°C to 70°C; the edge computing unit is equipped with a computing chip, a built-in LSTM+ Kalman filter trajectory prediction algorithm, a Transformer multimodal fusion model and a dynamic threshold adjustment module to realize real-time processing of multimodal data; the cloud server deploys a federated learning platform to store a bird voiceprint library, a thermal signal feature library and a bird-repelling strategy database, and supports encrypted gradient aggregation and global model updates; the intelligent bird-repelling execution module includes a hybrid drone and a ground-fixed bird-repelling device, and the drone is equipped with a lightweight thermal imaging module and a directional sound wave emission device to support dynamic path planning based on a thermal signal density heat map.

[0016] Furthermore, in the multimodal perception module, the thermal imaging sensor has a temperature measurement accuracy of ±0.05°C; the millimeter-wave radar supports three-dimensional trajectory tracking within 200 meters, the point cloud data update frequency is ≥10Hz, and the voiceprint sensor supports broadband signal acquisition.

[0017] Furthermore, in the edge computing unit, the dynamic threshold adjustment module adopts the formula:

[0018] θ t =θ0+αG t +βVar(T env );

[0019] Among them, the initial threshold θ0 = 35°C, the temperature gradient coefficient α = 0.8, and the ambient temperature variance coefficient β = 0.5, which realizes dynamic filtering of ambient heat source interference.

[0020] Furthermore, in the intelligent bird-repellent execution module, the hybrid drone has a flight time of ≥5 hours, a wind resistance level of ≥8, supports operation in a temperature difference environment of -20℃ to 50℃, and is equipped with a directional sound wave transmitter covering a frequency range of 20-200kHz and a sound pressure level of ≥120dB.

[0021] Furthermore, in the cloud server, the bird voiceprint library pre-stores MFCC features of ≥50 bird species; the cloud server implements model updates through federated learning.

[0022] Compared with existing technologies, the significant advancements of this invention are: 1) Multimodal data fusion to enhance complex environment recognition capabilities: Through the coordinated collection of thermal imaging, visible light, millimeter-wave radar, and voiceprint sensors, a full-dimensional dataset of bird body temperature distribution, texture features, three-dimensional trajectory, and voiceprint signals is constructed. Thermal imaging achieves a temperature detection accuracy of ±0.05°C at night and in low-light environments. Combined with DDE detail enhancement of visible light images (20% improvement in edge clarity) and YOLOv8 small target detection (85% accuracy), the bird recognition accuracy in complex weather (such as light rain + level 6 wind) is increased from 65% of traditional solutions to 89%, breaking through the environmental limitations of a single sensor; 2) Dynamic trajectory prediction and precise expulsion to shorten response time: Utilizing an LSTM network to model a 4-frame historical thermal signal sequence, combined with a Kalman filter to correct the trajectory prediction error to <5 pixels, the bird's flight path can be predicted 3-5 seconds in advance. The intelligent bird repellent execution module dynamically selects a strategy based on the prediction results: small birds are repelled with 20-40kHz directional sound waves at a distance of 50 meters; large birds of prey are deterred by 40°C heat simulation and predator calls; and flocking birds are repelled in a circular manner using a multi-drone sound matrix. The overall bird repellent response time has been reduced from 5 seconds to 0.8 seconds, and line tripping incidents in power scenarios have been reduced from 3 to 0.5 per month. 3) Federated learning and privacy protection enhance system scalability: Edge nodes perform local training using the MobileNetV3+CBAM model. The gradients are encrypted and uploaded to the cloud after Laplace noise is added, enabling federated learning updates without leaving the device. Cloud computing pressure is reduced by 70% compared to traditional centralized training. It also supports the expansion of the voiceprint library for 50 or more bird species, improving the ability to identify new species by 60% and significantly enhancing system scalability. 4) Hardware integration and ecosystem optimization to adapt to multi-scenario needs: The multimodal perception module supports wide operating temperature ranges from -40°C to 70°C. The hybrid drone has a flight time of 5 hours and is resistant to winds of level 8. It is equipped with a lightweight thermal imaging module and directional sonic emitters, combined with ground-mounted infrared radiation panels, to form an "air-ground" coordinated protection network. Bird strikes at airports have been reduced from 20 to 4 per year, and the adaptability period of the bird repellent strategy has been extended from 7 days to over 30 days, avoiding the ecological pollution problems associated with traditional chemical bird repellents.

[0023] In order to more clearly illustrate the functional characteristics and structural parameters of the present invention, further description is given below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1This is a flow chart of a multi-dimensional bird prevention intelligent identification method based on thermal imaging;

[0026] Figure 2 This is a principle block diagram of a multi-dimensional bird prevention intelligent identification system based on thermal imaging. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0028] The present invention provides a multi-dimensional bird prevention intelligent identification method based on thermal imaging, the specific implementation of which is as follows:

[0029] Step 1. Multimodal data acquisition: Obtain bird body temperature distribution data through thermal imaging sensors, collect bird texture features through visible light cameras, track bird three-dimensional trajectories through millimeter-wave radars, and collect bird singing signals through soundprint sensors; adopt a multimodal data acquisition method that comprehensively utilizes thermal imaging, visible light, millimeter-wave radars, and soundprint sensors to obtain bird information from multiple dimensions. Thermal imaging sensors can accurately capture bird body temperature distribution at night or in low-light environments, visible light cameras can provide bird texture features, millimeter-wave radars can track bird three-dimensional trajectories, and soundprint sensors can collect bird singing signals. This multi-dimensional data acquisition greatly improves the comprehensive perception of bird characteristics, overcomes the limitations of a single sensor in different environments and scenarios, and lays a solid foundation for the subsequent accurate identification of bird species, behaviors, and trajectories, thereby improving the reliability and effectiveness of the entire bird prevention system;

[0030] Step 2: Spatiotemporal Synchronization Preprocessing: Nanosecond-level time synchronization of multi-sensor data is performed based on the PTP protocol. Inter-sensor delay errors are eliminated using the Kalman filter algorithm. Dynamic range optimization and noise reduction are performed on thermal imaging data. Detail enhancement and target detection are performed on visible light images. Point cloud data is generated from millimeter-wave radar data. Mel-frequency cepstral coefficients (MFCCs) are extracted from voiceprint signals. Nanosecond-level time synchronization based on the PTP protocol ensures high temporal consistency of multi-sensor data. Combined with the Kalman filter algorithm to eliminate delay errors, data collected by different sensors accurately corresponds to the bird's state at the same moment, avoiding information bias caused by time asynchrony. Dynamic range optimization and noise reduction are performed on thermal imaging data, enhancing its quality and more clearly presenting the bird's body temperature distribution. Detail enhancement and target detection are performed on visible light images, improving the accuracy of bird texture recognition. Point cloud data is generated from millimeter-wave radar data, enabling a more accurate depiction of the bird's three-dimensional trajectory. MFCC features are extracted from voiceprint signals, facilitating accurate identification of bird calls. These preprocessing operations significantly improve the availability and accuracy of data, providing a high-quality data foundation for subsequent intelligent recognition;

[0031] Step 3. Dynamic thermal imaging modeling: Construct a spatiotemporal thermal distribution model of bird flight, use the LSTM network to predict the flight trajectory for the next 3-5 seconds, and use the Kalman filter algorithm to correct the predicted trajectory. When the trajectory error is less than 5 pixels, trigger the bird-repelling response; construct a spatiotemporal thermal distribution model of bird flight, which can comprehensively consider the thermal distribution changes of birds in space and time, and more accurately reflect the flight status of birds. Using the LSTM network to predict the flight trajectory for the next 3-5 seconds enables the system to predict the flight path of birds in advance, which buys valuable time for bird-repelling operations. Combining the Kalman filter algorithm to correct the predicted trajectory further improves the accuracy of trajectory prediction. Triggering the bird-repelling response when the trajectory error is less than 5 pixels ensures the timeliness and accuracy of the bird-repelling operation, and can respond quickly when birds approach the protected area, effectively reducing the invasion of birds on the target area;

[0032] Step 4, multimodal feature fusion: uniformly encode thermal imaging temperature features, voiceprint frequency features, and visible light texture features into a token sequence, and implement cross-modal feature interaction through the Transformer architecture to generate a fused feature vector; uniformly encode the features of different modalities into a token sequence, and implement cross-modal feature interaction through the Transformer architecture, which can fully explore the association and complementary information between different modal features. Thermal imaging temperature features, voiceprint frequency features, and visible light texture features each have their own characteristics. The fused feature vector generated by fusing these features contains richer and more comprehensive bird information, greatly improving the accuracy and reliability of bird identification. Compared with single-modality recognition methods, multimodal feature fusion can effectively cope with the challenges brought by complex environments and changes in bird behavior, and improve the robustness and adaptability of the system;

[0033] Step 5. Edge-cloud collaborative decision-making: The edge computing device processes multimodal data in real time and generates a preliminary bird-repelling strategy. The cloud server aggregates the encrypted model gradients uploaded by the edge nodes based on the federated learning framework and updates the global recognition model. The edge computing device processes multimodal data in real time and generates a preliminary bird-repelling strategy, which reduces data transmission delays, can quickly respond to the appearance of birds, and take timely bird-repelling measures. The cloud server aggregates the encrypted model gradients uploaded by the edge nodes based on the federated learning framework and updates the global recognition model. This not only ensures the privacy and security of the data, but also makes full use of the data and computing resources of each edge node to continuously optimize the global recognition model. This edge-cloud collaborative decision-making method realizes the rational allocation and efficient utilization of computing resources, improves the overall performance and scalability of the system, and can also adapt to the bird prevention needs in different scenarios.

[0034] Step 6: Intelligent Bird Repellent Strategy Execution: Based on the bird species, population, and predicted trajectory, strategies such as directional acoustic repellent, laser stroboscopic deterrent, or natural enemy heat signature simulation are dynamically selected to form differentiated bird repellent solutions. Dynamic selection of bird repellent strategies based on bird species, population, and predicted trajectory allows for customized bird repellent solutions to be developed for different bird species. Strategies such as directional acoustic repellent, laser stroboscopic deterrent, and natural enemy heat signature simulation each have their own unique characteristics, suitable for different bird species and scenarios. By forming differentiated bird repellent solutions, the targeted and effective nature of bird repellent is improved, avoiding the limitations of traditional single-method bird repellent methods. This allows for more effective bird repellent, protecting target areas from bird infestations, and reducing the impact on the environment and other organisms.

[0035] Furthermore, in the spatiotemporal synchronization preprocessing, AGC4.0 technology is used to optimize the dynamic range of thermal imaging data, and the 3D DNR algorithm is used for noise reduction processing; DDE technology is used to enhance the details of visible light images, and the YOLOv8 algorithm is used for target detection; AGC4.0 technology increases the contrast of bird body temperature areas by 60%, and combined with the 3D DNR algorithm, the noise level is reduced by 70%, significantly enhancing the thermal signal characteristics; DDE technology improves the edge clarity of visible light images by 20%, and the YOLOv8 algorithm has a detection accuracy of 85% for small birds (such as sparrows), which is 25% higher than the traditional YOLOv5, effectively solving the problem of missed detection of small targets.

[0036] Specifically, in dynamic thermal imaging modeling, the hidden layer dimension of the LSTM network is 64, the input sequence length is 4 frames, and the prediction step corresponds to a time of 3-5 seconds; the state transition matrix of the Kalman filter algorithm contains position, speed and temperature change rate parameters; the LSTM network is modeled through 4 frames of historical data, combined with the state correction of the Kalman filter, so that the trajectory prediction error is less than 5 pixels, and the prediction lead time is 3-5 seconds, reserving sufficient response time for the bird-repelling strategy; the state transition matrix introduces the temperature change rate parameter, which can synchronously track changes in the physiological state of birds and improve the accuracy of behavior prediction.

[0037] Specifically, in multimodal feature fusion, the Transformer architecture contains 6 encoder layers, each encoder layer adopts an 8-head self-attention mechanism with a single-head dimension of 32, and realizes weighted fusion of cross-modal features through the multi-head attention mechanism; the 6-layer encoder and the 8-head self-attention mechanism realize deep interaction of cross-modal features, which improves the expression ability of the fusion features of thermal imaging temperature, voiceprint frequency, and visible light texture by 40%, and the accuracy of bird classification in complex environments reaches 95%, which is more than 25% higher than the single-modal algorithm.

[0038] Specifically, in edge-cloud collaborative decision-making, the edge nodes of the federated learning framework add Laplace noise to the local gradient to achieve differential privacy protection, with a privacy budget parameter ε≤1 and a model update cycle ≤24 hours. Laplace noise is added to the edge node gradient to ensure that data does not leave the device under the constraint of a privacy budget ε≤1, complying with privacy regulations such as GDPR. The model update cycle is shortened to 24 hours, reducing cloud computing pressure by 70%, while maintaining a recognition accuracy drop of <1%, achieving a balance between privacy protection and model iteration efficiency.

[0039] A multi-dimensional bird-repellent intelligent identification system based on thermal imaging, including a multimodal perception module, an edge computing unit, a cloud server, and an intelligent bird-repellent execution module; the details are as follows:

[0040] Multimodal perception module: Integrates thermal imaging sensors, visible light cameras, millimeter-wave radars, and voiceprint sensors, is equipped with a PTP protocol synchronization circuit, and supports operation in a wide temperature range of -40°C to 70°C. The multimodal perception module integrates a variety of sensors and can perceive bird information from multiple dimensions, comprehensively covering characteristics such as bird body temperature, texture, trajectory, and voiceprint, thereby improving the accuracy and reliability of bird recognition. The configuration of the PTP protocol synchronization circuit ensures the time synchronization of the data from each sensor, so that the collected data can accurately reflect the status of the bird at the same moment. Supporting operation in a wide temperature range of -40°C to 70°C allows the system to operate normally under extreme temperature conditions and adapt to various complex outdoor environments, expanding the application range of the system and improving the stability and reliability of the system.

[0041] Edge computing unit: Equipped with the NVIDIA Jetson AGX Orin computing chip, it has built-in LSTM+Kalman filter trajectory prediction algorithm, Transformer multimodal fusion model and dynamic threshold adjustment module to realize real-time processing of multimodal data; equipped with the NVIDIA Jetson AGX Orin chip, it has powerful computing capabilities and can quickly process multimodal data. The built-in LSTM+Kalman filter trajectory prediction algorithm can accurately predict the flight trajectory of birds; the Transformer multimodal fusion model can achieve deep fusion of different modal features to improve the accuracy of bird recognition; the dynamic threshold adjustment module can adaptively adjust the recognition threshold according to environmental changes, enhancing the robustness of the system. Through the integration of these algorithms and modules, the edge computing unit can realize real-time processing of multimodal data, quickly generate preliminary bird-repelling strategies, reduce data transmission and processing delays, and improve the response speed of the system;

[0042] Cloud server: deploys a federated learning platform to store bird voiceprint libraries, thermal signal feature libraries, and bird repellent strategy databases, and supports encrypted gradient aggregation and global model updates; cloud servers deploy a federated learning platform that can aggregate encrypted model gradients uploaded by each edge node, update the global recognition model while ensuring data privacy, and continuously improve the performance and adaptability of the model. Storing bird voiceprint libraries, thermal signal feature libraries, and bird repellent strategy databases provides the system with rich reference data, which helps improve the accuracy of bird identification and the effectiveness of bird repellent strategies. Supporting encrypted gradient aggregation and global model updates enables the system to utilize large amounts of data for model training and optimization, while protecting the security and privacy of user data and enhancing the scalability and intelligence of the system.

[0043] The Intelligent Bird Repellent Execution Module includes a hybrid drone and a ground-based fixed-mounted bird repellent device. The drone is equipped with a lightweight thermal imaging module (weighing ≤500g) and a directional acoustic wave transmitter, supporting dynamic path planning based on thermal signal density heat maps. Combining the hybrid drone and the ground-based fixed-mounted bird repellent device, the Intelligent Bird Repellent Execution Module enables all-round, multi-level bird repellent. The drone, equipped with a lightweight thermal imaging module and directional acoustic wave transmitter, ensures flight flexibility and endurance while also capturing bird heat signatures in real time and emitting directional acoustic waves to repel them. Dynamic path planning based on thermal signal density heat maps allows the drone to automatically plan flight paths based on bird distribution, improving the efficiency and targeted nature of bird repellent. The ground-based fixed-mounted bird repellent device provides continuous bird repellent protection in a fixed area, working in conjunction with the drone to form an efficient and intelligent bird repellent network.

[0044] Specifically, in the multimodal perception module, the thermal imaging sensor uses the FLIR Boson320×240 module with a temperature measurement accuracy of ±0.05°C; the millimeter-wave radar uses the TIAWR1843 chip, which supports three-dimensional trajectory tracking within 200 meters and a point cloud data update frequency of ≥10Hz; the voiceprint sensor uses the KnowlesSPH0641 module, which supports broadband signal acquisition, ensuring continuous monitoring of high-speed flying targets.

[0045] Specifically, in the edge computing unit, the dynamic threshold adjustment module adopts the formula:

[0046] θ t =θ0+αG t +βVar(T env );

[0047] Among them, the initial threshold θ0 = 35°C, the temperature gradient coefficient α = 0.8, and the ambient temperature variance coefficient β = 0.5, realizing dynamic filtering of environmental heat source interference; the dynamic threshold formula introduces temperature gradient and environmental variance parameters, so that the threshold can be adaptively adjusted with the ambient temperature, and the false alarm rate is reduced from 15% / hour to 2% / hour, effectively filtering out heat source interference during sunrise / sunset periods.

[0048] Specifically, in the intelligent bird-repellent execution module, the hybrid drone has a flight time of ≥5 hours, a wind resistance level of ≥8, supports operation in a temperature difference environment of -20℃ to 50℃, and is equipped with a directional sound wave transmitter covering a frequency range of 20-200kHz and a sound pressure level of ≥120dB; it can effectively repel birds within 50 meters and avoid the ecological pollution of traditional chemical bird repellents.

[0049] Specifically, in the cloud server, the bird voiceprint library pre-stores the MFCC features of ≥50 bird species; when the cloud server updates the model through federated learning; the cloud computing pressure is reduced by more than 70% compared with traditional centralized training, significantly improving the system scalability and making it suitable for large-scale distributed deployment scenarios.

[0050] Example

[0051] See also Figure 1-2 In one embodiment, a multi-dimensional bird prevention intelligent identification method and system based on thermal imaging includes:

[0052] 1. System hardware deployment implementation steps:

[0053] 1. Multimodal sensor array installation:

[0054] 1.1 Thermal Imaging Sensor: Model: FLIR Boson 320×240 (uncooled microbolometer); Installation Location: 5-10 meters above the ground (power tower / airport control tower / farmland support), with the lens facing an area where birds are most active; Parameters: 10Hz frame rate, temperature measurement range -40°C to 80°C, accuracy ±0.05°C, and support for AGC4.0 automatic gain control;

[0055] 1.2 Visible light camera: Model: Sony IMX585 (48 megapixels); Mounting method: Coaxially fixed with the thermal imaging sensor, error ≤ 0.5°; Parameter configuration: 1080P@30fps, support for DDE detail enhancement technology, built-in IR-CUT dual filter switching;

[0056] 1.3mm-wave radar: Model: TIAWR1843 (76-81GHz); Installation angle: horizontal ±45° scanning, vertical ±20° field of view; Parameter configuration: maximum detection range 200 meters, point cloud density 5000 points / second, update frequency 20Hz;

[0057] 1.4 Voiceprint Sensor: Model: Knowles SPH0641 (MEMS microphone); Installation Requirements: Facing an open space, ≥1 meter from a heat source; Parameter Configuration: Acquisition frequency 48kHz, dynamic range 80dB, support for 20-200kHz broadband signals.

[0058] 2. Edge computing device deployment:

[0059] 2.1 Hardware Platform: NVIDIA JetsonAGX Orin (6-core ARM Cortex-A78AE, 200TOPS computing power);

[0060] 2.2 Installation environment: IP67 protection grade chassis, built-in temperature-controlled fan (operating temperature -40℃~70℃);

[0061] 2.3 Interface Configuration: Sensor Interface: Gigabit Ethernet (PTP protocol synchronization), USB3.2 (thermal imaging data), SPI (millimeter wave radar); Communication Module: 5G / 4G module (upload data to the cloud), Wi-Fi6 (local configuration);

[0062] 3. Installation of intelligent bird repellent equipment:

[0063] 3.1 Hybrid UAV: ​​Model: Customized DLE70MU platform (5-hour flight time, 3kg payload); Payload configuration: Lightweight thermal imaging module (200g), directional acoustic wave transmitter (120dB, 20-200kHz), laser strobe device (520nm, compliant with IEC60825-1 Class 1); Flight parameters: Maximum speed 20m / s, wind resistance level 8, support for RTK centimeter-level positioning;

[0064] 3.2 Ground-mounted bird repellent device: Composition: sonic speaker array (360° coverage), infrared radiation panel (simulating the body temperature of natural enemies at 39-42°C); deployment spacing: 50 meters (power scenario) / 200 meters (airport scenario), via LoRa wireless networking.

[0065] 2. Data collection and preprocessing implementation steps:

[0066] 1. Multi-sensor spatiotemporal synchronization:

[0067] 1.1 Time synchronization: The edge computing device acts as the PTP master clock and broadcasts 1588v2 time packets to sensors via Ethernet. Each sensor has a built-in PTP slave clock module to achieve nanosecond-level timestamp alignment (synchronization error <10ns).

[0068] 1.2 Spatial Calibration: A checkerboard calibration plate (10 × 10 grid, 5 cm side length) was used to collect 20 sets of multi-view data. The extrinsic parameter matrices (rotation matrix R and translation vector T) of thermal imaging and visible light were calculated using the Zhang calibration method. The millimeter-wave radar point cloud data was mapped to the image coordinate system using the coordinate transformation matrix P = R·X+T.

[0069] 2. Data preprocessing process:

[0070] 2.1 Thermal imaging processing:

[0071] 2.11 Dynamic Range Optimization (AGC4.0 technology): Improves thermal imaging contrast, highlighting the difference between bird body temperature (35-42°C) and the ambient temperature (e.g., -20°C to 50°C).

[0072] Global histogram equalization: original temperature range: T raw ∈[T min ,Tmax ](measured -25℃ to 60℃); mapping formula:

[0073]

[0074] Example: If the original temperature of a pixel is 35°C, T min =-20℃, T max =50℃, then:

[0075] (8-bit grayscale value);

[0076] Adaptive gain control: Set the bird body temperature ROI (30-45°C) and increase the gain in this area by 1.5 times:

[0077]

[0078] For example, the grayscale value of the above 35°C pixel after AGC is 200×1.5=300 (overflowing to 255, i.e. saturated white).

[0079] 2.12 Noise reduction processing (3D DNR algorithm):

[0080] Objective: To eliminate noise in the temporal dimension (consecutive frames) and spatial dimension (neighborhood pixels). The noise types are mainly Gaussian noise (σ = 10 grayscale values) and salt and pepper noise (density = 5%).

[0081] Time dimension denoising (3-frame median filtering): 3 consecutive frames of image f t (x,y),f t-1 (x,y),f t+1 (x,y), take the median:

[0082] f temp (x,y)=median{f t (x,y),f t-1 (x,y),f t+1 (x,y)};

[0083] For example, if the grayscale value of a pixel in three consecutive frames is [200, 210, 195], the median value is 200.

[0084] Spatial dimension denoising (5×5 neighborhood mean filtering): Calculate the mean of non-edge pixels in a 5×5 neighborhood:

[0085]

[0086] Where Ω is a 5×5 neighborhood, and N is the number of pixels in the neighborhood whose gradient difference with the center pixel is less than 15 (edge ​​detection threshold = 15).

[0087] Example: If the gradient difference of 8 pixels in a neighborhood is less than 15, the mean is:

[0088]

[0089] 2.13. Centroid calculation (temperature weighted method):

[0090] Objective: Determine the geometric center of a bird's heat signature for trajectory tracking.

[0091] formula:

[0092]

[0093] Parameter description: T(x,y): Temperature grayscale value after noise reduction (0-255), only retain T(x,y)≥T thresh Pixels (T thresh Determined by a dynamic thresholding algorithm, see below).

[0094] Image resolution: 320 × 240 pixels (FLIR Boson sensor).

[0095] Calculation example: Assume that the coordinates and grayscale values ​​of three valid pixels in a frame are (100, 150, 200), (105, 155, 220), and (95, 145, 180);

[0096] Then the coordinates of the center of mass are:

[0097]

[0098] 2.14. Dynamic threshold algorithm (ambient temperature gradient compensation):

[0099] formula:

[0100] θ t =θ0+αG t +β·Var(T env );

[0101] Parameter values: θ0 = 35°C (basal body temperature threshold of birds); α = 0.8 (temperature gradient coefficient, the threshold increases by 0.8°C for every 1°C increase in actual measurement); β = 0.5 (ambient temperature variance coefficient, the variance increases by 1°C). 2 , the threshold value is reduced by 0.5℃); (Temperature gradient within 10 minutes, Δt = 600 seconds); Var(T env ): Ambient temperature variance in the past hour (unit: ℃ 2 ).

[0102] Calculation example: If the current ambient temperature T t =25℃, 10 minutes ago T t-10=20℃, the variance Var( Tenv )=4℃ 2 ,but:

[0103]

[0104] θ t =35+0.8×0.0083×600+0.5×4=35+4+2=41°C;

[0105] (Note: The second-level gradient is converted to °C / 10 minutes, so G t ×600 is the temperature change within 10 minutes).

[0106] As shown in Table 1 below:

[0107] Table 1 Comparison of key indicators of thermal imaging processing

[0108]

[0109] What needs to be explained here is:

[0110] AGC4.0 parameters: Refer to the official FLIR SDK documentation. The empirical value of the bird body temperature ROI gain coefficient is 1.5-2.0, and the middle value of 1.5 is used.

[0111] 3D DNR window size: Based on the thermal imaging frame rate (10 Hz), a time dimension of 3 frames (0.3 seconds) can effectively suppress dynamic noise; a 5×5 neighborhood in the spatial dimension balances computational efficiency and noise reduction effect.

[0112] Dynamic threshold coefficient: Calculated through fitting measured data, the α and β values ​​ensure that the threshold fluctuation is ≤5°C in a 20°C temperature difference scenario, achieving the optimal false alarm rate.

[0113] 2.2 Visible light treatment:

[0114] 2.21. Image enhancement (DDE technology):

[0115] Goal: Enhance the texture details of bird feathers, enhance edge contrast, and resolve image blur in low-light or hazy environments.

[0116] Technical implementation:

[0117] Dynamic Detail Enhancement (DDE):

[0118] 1. Gradient calculation: Use the Sobel operator to calculate the image gradient magnitude G(x,y):

[0119]

[0120] Where I is the original image and * represents the convolution operation.

[0121] 2. Detail extraction: Separate the low-frequency component (background) and high-frequency component (detail) of the image:

[0122] I low =I*KG auss ,I high =II low ;

[0123] Gaussian kernel K Gauss The size is 11×11, and the standard deviation σ=3.

[0124] 3. Detail enhancement: Apply a nonlinear gain γ = 1.5 to the high-frequency components:

[0125] I enhanced =I lo w+(I high ×γ);

[0126] For example, if the high-frequency component of an edge pixel is 20, it becomes 20×1.5=30 after enhancement, and the contrast is improved by 50%.

[0127] 2.22. Noise suppression (adaptive median filtering):

[0128] Goal: Remove salt and pepper noise (e.g., random sensor noise) from images while preserving edge details.

[0129] Technical implementation:

[0130] Adaptive median filtering:

[0131] 1. Define the sliding window S xy (maximum size 7×7), calculate the median Z of the pixels in the window med , minimum value Z min , maximum value Z max .

[0132] 2. Calculate window statistics:

[0133] A=Z med -Z min ,B=Z med -Z max ;

[0134] 3. If A>0 and B<0, it means there is noise in the window, and the output is Z med ; Otherwise, output the center pixel value Z xy .

[0135] Effect: For salt and pepper noise with a density of 5%, the residual noise after noise reduction is ≤1%, and the edge blur increase is ≤5%.

[0136] 2.23. Color space conversion (RGB to HSV):

[0137] Goal: Separate brightness and color information to facilitate subsequent target detection algorithms (such as YOLOv8) to extract bird feather color features (such as brown for sparrows and dark brown for hawks).

[0138] Conversion formula:

[0139]

[0140] Parameter Description:

[0141] R, G, B ∈ [0, 255], after conversion H ∈ [0°, 360°], S, V ∈ [0, 1].

[0142] Example: RGB(255,165,0) (orange) converted to HSV is H=30°, S=1, V=1.

[0143] 2.24. Object Detection (YOLOv8 Algorithm):

[0144] Goal: Locate the bird in the image and output the bounding box coordinates (x1, y1, x2, y2) and confidence level.

[0145] Implementation steps:

[0146] 1. Input preprocessing:

[0147] Image scaling: Resize the original 1080P (1920×1080) image to 640×640 pixels, maintaining the aspect ratio and filling the edges with black.

[0148] Normalization: Pixel values ​​are divided by 255 and converted to the range [0, 1].

[0149] 2. Model Reasoning:

[0150] Backbone: C2f structure, consisting of 6 BottleneckCSP modules, extracts multi-scale features.

[0151] Neck: PAN structure, which integrates features from different levels to enhance small target detection capabilities.

[0152] head: DetectHead, outputs the category probabilities and bounding box parameters of 80 categories of targets (including 20 bird species).

[0153] 3. Post-processing:

[0154] NMS (non-maximum suppression): IOU threshold 0.6, confidence threshold 0.5.

[0155] Coordinate mapping: restore the 640×640 scale detection results to the original image size.

[0156] 4. Performance testing:

[0157] Detection frame rate: 30fps (JetsonAGX Orin platform);

[0158] Small targets (pixel area < 100 2 )Detection accuracy: 75% (traditional YOLOv5 is 60%).

[0159] 2.25. Feature fusion preparation (alignment with thermal imaging data):

[0160] Objective: To match the bounding boxes of birds detected by visible light with the centroid coordinates of thermal images to achieve spatiotemporal alignment of multimodal data.

[0161] Implementation steps:

[0162] 1. External parameter calibration:

[0163] The Zhang calibration method is used to obtain the rotation matrix R and translation vector RT of thermal imaging and visible light, with an accuracy of ≤0.5° / 1cm.

[0164] Thermal imaging centroid Convert to visible light coordinate system:

[0165]

[0166] 2. Time synchronization:

[0167] Based on the PTP protocol, the timestamp error between visible light and thermal imaging data is less than 10ns, ensuring data alignment at the same moment.

[0168] As shown in Table 2 below:

[0169] Table 2 Comparison of key indicators of visible light processing

[0170]

[0171] It should be noted here that:

[0172] 1. DDE gain factor γ = 1.5: Determined through subjective evaluation experiments, this value enhances details while avoiding overexposure (such as saturation of the sky area).

[0173] 2. The maximum adaptive filter window size is 7×7: This balances computational efficiency and noise suppression capabilities. The measured processing time for 5% salt and pepper noise is <10ms / frame.

[0174] 3. YOLOv8 Small Target Optimization: By increasing the neck feature fusion level (PAN structure) and using smaller anchor boxes (such as (10,13) and (16,30)), the detection ability of small birds such as sparrows is improved.

[0175] 2.3 mmWave radar processing:

[0176] CFAR algorithm detects moving targets (constant false alarm rate is set to 10 -4 );

[0177] Generate point cloud data (x,y,z,v x ,v y ,v z ), filter stationary targets with a speed < 1m / s.

[0178] 2.4 Voiceprint processing:

[0179] Frame division and windowing (Hamming window, frame length 512ms, overlap rate 50%);

[0180] Calculate MFCC features (13th order coefficients + 13th order differences, a total of 26 dimensions);

[0181] Through VAD voice activity detection, environmental noise is filtered (SNR>10dB).

[0182] 3. Implementation steps of intelligent recognition algorithm:

[0183] 1. Dynamic thermal imaging spatiotemporal modeling (LSTM+Kalman filter).

[0184] 1.1 Data preparation:

[0185] State definition: Determine the vector X used to describe the bird's motion state t , which contains information about position, velocity and temperature:

[0186]

[0187] Among them, x c and y c is the center of mass coordinate of the bird target in thermal imaging, v x and v y are the velocities in the x and y directions, respectively, which are calculated by dividing the difference in the center of mass coordinates of two adjacent frames by the time interval Δt, that is:

[0188]

[0189] T avg is the average temperature of the target area, is the rate of temperature change,

[0190]

[0191] Here Δt=0.1 seconds.

[0192] Data collection and preprocessing:

[0193] Acquire continuous thermal imaging frame data from the thermal imaging sensor.

[0194] Each frame of data is preprocessed, including the steps mentioned above such as dynamic range optimization, noise reduction and centroid calculation, to obtain the X t .

[0195] In chronological order, X t Combined into training data sequences, each sequence is 4 frames long, which are used as input for the subsequent LSTM network.

[0196] 1.2LSTM network training:

[0197] Network structure design:

[0198] Input layer: The input dimension is 6×4, corresponding to 4 frames of historical state X t , each frame X t There are 6 elements.

[0199] Hidden layer: 64 neurons are set, and the activation function uses the tanh function, which can map the input value to the [-1, 1] interval, which helps to deal with the gradient disappearance problem.

[0200] Output layer: The output dimension is 6×3 and is used to predict the state of the next 3 frames.

[0201] Training process:

[0202] Data division: The collected 5,000 sets of data containing 10 bird species’ tracks were divided into training set, validation set, and test set in a ratio of 8:1:1.

[0203] Optimizer and loss function: We use the Adam optimizer, which combines the advantages of Adagrad and RMSProp and can adaptively adjust the learning rate. The learning rate is set to 0.001. The loss function uses the mean squared error (MSE), which measures the difference between the predicted value and the true value. The formula is:

[0204]

[0205] Where n is the number of samples, is the predicted value, y i is the true value.

[0206] Training iterations: Set the batch size to 32, meaning 32 samples are used for training each iteration. Train the LSTM network over multiple iterations and continuously adjust the network parameters to gradually reduce the loss function value.

[0207] Prediction stage (LSTM):

[0208] Input preparation:

[0209] In practical applications, the thermal imaging data of the current moment and the previous three frames are obtained, and the corresponding X t sequence, as input to the LSTM network.

[0210] Predictive Execution:

[0211] Pass the input sequence into the trained LSTM network to obtain the state prediction value of the next 3 frames

[0212] 1.3 Kalman filter correction:

[0213] State transition matrix:

[0214] Define the state transfer matrix F to describe the change of state from one moment to the next:

[0215]

[0216] For example, the change in position is related to the speed and time interval, so x t+1 =x t +v x Δt, the value of the first row and third column in the corresponding matrix is ​​Δt.

[0217] Observation matrix:

[0218] Define the observation matrix H to extract observable information from the state vector, here we extract the position and average temperature information:

[0219]

[0220] Covariance update:

[0221] Process noise covariance matrix Q: represents the uncertainty in the state transition process and is set to a diagonal matrix Q = diag(0.1, 0.1, 0.01, 0.01, 0.5, 0.1). It reflects the degree of uncertainty of different state components. For example, the uncertainty of position is larger than that of speed.

[0222] Observation noise covariance matrix R: represents the uncertainty in the observation process and is set to a diagonal matrix R = diag(1,1,1).

[0223] At each moment, the state and covariance are updated according to the Kalman filter update formula based on the predicted value and the actual observation value:

[0224] Prediction steps:

[0225] Prediction status:

[0226] Prediction covariance: P t|t-1 =FP t-1|t-1 F T +Q;

[0227] Update steps:

[0228] Calculate the Kalman gain: K t =P t|t-1 H T (HP t|t-1 H T +R) -1 ;

[0229] Update status:

[0230] Update covariance: P t|t =(IK t H)P t|t-1 ;

[0231] where Z t is the actual observed value, is the predicted state, is the updated state, P t|t-1 is the prediction covariance, P t|t is the updated covariance, K t is the Kalman gain.

[0232] 1.4 Trajectory Prediction and Decision-making:

[0233] Trajectory prediction:

[0234] By combining the LSTM network and Kalman filtering, the bird's state prediction value is continuously updated to obtain continuous trajectory prediction points.

[0235] Decision trigger:

[0236] The error between the predicted trajectory and the actual observed trajectory is calculated. When the error is less than 5 pixels, a bird-repelling response is triggered, such as activating the corresponding bird-repelling equipment.

[0237] 2. Multimodal feature fusion (Transformer architecture):

[0238] 2.1 Multimodal Data Input Encoding:

[0239] Objective: To uniformly encode three modal data types—thermal imaging, voiceprint, and visible light—into feature vectors suitable for Transformer processing.

[0240] step:

[0241] 2.11 Thermal imaging feature coding:

[0242] 1. Input: Thermal imaging centroid region temperature matrix (size 16×16, local ROI at resolution 320×240).

[0243] 2. Processing: Flatten to a 256-dimensional vector T and add sinusoidal position encoding:

[0244]

[0245] Where pos is the pixel position (1-256), d model =256 is the feature dimension, and i is the dimension index.

[0246] 3. Output: Encoded vector T enc =T+PE.

[0247] 2.12 Voiceprint Feature Coding:

[0248] 1. Input: MFCC feature vector (26 dimensions, including 13th-order coefficients + 13th-order differences).

[0249] 2. Processing: through the linear layer W S ∈R 256×26 Mapping to 256 dimensions:

[0250] S enc =ReLU(W S S);

[0251] 3. Output: Normalized vector S enc .

[0252] 2.13 Visible light feature encoding:

[0253] 1. Input: Bird bounding box feature vector output by YOLOv8 (1024 dimensions, including target confidence, classification probability, and position encoding).

[0254] 2. Processing: through the linear layer W V ∈R 256×1024 Dimensionality reduction to 256 dimensions:

[0255] V enc =LayerNorm(W V V);

[0256] 3. Output: Normalized vector V enc .

[0257] Modal splicing:

[0258] The three modal coding vectors are concatenated into the input sequence X = [T enc ;S enc ; V enc ]∈R 3×256 , corresponding to 3 modal tokens.

[0259] 2.2 Transformer encoder processing:

[0260] Objective: To achieve cross-modal feature interaction through a multi-head self-attention mechanism and capture the correlation between temperature, sound, and visual features.

[0261] step:

[0262] Multi-head attention calculation (SingleHead):

[0263] 1. Query / Key / Value generation:

[0264] Q=X·W Q ,K=X·W K ,V=X·W V ;

[0265] Where W Q ,W K ,W V ∈R 256×32 , single head dimension d k =32.

[0266] 2. Attention score calculation:

[0267]

[0268] where q T ,q S ,q V is the query vector of each modality, k T ,k S ,k V is the Key vector.

[0269] 3. Weight normalization and feature aggregation:

[0270] Attn h =softmax(Score)·V,h∈{1,,8};

[0271] Multi-head attention splicing:

[0272] Concatenate the 8-head attention output into a 256-dimensional vector:

[0273] MultiAttn=Concat(Attn1,…,Attn8)·W O ,W O ∈R 256×256 ;

[0274] Feed-Forward Network (FFN):

[0275] FFN(x)=GELU(x·W1+b1)·W2+b2;

[0276] where W1∈R 256×1024 ,W2∈R 1024×256 , the activation function uses GELU to enhance nonlinearity.

[0277] Layer normalization and residual connection:

[0278] x out =LayerNorm(x+MultiAttn+FFN(x));

[0279] 2.3 Feature fusion and classification decision:

[0280] Goal: Use the fused features output by Transformer for bird species classification or behavior prediction.

[0281] step:

[0282] Global feature aggregation:

[0283] Take the average of the output of the three modal tokens to obtain the global fusion feature

[0284] Classifier:

[0285] Mapped to category space through a fully connected layer:

[0286]

[0287] Where W class ∈R 256×50 , corresponding to 50 bird categories.

[0288] Loss function:

[0289] Cross entropy loss is combined with focal loss to alleviate the problem of category imbalance:

[0290]

[0291] where α i is the category weight, and γ = 2 is the focusing parameter.

[0292] 2.4 Model training and reasoning:

[0293] Objective: Optimize Transformer parameters and improve the accuracy of multimodal feature fusion.

[0294] step:

[0295] Training data:

[0296] Dataset: Contains 10,000 sets of thermal imaging, voiceprint, and visible light triple samples of 30 bird species (8,000 training sets, 1,000 validation sets, and 1,000 test sets).

[0297] Data enhancement: Gaussian noise (σ=0.05) is added to thermal imaging, visible light is randomly cropped (ratio 80%), and white noise (SNR=15dB) is added to voiceprints.

[0298] Training parameters:

[0299] Optimizer: AdamW (weight decay 0.01), initial learning rate 5e-5, cosine annealing decay.

[0300] Batch size: 32, training rounds (Epoch): 50.

[0301] Verification indicators: classification accuracy, F1 score, early stopping mechanism (termination if there is no improvement in the validation set for 5 consecutive rounds).

[0302] Reasoning process:

[0303] 1. Real-time collection of multimodal data, pre-processing to generate T enc ,S enc ,V enc .

[0304] 2. Obtain the fusion feature F through Transformer forward propagation.

[0305] 3. The classifier outputs the category probability. When the threshold is ≥ 0.5, the corresponding bird-repelling strategy is triggered.

[0306] As shown in Table 3 below:

[0307] Table 3 Key parameters and effects

[0308]

[0309] It should be noted here that:

[0310] Modality alignment: The thermal imaging ROI must be spatially aligned with the visible light bounding box (error ≤ 2 pixels), and the voiceprint signal must be temporally aligned with the video frame (error ≤ 50ms).

[0311] Computational efficiency: Single-frame processing takes approximately 50ms (Jetson AGX Orin), meeting 10Hz real-time requirements.

[0312] Interpretability: By visualizing attention weights (such as thermal imaging-voiceprint association weights), the correlation between bird body temperature and song can be analyzed, assisting ecological research.

[0313] 3. Edge-Cloud Collaborative Federated Learning:

[0314] 3.1 Edge node training: Local dataset: 1000 annotated thermal imaging images (resolution 64×64); Model structure: MobileNetV3+CBAM (channel attention mechanism); Training process: 5 rounds of iterations, learning rate 0.001, and Laplace noise added after gradient calculation (scale parameter 1 / ε, ε=0.5).

[0315] 3.2 Cloud aggregation process: Receive encrypted gradients uploaded by 50 edge nodes (each node has about 1000 samples); Federated average aggregation:

[0316]

[0317] 3.3 Model distribution: The updated weight file (size ≤ 10MB) is pushed via the 5G network, and the model update cycle is ≤ 24 hours.

[0318] 4. Implementation steps of intelligent bird repellent strategy:

[0319] 1. Dynamic protection area generation:

[0320] 1.1 Heatmap calculation: Based on 7 days of historical bird activity data, the frequency of heat signals was counted in a 50 × 50 m grid. The heatmap was generated using Gaussian kernel smoothing (σ = 2), and the threshold was set to the average frequency + 2σ.

[0321] 1.2 UAV path planning: Improved A* algorithm: The cost function includes thermal signal density (weight 0.6), obstacle distance (weight 0.3), and remaining battery power (weight 0.1); Waypoint generation: The path is updated every 20 seconds, and the distance between adjacent waypoints is ≤50 meters.

[0322] 2. Differentiated bird repellent strategies:

[0323] 2.1 Small birds (body length <15 cm, such as sparrows):

[0324] Directional sound waves: frequency 20-40kHz (sparrow sensitive frequency band), duration 5 seconds / time, interval 30 seconds;

[0325] Laser strobe: 520nm green laser, scanning rate 10Hz, irradiation distance 50 meters (avoid direct exposure to birds' eyes);

[0326] 2.2 Large birds of prey (body length > 50 cm, such as eagles):

[0327] Natural enemy heat signal simulation: infrared radiation panel heated to 40±1℃, covering an area of ​​1m 2 , duration 10 seconds;

[0328] Sound deterrent: Play hawk calls (8-12kHz) at a sound pressure level of 110dB, directed toward the raptor's location;

[0329] 2.3 Flocks of birds (>50 birds):

[0330] Multi-UAV coordination: 3 UAVs in a triangle formation (100 meters apart), synchronously emitting 120dB sound waves (frequency sweep 10-20kHz);

[0331] Dynamic protection circle: Circular expulsion is implemented with a radius of 200 meters, centered on the center of mass of the flock.

[0332] 3. Feedback on bird repellent effect:

[0333] The edge node monitors the thermal signal changes within 30 seconds after the expulsion in real time:

[0334] If the bird stays for >10 seconds, switch the bird repellent strategy (e.g., increase the sound wave frequency to 50 kHz);

[0335] Upload bird-repelling success / failure events to the cloud every hour (after desensitization processing for model optimization).

[0336] 5. System calibration and maintenance implementation steps:

[0337] 1. Thermal imaging sensor self-calibration:

[0338] Trigger blackbody calibration at 2:00 AM every day:

[0339] 1.1 Align the built-in blackbody radiation source (temperature 40°C, emissivity 0.95);

[0340] 1.2 Calculate the offset coefficient: T cal =T raw K+B, where K = 10.2 and B = -0.5°C;

[0341] 1.3 Calibration error: ≤0.1℃ (full temperature range).

[0342] 2. Model performance monitoring:

[0343] Cloud-based monitoring platform deployment:

[0344] 2.1 Real-time indicators: recognition accuracy (target ≥ 95%), false alarm rate (target ≤ 2% / hour), response time (target ≤ 1.5 seconds)

[0345] 2.2 Abnormal alarm: When the accuracy is <90% for 1 consecutive hour, the federated learning model update is automatically triggered.

[0346] 6. Application scenarios:

[0347] 1. Bird prevention scenarios for power facilities:

[0348] Deployment environment: 220kV substation, area 5000m 2 , Main birds: sparrows, turtledoves;

[0349] data:

[0350] Nighttime recognition accuracy: 92% (traditional solution 65%);

[0351] Bird repellent response time: 0.8 seconds (traditional solution 5 seconds);

[0352] Line tripping accidents: reduced from 3 times per month to 0.5 times;

[0353] 2. Airport runway bird-repelling scene:

[0354] Deployment environment: 4E-class airport, runway length 3,000 meters, main bird species: terns and crows;

[0355] data:

[0356] Recognition accuracy in complex weather (light rain + force 6 wind): 89% (traditional solution 65%);

[0357] Bird strikes: from 20 to 4 per year;

[0358] Drone endurance: single flight coverage 20km 2 , meeting the needs of monitoring the entire airport area.

[0359] In summary, the present invention includes: Multimodal data fusion means: Through nanosecond-level spatiotemporal synchronization of thermal imaging, visible light, millimeter-wave radar, and voiceprint sensors, combined with pre-processing technologies such as AGC4.0 and 3D DNR, data quality and alignment accuracy are improved, laying the foundation for intelligent recognition. Dynamic modeling and intelligent decision-making means: Utilize LSTM networks to predict bird trajectories, Kalman filtering to correct errors, and Transformer architecture to achieve cross-modal feature interaction and generate fused feature vectors. Combined with edge computing, bird repellent strategies are generated in real time, and cloud-based federated learning optimizes the global model to achieve a "monitoring-prediction-repellent" closed loop. Differentiated bird repellent and system optimization means: Dynamically select bird repellent strategies (such as directional sound waves and natural enemy thermal simulation) based on bird species, number, and trajectory, combine thermal signal density heat maps with the A* algorithm to plan drone paths, and ensure long-term system reliability through blackbody self-calibration (error ≤ 0.1°C) and model monitoring (accuracy ≥ 95%).

[0360] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0361] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A multi-dimensional bird prevention intelligent identification method based on thermal imaging, characterized in that: The following steps are involved: Step 1: Multimodal data collection: Use thermal imaging sensors to obtain bird body temperature distribution data, visible light cameras to collect bird texture features, millimeter wave radar to track the three-dimensional trajectory of birds, and voiceprint sensors to collect bird sound signals; Step 2: Spatiotemporal synchronization preprocessing: Time synchronization of multi-sensor data based on the PTP protocol, combined with the Kalman filter algorithm to eliminate inter-sensor delay errors, dynamic range optimization and noise reduction processing of thermal imaging data, detail enhancement and target detection of visible light images, point cloud data generation for millimeter-wave radar data, and Mel-frequency cepstral coefficients extraction for voiceprint signals; Step 3: Dynamic thermal imaging modeling: Build a spatiotemporal thermal distribution model of bird flight, use the LSTM network to predict the flight trajectory for the next 3-5 seconds, and use the Kalman filter algorithm to correct the predicted trajectory. When the trajectory error is less than 5 pixels, trigger the bird repelling response. Step 4: Multimodal feature fusion: Encode the thermal imaging temperature features, voiceprint frequency features, and visible light texture features into a token sequence, implement cross-modal feature interaction through the Transformer architecture, and generate a fused feature vector. Step 5: Edge-cloud collaborative decision-making: The edge computing device processes multimodal data in real time and generates a preliminary bird-repelling strategy. The cloud server aggregates the encrypted model gradients uploaded by the edge nodes based on the federated learning framework and updates the global recognition model. Step 6: Execute intelligent bird repellent strategies; based on the bird species, number, and predicted trajectory, dynamically select strategies such as directional acoustic repellent, laser stroboscopic deterrence, or natural enemy heat signal simulation to form differentiated bird repellent solutions.

2. The multi-dimensional bird prevention intelligent identification method based on thermal imaging according to claim 1 is characterized in that: In step 2, the dynamic range optimization of thermal imaging data adopts AGC4.0 technology, and the noise reduction processing adopts 3D DNR algorithm; the visible light image detail enhancement adopts DDE technology, and the target detection adopts YOLOv8 algorithm.

3. The multi-dimensional bird prevention intelligent identification method based on thermal imaging according to claim 1 is characterized in that: In step 3, the hidden layer dimension of the LSTM network is 64, the input sequence length is 4 frames, and the prediction step length corresponds to a time of 3-5 seconds; the state transition matrix of the Kalman filter algorithm includes position, velocity, and temperature change rate parameters.

4. The multi-dimensional bird prevention intelligent identification method based on thermal imaging according to claim 1 is characterized in that: In step 4, the Transformer architecture contains 6 encoder layers, each of which uses an 8-head self-attention mechanism with a single-head dimension of 32. The weighted fusion of cross-modal features is achieved through the multi-head attention mechanism.

5. The multi-dimensional bird prevention intelligent identification method based on thermal imaging according to claim 1 is characterized in that: In step 5, the edge nodes of the federated learning framework add Laplace noise to the local gradient to achieve differential privacy protection, with a privacy budget parameter ε≤1 and a model update cycle ≤24 hours.

6. A multi-dimensional bird prevention intelligent identification system based on thermal imaging, characterized in that: It includes a multimodal perception module, an edge computing unit, a cloud server and an intelligent bird-repelling execution module; the multimodal perception module integrates thermal imaging sensors, visible light cameras, millimeter-wave radars and voiceprint sensors, is configured with a PTP protocol synchronization circuit, and supports operation in a wide temperature environment of -40°C to 70°C; the edge computing unit is equipped with a computing chip, a built-in LSTM+ Kalman filter trajectory prediction algorithm, a Transformer multimodal fusion model and a dynamic threshold adjustment module to realize real-time processing of multimodal data; the cloud server deploys a federated learning platform to store a bird voiceprint library, a thermal signal feature library and a bird-repelling strategy database, and supports encrypted gradient aggregation and global model updates; the intelligent bird-repelling execution module includes a hybrid drone and a ground-fixed bird-repelling device. The drone is equipped with a lightweight thermal imaging module and a directional sound wave emission device, and supports dynamic path planning based on a thermal signal density heat map.

7. The multi-dimensional bird prevention intelligent identification system based on thermal imaging according to claim 6 is characterized in that: In the multimodal perception module, the thermal imaging sensor has a temperature measurement accuracy of ±0.05°C; the millimeter-wave radar supports three-dimensional trajectory tracking within 200 meters, the point cloud data update frequency is ≥10Hz, and the voiceprint sensor supports broadband signal acquisition.

8. The multi-dimensional bird prevention intelligent identification system based on thermal imaging according to claim 6 is characterized in that: In the edge computing unit, the dynamic threshold adjustment module adopts the formula: i t =θ0+αG t +βVar(T env ); Among them, the initial threshold θ0 = 35°C, the temperature gradient coefficient α = 0.8, and the ambient temperature variance coefficient β = 0.5, which realizes dynamic filtering of ambient heat source interference.

9. The multi-dimensional bird prevention intelligent identification system based on thermal imaging according to claim 6, characterized in that: In the intelligent bird-repellent execution module, the hybrid drone has a flight time of ≥5 hours, a wind resistance level of ≥8, supports operation in a temperature difference environment of -20℃ to 50℃, and is equipped with a directional sound wave transmitter covering a frequency range of 20-200kHz and a sound pressure level of ≥120dB.

10. The multi-dimensional bird prevention intelligent identification system based on thermal imaging according to claim 6, characterized in that: In the cloud server, the bird voiceprint library pre-stores MFCC features of ≥50 bird species; When the cloud server implements model update through federated learning.

Images