A method and system for real-time early warning of bird strike risk based on video stream analysis

Abstract

This invention relates to the fields of computer vision and intelligent monitoring technology, specifically a real-time bird strike risk warning method and system based on video stream analysis. The invention employs a local adaptive enhancement algorithm to perform differentiated preprocessing of the video stream based on local neighborhood features of pixels; it uses a YOLOv8m-Bird model with embedded convolutional block attention modules for bird detection; based on the detection results, it constructs a dynamic spatiotemporal graph encoding social following interaction forces, and after fusing fine-grained behavioral features, generates predicted trajectories through a temporal prediction network; it quantifies the risk intensity value by comprehensively considering spatial distance, approach speed, intersection time, and behavioral uncertainty indicators; it fuses the risk intensity value with environmental data to generate a dynamic risk heatmap, which is then overlaid and displayed. When the threshold is exceeded, an early warning is triggered and bird deterrence equipment is activated, achieving real-time and accurate early warning of bird strike risk.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent monitoring technology, specifically to a real-time bird strike risk warning method and system based on video stream analysis. Background Technology

[0002] Bird strikes are a major safety issue in aviation, wind power, and other fields, causing enormous economic losses and safety hazards globally every year. With the rapid development of the wind power industry, bird strikes on wind turbine blades are becoming increasingly frequent, negatively impacting the ecological environment and sustainable industrial development, necessitating effective early warning and prevention measures.

[0003] Currently, bird strike prevention mainly employs two methods: manual bird deterrence and radar monitoring. Manual bird deterrence relies on manpower, has a slow response time, and cannot achieve 24 / 7 monitoring. While radar monitoring can detect bird activity over a large area, it is costly to deploy, has weak detection capabilities for low-flying, small birds, a high false alarm rate, and cannot identify species. In recent years, vision-based monitoring solutions have begun to be applied, but existing technologies are mostly focused on simple detection and suffer from poor environmental adaptability, lack of risk quantification capabilities, processing speeds that cannot meet real-time requirements, and weak multi-target tracking capabilities, making it difficult to form continuous trajectories and effectively predict flight intentions. Summary of the Invention

[0004] This invention provides a method and system for real-time early warning of bird strike risk based on video stream analysis, enabling real-time, accurate early warning and visualized control of bird strike risk.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] This invention discloses a real-time bird strike risk warning method based on video stream analysis, comprising:

[0007] S100: Acquire the monitoring video stream of the key area, preprocess the monitoring video stream using a local adaptive enhancement algorithm, construct a local feature descriptor based on the local neighborhood features of the pixels, and apply different enhancement parameters to different areas based on the local feature descriptor;

[0008] S200: The YOLOv8m-Bird detection model is used to detect bird targets in the preprocessed video stream, and the detection results of each bird target are output.

[0009] S300: Construct a dynamic spatiotemporal map based on the detection results to obtain dynamic spatiotemporal map features; extract fine-grained behavioral features of each bird target, fuse the fine-grained behavioral features with the dynamic spatiotemporal map features and input them into a temporal prediction network to generate predicted trajectories of each bird target;

[0010] S400: Based on the predicted trajectory, calculate the spatial distance, approach speed, and intersection time between each bird target and the protected airspace, analyze the behavioral uncertainty index generated by the predicted trajectory, and combine the spatial distance, approach speed, intersection time, and behavioral uncertainty index to obtain the risk intensity value of each bird target;

[0011] S500: The risk intensity value and environmental data are fused to generate a dynamic risk heat map and overlaid on the monitoring video stream for display. When the risk intensity value exceeds a preset threshold, an early warning is triggered and the bird deterrent device is activated.

[0012] As a preferred embodiment of the present invention, the step of preprocessing the surveillance video stream using a local adaptive enhancement algorithm includes:

[0013] Calculate the consistency between texture intensity and gradient direction in the local neighborhood of a pixel to construct a local feature descriptor;

[0014] The texture complexity of the region is determined based on the local feature descriptor.

[0015] Apply a high-gain enhancement parameter to regions with high texture complexity and a low-gain enhancement parameter to regions with low texture complexity.

[0016] As a preferred embodiment of the present invention, applying a high-gain enhancement parameter to regions with high texture complexity and applying a low-gain enhancement parameter to regions with low texture complexity includes:

[0017] The morphological parameters of the grayscale transformation curve are dynamically adjusted based on the local feature descriptor, generating steep transformation curves for regions with high texture complexity and flat transformation curves for regions with low texture complexity.

[0018] As a preferred embodiment of the present invention, the YOLOv8m-Bird detection model is an improved model in which convolutional block attention modules are embedded in the backbone network and neck network of the YOLOv8m model.

[0019] The convolutional block attention module includes a channel attention submodule and a spatial attention submodule;

[0020] The channel attention submodule assigns differentiated weights to feature channels through global average pooling and max pooling;

[0021] The spatial attention submodule generates a spatial weight mask through convolution operations.

[0022] As a preferred embodiment of the present invention, the step of constructing a dynamic spatiotemporal graph based on the detection results includes:

[0023] A node is created for each bird target, and the node represents the visual features and motion state of each bird target;

[0024] Establish edges between nodes, which represent the interaction relationships within the bird flock and the spatial relationship between each bird target and the protected airspace;

[0025] The dynamic spatiotemporal graph is processed to obtain dynamic spatiotemporal graph features containing group interaction information.

[0026] As a preferred embodiment of the present invention, the step of generating the predicted trajectories of each bird target includes:

[0027] A three-dimensional convolutional network was used to extract the flight posture and wingbeat frequency of each bird target to obtain fine-grained behavioral features;

[0028] The fine-grained behavioral features are fused with the dynamic spatiotemporal graph features;

[0029] The fused features are input into an LSTM-based temporal prediction network to generate predicted trajectories for each bird target.

[0030] As a preferred embodiment of the present invention, the analysis of the uncertainty index of the predicted trajectory generation behavior includes:

[0031] Extract the acceleration change features from the predicted trajectory;

[0032] By analyzing the acceleration change characteristics, abrupt changes in motion patterns can be identified.

[0033] The behavioral uncertainty index is calculated based on the abrupt change characteristics of the aforementioned motion pattern.

[0034] As a preferred embodiment of the present invention, the step of obtaining the risk intensity value of each bird target includes:

[0035] A basic risk value is calculated based on the aforementioned spatial distance, approach speed, and rendezvous time.

[0036] The aforementioned behavioral uncertainty index is quantified into a risk correction coefficient;

[0037] The risk intensity value of each bird target is obtained by correcting the base risk value using the risk correction coefficient.

[0038] As a preferred embodiment of the present invention, the step of fusing the risk intensity value and environmental data to generate a dynamic risk heat map includes:

[0039] Acquire environmental wind speed and direction data, and convert the wind speed and direction data into spatial influence factors;

[0040] The risk intensity value is fused with the spatial impact factor to calculate the comprehensive risk intensity value of each spatial unit;

[0041] A dynamic risk heatmap is generated based on the comprehensive risk intensity value.

[0042] This invention also proposes a real-time bird strike risk early warning system based on video stream analysis, comprising:

[0043] The video preprocessing module is used to acquire the monitoring video stream of key areas, preprocess the monitoring video stream using a local adaptive enhancement algorithm, construct local feature descriptors based on the local neighborhood features of pixels, and apply different enhancement parameters to different areas based on the local feature descriptors.

[0044] The bird detection module is used to perform bird target detection on the preprocessed video stream using the YOLOv8m-Bird detection model and output the detection results of each bird target.

[0045] The trajectory prediction module is used to construct a dynamic spatiotemporal map based on the detection results, obtain dynamic spatiotemporal map features, extract fine-grained behavioral features of each bird target, fuse the fine-grained behavioral features with the dynamic spatiotemporal map features and input them into a temporal prediction network to generate the predicted trajectory of each bird target.

[0046] The risk assessment module is used to calculate the spatial distance, approach speed, and intersection time between each bird target and the protected airspace based on the predicted trajectory, analyze the behavioral uncertainty index generated by the predicted trajectory, and obtain the risk intensity value of each bird target by combining the spatial distance, approach speed, intersection time, and behavioral uncertainty index.

[0047] The early warning display module is used to fuse the risk intensity value and environmental data to generate a dynamic risk heat map and overlay it with the monitoring video stream. When the risk intensity value exceeds a preset threshold, an early warning is triggered and the bird deterrent device is activated.

[0048] The beneficial effects of this invention are:

[0049] 1. This invention applies differentiated processing to different texture regions through an adaptive enhancement algorithm driven by local feature descriptors, and combines it with the YOLOv8m-Bird model with embedded CBAM attention mechanism, which significantly improves the detection accuracy and anti-interference ability of bird targets in complex scenes.

[0050] 2. This invention constructs a dynamic spatiotemporal graph that encodes social following interaction force, integrates fine-grained behavioral features for trajectory prediction, and introduces behavioral uncertainty indicators to comprehensively quantify risk intensity values, thus achieving a leap from simple detection to accurate risk assessment.

[0051] 3. This invention generates a dynamic risk heat map through multimodal data fusion and overlays it with a video stream to form an augmented reality display. It supports automatic early warning and linkage bird deterrence at multiple thresholds, realizing visualized risk management and efficient emergency response. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is a flowchart illustrating a real-time bird strike risk warning method based on video stream analysis according to the present invention.

[0054] Figure 2 This is a schematic diagram of the structure of a real-time bird strike risk early warning system based on video stream analysis according to the present invention. Detailed Implementation

[0055] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0056] Example 1: As Figure 1 As shown, the present invention provides a real-time bird strike risk warning method based on video stream analysis, comprising:

[0057] S100: Acquire the monitoring video stream of the key area, preprocess the monitoring video stream using a local adaptive enhancement algorithm, construct a local feature descriptor based on the local neighborhood features of the pixels, and apply different enhancement parameters to different areas based on the local feature descriptor;

[0058] Furthermore, the step of preprocessing the surveillance video stream using a local adaptive enhancement algorithm includes:

[0059] Calculate the consistency between texture intensity and gradient direction in the local neighborhood of a pixel to construct a local feature descriptor;

[0060] The texture complexity of the region is determined based on the local feature descriptor.

[0061] Apply a high-gain enhancement parameter to regions with high texture complexity and a low-gain enhancement parameter to regions with low texture complexity.

[0062] Furthermore, applying a high-gain enhancement parameter to regions with high texture complexity and a low-gain enhancement parameter to regions with low texture complexity includes:

[0063] The morphological parameters of the grayscale transformation curve are dynamically adjusted based on the local feature descriptor, generating steep transformation curves for regions with high texture complexity and flat transformation curves for regions with low texture complexity.

[0064] Specifically, multiple high-performance PTZ cameras are deployed in key areas around the wind turbine generators, along with wind speed and direction monitoring devices. Monitoring points are set up on the windward and leeward sides of the generators, as well as in the main bird activity directions. The wind speed and direction monitoring devices are spatially adapted to the corresponding PTZ cameras to ensure that the acquired wind speed and direction data accurately match the video surveillance airspace of the monitoring point, providing real-time environmental parameter support for subsequent multimodal data fusion.

[0065] Preprocessing is performed on the acquired raw surveillance video stream, employing a local adaptive enhancement algorithm that fuses the spatial distribution features of pixel values. This algorithm first calculates features such as texture intensity and gradient direction consistency within the local neighborhood of each pixel, constructing a local feature descriptor accordingly. Specifically, texture intensity... Obtained by calculating the standard deviation of pixel grayscale values ​​within a local neighborhood, representing the texture complexity of the region; gradient direction consistency. After extracting the gradient direction of each pixel using the Sobel operator, the variance of the gradient direction within the local neighborhood is calculated. A smaller variance indicates higher consistency and represents the regularity of the edges in that region. Local feature descriptor Constructed as a linear combination of texture intensity and gradient direction consistency:

[0066] ;

[0067] in For the combined weight coefficients, satisfying The importance of texture features in region classification is determined.

[0068] The texture complexity of a region is determined based on the local feature descriptors: regions with high texture intensity and low gradient direction variance are classified as high-texture-complexity regions, such as the edges of bird feathers; regions with low texture intensity or high gradient direction variance are classified as low-texture-complexity regions, such as the sky background. Specifically, the Otsu automatic thresholding algorithm is used to evaluate all local feature descriptors in the current frame. Perform segmentation to obtain the texture complexity threshold. ;when When it is determined to be a region with high texture complexity, It is then identified as a region with low texture complexity.

[0069] For regions with high texture complexity, the algorithm automatically applies high-gain enhancement parameters, including higher enhancement gain and steeper transformation curves, to significantly improve feather detail resolution and main body outline sharpness; for regions with low texture complexity, a low-gain smooth mapping is used to avoid overprocessing.

[0070] During the enhancement process, a morphological modulation factor linked to the local feature descriptor is introduced simultaneously. This factor adjusts the morphological parameters of the S-curve used for grayscale transformation in real time based on the calculated local features. Specifically, the grayscale transformation uses an S-curve in the form of a power function.

[0071] ;

[0072] in Input grayscale values ​​(normalized to [0,1]). To output grayscale values, These are morphological parameters. Based on local feature descriptors Dynamic adjustment:

[0073] For regions with high texture complexity ( ),set up This generates a steep transformation curve. The slope of the curve increases in the middle grayscale range, which strongly stretches the grayscale dynamic range to highlight the detailed structure.

[0074] For regions with low texture complexity ( ),set up This generates a flattened transformation curve. The slope of the curve decreases in the middle grayscale range, compressing the grayscale dynamic range to effectively suppress background interference such as sensor noise and illumination noise.

[0075] in The texture complexity threshold is determined by segmenting all local feature descriptors of the current frame using the Otsu automatic thresholding algorithm. The modulation factor controls the range of change in the curve shape.

[0076] Through the above processing, a preprocessed video stream is obtained that enhances the details and outline features of bird feathers while suppressing background noise.

[0077] S200: The YOLOv8m-Bird detection model is used to detect bird targets in the preprocessed video stream, and the detection results of each bird target are output.

[0078] Furthermore, the YOLOv8m-Bird detection model is an improved model that embeds convolutional block attention modules into the backbone and neck networks of the YOLOv8m model;

[0079] The convolutional block attention module includes a channel attention submodule and a spatial attention submodule;

[0080] The channel attention submodule assigns differentiated weights to feature channels through global average pooling and max pooling;

[0081] The spatial attention submodule generates a spatial weight mask through convolution operations.

[0082] Specifically, an improved YOLOv8m-Bird detection model is used to detect bird targets in the preprocessed video stream. This model is based on the baseline YOLOv8m model and incorporates a convolutional block attention module (CBAM) into its backbone and neck network.

[0083] The Convolutional Block Attention (CBAM) module comprises a channel attention submodule and a spatial attention submodule. The channel attention submodule performs global average pooling and global max pooling operations on the input feature map, inputs the pooling results into a shared multilayer perceptron network, and then performs element-wise summation of the two outputs before passing them through a sigmoid activation function to generate a channel attention weight vector. This weight vector is multiplied by the original feature map to achieve differentiated weighting for different feature channels, thereby highlighting channel features important for bird identification.

[0084] The spatial attention submodule receives the feature map after channel attention weighting, performs average pooling and max pooling operations along the channel dimension, concatenates the two pooling results along the channel dimension, and then passes them through a convolutional layer and a sigmoid activation function to generate a spatial weight mask. This mask is multiplied with the input feature map, making the model more focused on the key spatial regions in the feature map where bird targets may appear, thereby achieving enhanced feature extraction of bird targets.

[0085] During the model training phase, data transformation preprocessing based on semantic preservation is applied to the input training image set. Specific transformation methods include random rectangular occlusion to simulate the actual situation of birds being occluded by trees or other birds; and random color space perturbation, which makes small random adjustments to the hue, saturation, and brightness of the image in the HSV color space to simulate the impact of changes in lighting at different times, thereby improving the model's generalization ability and robustness when facing complex scenes.

[0086] After the trained YOLOv8m-Bird model is deployed, it performs forward inference on the preprocessed video frames, outputs the bounding box and category confidence of each bird target in the image, and generates real-time bird density statistics and individual initial movement trajectory point data for the entire monitoring scene, providing accurate input for subsequent tracking and trajectory prediction.

[0087] S300: Construct a dynamic spatiotemporal map based on the detection results to obtain dynamic spatiotemporal map features; extract fine-grained behavioral features of each bird target, fuse the fine-grained behavioral features with the dynamic spatiotemporal map features and input them into a temporal prediction network to generate predicted trajectories of each bird target;

[0088] Furthermore, the step of constructing a dynamic spatiotemporal graph based on the detection results includes:

[0089] A node is created for each bird target, and the node represents the visual features and motion state of each bird target;

[0090] Establish edges between nodes, which represent the interaction relationships within the bird flock and the spatial relationship between each bird target and the protected airspace;

[0091] The dynamic spatiotemporal graph is processed to obtain dynamic spatiotemporal graph features containing group interaction information.

[0092] Specifically, based on the multi-target bird detection results output by the S200 stage, nodes are established for each individual bird in each frame, thereby constructing a dynamically evolving spatiotemporal graph model. Nodes represent the visual characteristics and motion state of individual birds. Visual characteristics include appearance features such as color and texture of the target region extracted from the detection results, while motion state includes the target's position coordinates, speed, and direction of motion.

[0093] Edges are established between nodes, representing two key relationships: first, the social following interaction force within the flock, calculated based on the relative positions and movement directions of individuals, used to simulate group behavior patterns within the flock; and second, the spatial relationship between each individual and key protected airspace, used to characterize the impact of the external environment on individual behavior. The social following interaction force is quantified by calculating the similarity of relative distances and velocity directions between individuals. In this embodiment, for individuals... With individuals Interaction between The calculation is as follows:

[0094] ;

[0095] In the formula, For individuals With individuals The Euclidean distance between them The angle between the directions of the two body velocities. The reference distance represents the typical activity range of bird flocks within the monitored area.

[0096] The spatial relationship with the protected airspace is characterized by calculating the Euclidean distance between the individual's location and the boundary of the protected airspace.

[0097] The dynamic spatiotemporal graph is processed by a graph convolutional network, and the information of neighboring nodes is aggregated through the message passing mechanism between nodes to obtain dynamic spatiotemporal graph features containing group interaction information.

[0098] Furthermore, the step of generating the predicted trajectories for each bird target includes:

[0099] A three-dimensional convolutional network was used to extract the flight posture and wingbeat frequency of each bird target to obtain fine-grained behavioral features;

[0100] The fine-grained behavioral features are fused with the dynamic spatiotemporal graph features;

[0101] The fused features are input into an LSTM-based temporal prediction network to generate predicted trajectories for each bird target.

[0102] Specifically, a three-dimensional convolutional network is employed to extract the spatiotemporal cube centered on each bird target across a continuous sequence of video frames. This three-dimensional convolutional network can capture and output fine-grained behavioral feature vectors characterizing microscopic movements such as bird flight posture and wingbeat frequency. Specifically, flight posture is obtained by calculating the rate of change of the aspect ratio and the rotation angle of the target bounding box in consecutive frames, and these are combined to form the posture feature vector. Wingbeat frequency is obtained by performing a Fast Fourier Transform on the time series of pixel intensity within the target region across consecutive frames, extracting the frequency component with the largest power spectrum amplitude as the wingbeat frequency feature.

[0103] The fine-grained behavioral features are fused with the dynamic spatiotemporal graph features in a multimodal manner. The fusion process involves concatenating features and then passing them through fully connected layers. Two fully connected network layers map the concatenated features to a unified feature space, which is then used as the fused features input to the subsequent temporal prediction network.

[0104] Subsequently, the fused high-dimensional features are input into an LSTM-based temporal prediction network to learn the mapping relationship from the current and historical states to the future states, thereby deducing the motion trajectory of each bird target in future images, that is, outputting the sequence of its predicted path points, and completing the inference from the current observation to the future trend.

[0105] S400: Based on the predicted trajectory, calculate the spatial distance, approach speed, and intersection time between each bird target and the protected airspace, analyze the behavioral uncertainty index generated by the predicted trajectory, and combine the spatial distance, approach speed, intersection time, and behavioral uncertainty index to obtain the risk intensity value of each bird target;

[0106] Furthermore, the analysis of the uncertainty indicators of the predicted trajectory generation behavior includes:

[0107] Extract the acceleration change features from the predicted trajectory;

[0108] By analyzing the acceleration change characteristics, abrupt changes in motion patterns can be identified.

[0109] The behavioral uncertainty index is calculated based on the abrupt change characteristics of the aforementioned motion pattern.

[0110] Specifically, based on the predicted trajectory point sequences of each bird target obtained from S300, three types of basic physical risk parameters between the bird and the protected airspace are calculated.

[0111] The shortest spatial distance between each predicted point in the trajectory sequence and the protected airspace boundary is calculated by traversing each point on the predicted path, calculating the Euclidean distance from each point to the protected airspace boundary, and taking the minimum value as the shortest spatial distance.

[0112] The analysis examines the approaching trajectory towards the protected airspace and calculates its average approach velocity. Specifically, this is obtained by dividing the displacement change between consecutive predicted points towards the protected airspace by the time interval. Let the... The target location at any given time is The nearest point to the protected airspace boundary is Then the approximate speed The calculation is as follows:

[0113] ;

[0114] in, For time intervals, This represents the Euclidean distance.

[0115] Based on the current motion state, the estimated rendezvous time between the target and the protected airspace is calculated by dividing the shortest spatial distance by the average approach speed.

[0116] By analyzing abrupt changes in movement patterns in the historical and predicted trajectory sequences of target birds, a behavioral uncertainty index is generated to quantify the additional risks associated with maneuvering behavior. Specifically, acceleration variation features are extracted from the trajectory sequences, and an acceleration sequence is obtained by calculating the rate of change of the velocity vector over a continuous time period. Abrupt changes in movement patterns are identified by analyzing these acceleration variation features; abrupt changes are defined as significant changes in the amplitude or direction of acceleration. Based on these abrupt changes in movement patterns, a behavioral uncertainty index is calculated. By combining the mean and standard deviation of acceleration changes, the probability and magnitude of sudden turns, changes in speed, and other behaviors in birds are assessed. Behavioral Uncertainty Index The calculation is as follows:

[0117] ;

[0118] in, The mean of the acceleration change. This represents the standard deviation of acceleration variation. This indicator considers both the average level and the degree of fluctuation in acceleration variation, thus obtaining a quantitative value that reflects the unpredictability of its behavior.

[0119] Furthermore, the step of obtaining the risk intensity value for each bird target includes:

[0120] A basic risk value is calculated based on the aforementioned spatial distance, approach speed, and rendezvous time.

[0121] The aforementioned behavioral uncertainty index is quantified into a risk correction coefficient;

[0122] The risk intensity value of each bird target is obtained by correcting the base risk value using the risk correction coefficient.

[0123] Specifically, the basic risk value is calculated based on the aforementioned spatial distance, approach speed, and convergence time.

[0124] Time risk value Based on the intersection time, a normalized nonlinear mapping is used:

[0125] ;

[0126] in To estimate the meeting time, This is a time scale parameter. The shorter the convergence time, the better. The closer it is to 1, the higher the time risk.

[0127] Distance risk value Calculated based on spatial distance:

[0128] ;

[0129] in For the shortest spatial distance, This is a distance scale parameter. The closer the distance, the greater the distance. The closer it is to 1, the higher the risk.

[0130] Speed ​​risk value Calculated based on approach speed:

[0131] ;

[0132] in To approach the speed, This is a velocity-scale parameter. The faster the approach velocity, the higher the velocity. The closer it is to 1, the higher the speed risk.

[0133] All three risk values ​​mentioned above are normalized to Interval.

[0134] The behavioral uncertainty index is quantified into a risk correction coefficient, which reflects the risk increment caused by bird maneuvering. The base risk value is then corrected using this risk correction coefficient to obtain the risk intensity value for each bird target.

[0135] Specifically, the aforementioned basic physical parameters and behavioral uncertainty indicators are input into the airspace risk attention module. This module assigns importance to different risk parameters using preset attention weights, and the comprehensive risk intensity value R can be expressed as:

[0136] ;

[0137] in, Indicates the time risk value. Indicates the distance to the risk value. Indicates the speed risk value. , , For attention weights, and , As an uncertainty amplification factor, This is a behavioral uncertainty indicator. The attention weight and uncertainty amplification factor are determined based on historical data statistics and field testing, and are used to adjust the relative importance of each risk parameter.

[0138] The aforementioned nonlinear fusion model maps multi-source risk information to the comprehensive risk intensity value of each target, thus completing a dynamic quantitative assessment of bird strike risk.

[0139] S500: The risk intensity value and environmental data are fused to generate a dynamic risk heat map and overlaid on the monitoring video stream for display. When the risk intensity value exceeds a preset threshold, an early warning is triggered and the bird deterrent device is activated.

[0140] Furthermore, the step of fusing the risk intensity value and environmental data to generate a dynamic risk heatmap includes:

[0141] Acquire environmental wind speed and direction data, and convert the wind speed and direction data into spatial influence factors;

[0142] The risk intensity value is fused with the spatial impact factor to calculate the comprehensive risk intensity value of each spatial unit;

[0143] A dynamic risk heatmap is generated based on the comprehensive risk intensity value.

[0144] Specifically, the risk intensity values ​​and behavioral uncertainty indicators of each bird target obtained by S400 and the environmental wind speed and direction data collected in real time by S100 are normalized to unify the data of different dimensions into the same numerical range.

[0145] Environmental wind speed and direction data are acquired and converted into spatial influence factors. Specifically, the impact of the wind field on the deviation of bird flight paths is calculated based on wind speed and wind direction. Spatial Influence Factors The calculation is as follows:

[0146] ;

[0147] In the formula, For wind speed, For reference wind speed, This is the angle between the wind direction and the bird's flight direction. Normalized by dividing by a reference wind speed, the greater the wind speed and the larger the vertical component of the wind direction relative to the flight direction, the greater the spatial influence factor, characterizing the degree to which the wind field interferes with the stability of bird trajectories.

[0148] The risk intensity value is then fused with the spatial impact factor to calculate the comprehensive risk intensity value for each spatial unit. (Comprehensive Risk Intensity Value) The calculation is as follows:

[0149] ;

[0150] In the formula, This is the original risk intensity value. Spatial influence factor, Environmental impact coefficient. Weighting coefficient. It increases with the increase of spatial influence factors, reflecting the amplifying effect of environmental factors on risk.

[0151] A dynamic risk heatmap is generated based on the comprehensive risk intensity value. A Gaussian kernel interpolation algorithm is used to divide the monitored airspace into grids, mapping the comprehensive risk intensity value of each bird target location to the corresponding spatial grid cell. For the grid center point, its risk value is obtained by weighted summation of the risk contributions of surrounding detected bird targets. The weights are calculated using a Gaussian kernel function, with closer points having larger weights, forming a continuous risk distribution covering the entire monitored airspace. Based on the magnitude of the risk intensity value, a color mapping scheme is used to convert the values ​​into colors, such as green for low-risk areas, yellow for medium-risk areas, and red for high-risk areas, generating a dynamic risk heatmap.

[0152] The video fusion module precisely registers and overlays the heatmap with the spatial location in the original surveillance video stream in the form of a semi-transparent color layer, generating an augmented reality display. The registration process uses the camera's intrinsic and extrinsic parameter matrices to transform the heatmap's world coordinate system to the image pixel coordinate system, ensuring accurate spatial correspondence between the heatmap and the video frame, and providing a clear visual representation of the risk spatial distribution in real-time video.

[0153] The system has preset multiple risk intensity thresholds, including low-risk, medium-risk, and high-risk thresholds, corresponding to different warning levels. These thresholds are determined based on historical data statistics and on-site testing, and are used to divide risk level ranges. When the overall risk intensity value of a specific area in the augmented reality image exceeds the set threshold, a warning signal of the corresponding level is automatically triggered. Based on the warning level, corresponding intensity of sound and light bird deterrent equipment is activated; for example, sound deterrents are activated for low risk, a combination of sound and light deterrents is activated for medium risk, and high-intensity sound and light flashing deterrents are activated for high risk. Simultaneously, an alarm message containing the risk location, level, and recommended handling measures is sent to relevant control units.

[0154] In addition, the system continuously records multimodal data during operation, including bird detection events, movement trajectory data, early warning trigger records, and corresponding original video clips, and organizes them in a structured manner according to timestamps and event types to form a traceable historical database.

[0155] New data within a preset time window is periodically extracted from the historical database and incrementally trained on the bird detection model and risk prediction model in batches. An elastic learning rate strategy is employed during incremental training, with the initial learning rate set to one-tenth of the initial training rate. This learning rate is gradually reduced as training progresses using exponential decay or cosine annealing to ensure the model retains its ability to recognize existing scenes while absorbing new knowledge, thus avoiding catastrophic forgetting.

[0156] Specifically, for incremental training of the YOLOv8m-Bird detection model, fine-tuning was performed using newly added detection event data from the historical database, with a focus on optimizing the model's adaptability to newly emerging scene features. For incremental training of the risk prediction model, the correspondence between trajectory data in historical warning records and actual risk events was used to optimize the parameters of the temporal prediction network and the spatial risk attention module.

[0157] The performance of the incrementally trained model is evaluated by testing its accuracy, recall, and generalization ability on the validation set. When the evaluation metrics meet the preset standards, the optimized model parameters are deployed to the online inference system via hot update, completing the model replacement without interrupting system operation and achieving continuous adaptive improvement of the system's identification and early warning capabilities.

[0158] Through the aforementioned data archiving and incremental model optimization mechanisms, the system possesses the ability to adapt to environmental changes and evolve on its own over the long term, effectively ensuring the continuity and stability of system performance.

[0159] Example 2: This invention also provides a real-time bird strike risk early warning system based on video stream analysis, such as... Figure 2 As shown, it includes:

[0160] The video preprocessing module is used to acquire the monitoring video stream of key areas, preprocess the monitoring video stream using a local adaptive enhancement algorithm, construct local feature descriptors based on the local neighborhood features of pixels, and apply different enhancement parameters to different areas based on the local feature descriptors.

[0161] The bird detection module is used to perform bird target detection on the preprocessed video stream using the YOLOv8m-Bird detection model and output the detection results of each bird target.

[0162] The trajectory prediction module is used to construct a dynamic spatiotemporal map based on the detection results, obtain dynamic spatiotemporal map features, extract fine-grained behavioral features of each bird target, fuse the fine-grained behavioral features with the dynamic spatiotemporal map features and input them into a temporal prediction network to generate the predicted trajectory of each bird target.

[0163] The risk assessment module is used to calculate the spatial distance, approach speed, and intersection time between each bird target and the protected airspace based on the predicted trajectory, analyze the behavioral uncertainty index generated by the predicted trajectory, and obtain the risk intensity value of each bird target by combining the spatial distance, approach speed, intersection time, and behavioral uncertainty index.

[0164] The early warning display module is used to fuse the risk intensity value and environmental data to generate a dynamic risk heat map and overlay it with the monitoring video stream. When the risk intensity value exceeds a preset threshold, an early warning is triggered and the bird deterrent device is activated.

[0165] The implementation methods of each module of the system are the same as those in Embodiment 1, and will not be repeated here.

[0166] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time bird strike risk early warning method based on video stream analysis, characterized in that, include: S100: Acquire the monitoring video stream of the key area, preprocess the monitoring video stream using a local adaptive enhancement algorithm, construct a local feature descriptor based on the local neighborhood features of the pixels, and apply different enhancement parameters to different areas based on the local feature descriptor; S200: The YOLOv8m-Bird detection model is used to detect bird targets in the preprocessed video stream, and the detection results of each bird target are output. S300: Construct a dynamic spatiotemporal graph based on the detection results to obtain dynamic spatiotemporal graph features; Fine-grained behavioral features of each bird target are extracted, and the fine-grained behavioral features are fused with the dynamic spatiotemporal map features and then input into a temporal prediction network to generate the predicted trajectory of each bird target. S400: Based on the predicted trajectory, calculate the spatial distance, approach speed, and intersection time between each bird target and the protected airspace, analyze the behavioral uncertainty index generated by the predicted trajectory, and combine the spatial distance, approach speed, intersection time, and behavioral uncertainty index to obtain the risk intensity value of each bird target; S500: The risk intensity value and environmental data are fused to generate a dynamic risk heat map and overlaid on the monitoring video stream for display. When the risk intensity value exceeds a preset threshold, an early warning is triggered and the bird deterrent device is activated. The step of preprocessing the surveillance video stream using a local adaptive enhancement algorithm includes: Calculate the consistency between texture intensity and gradient direction in the local neighborhood of a pixel to construct a local feature descriptor; The texture complexity of the region is determined based on the local feature descriptor. Apply high-gain enhancement parameters to regions with high texture complexity and low-gain enhancement parameters to regions with low texture complexity. The application of high-gain enhancement parameters to regions with high texture complexity and low-gain enhancement parameters to regions with low texture complexity includes: The shape parameters of the grayscale transformation curve are dynamically adjusted according to the local feature descriptor, generating steep transformation curves for regions with high texture complexity and flat transformation curves for regions with low texture complexity. The steps for constructing a dynamic spatiotemporal graph based on the detection results include: A node is created for each bird target, and the node represents the visual features and motion state of each bird target; Establish edges between nodes, which represent the interaction relationships within the bird flock and the spatial relationship between each bird target and the protected airspace; The dynamic spatiotemporal graph is processed to obtain dynamic spatiotemporal graph features containing group interaction information.

2. The method for real-time early warning of bird strike risk based on video stream analysis according to claim 1, characterized in that, The YOLOv8m-Bird detection model is an improved model that embeds convolutional block attention modules into the backbone and neck networks of the YOLOv8m model. The convolutional block attention module includes a channel attention submodule and a spatial attention submodule; The channel attention submodule assigns differentiated weights to feature channels through global average pooling and max pooling; The spatial attention submodule generates a spatial weight mask through convolution operations.

3. The method for real-time early warning of bird strike risk based on video stream analysis according to claim 1, characterized in that, The steps for generating the predicted trajectories of each bird target include: A three-dimensional convolutional network was used to extract the flight posture and wingbeat frequency of each bird target to obtain fine-grained behavioral features; The fine-grained behavioral features are fused with the dynamic spatiotemporal graph features; The fused features are input into an LSTM-based temporal prediction network to generate predicted trajectories for each bird target.

4. The method for real-time early warning of bird strike risk based on video stream analysis according to claim 1, characterized in that, The analysis of the uncertainty indicators of the predicted trajectory generation behavior includes: Extract the acceleration change features from the predicted trajectory; By analyzing the acceleration change characteristics, abrupt changes in motion patterns can be identified. The behavioral uncertainty index is calculated based on the abrupt change characteristics of the aforementioned motion pattern.

5. A real-time bird strike risk warning method based on video stream analysis according to claim 1, characterized in that, The steps for obtaining the risk intensity value of each bird target include: A basic risk value is calculated based on the aforementioned spatial distance, approach speed, and rendezvous time. The aforementioned behavioral uncertainty index is quantified into a risk correction coefficient; The risk intensity value of each bird target is obtained by correcting the base risk value using the risk correction coefficient.

6. The method for real-time early warning of bird strike risk based on video stream analysis according to claim 1, characterized in that, The steps for fusing the risk intensity values ​​and environmental data to generate a dynamic risk heatmap include: Acquire environmental wind speed and direction data, and convert the wind speed and direction data into spatial influence factors; The risk intensity value is fused with the spatial impact factor to calculate the comprehensive risk intensity value of each spatial unit; A dynamic risk heatmap is generated based on the comprehensive risk intensity value.

7. A real-time bird strike risk early warning system based on video stream analysis, characterized in that, include: The video preprocessing module is used to acquire the monitoring video stream of key areas, preprocess the monitoring video stream using a local adaptive enhancement algorithm, construct local feature descriptors based on the local neighborhood features of pixels, and apply different enhancement parameters to different areas based on the local feature descriptors. The bird detection module is used to perform bird target detection on the preprocessed video stream using the YOLOv8m-Bird detection model and output the detection results of each bird target. The trajectory prediction module is used to construct a dynamic spatiotemporal map based on the detection results and obtain dynamic spatiotemporal map features. Fine-grained behavioral features of each bird target are extracted, and the fine-grained behavioral features are fused with the dynamic spatiotemporal map features and then input into a temporal prediction network to generate the predicted trajectory of each bird target. The risk assessment module is used to calculate the spatial distance, approach speed, and intersection time between each bird target and the protected airspace based on the predicted trajectory, analyze the behavioral uncertainty index generated by the predicted trajectory, and obtain the risk intensity value of each bird target by combining the spatial distance, approach speed, intersection time, and behavioral uncertainty index. The early warning display module is used to fuse the risk intensity value and environmental data to generate a dynamic risk heat map and overlay it with the monitoring video stream. When the risk intensity value exceeds a preset threshold, an early warning is triggered and the bird deterrent device is activated. The preprocessing of the surveillance video stream using the local adaptive enhancement algorithm includes: Calculate the consistency between texture intensity and gradient direction in the local neighborhood of a pixel to construct a local feature descriptor; The texture complexity of the region is determined based on the local feature descriptor. Apply high-gain enhancement parameters to regions with high texture complexity and low-gain enhancement parameters to regions with low texture complexity. The application of high-gain enhancement parameters to regions with high texture complexity and low-gain enhancement parameters to regions with low texture complexity includes: The shape parameters of the grayscale transformation curve are dynamically adjusted according to the local feature descriptor, generating steep transformation curves for regions with high texture complexity and flat transformation curves for regions with low texture complexity. Constructing a dynamic spatiotemporal graph based on the detection results includes: A node is created for each bird target, and the node represents the visual features and motion state of each bird target; Establish edges between nodes, which represent the interaction relationships within the bird flock and the spatial relationship between each bird target and the protected airspace; The dynamic spatiotemporal graph is processed to obtain dynamic spatiotemporal graph features containing group interaction information.

Images