A bird strike prevention intelligent management method, system, medium and equipment

Abstract

This invention relates to the field of multimodal data processing technology, and provides an intelligent management method, system, medium, and device for bird strike prevention. The method includes: for each image frame, extracting feature points in the foreground region, calculating the velocity variance and orientation variance of the feature points using sparse optical flow, and eliminating false foreground elements; for each image frame's foreground region, segmenting it to obtain several target detection boxes, and assigning a label number to each target detection box using a greedy matching method; for target detection boxes with the same label number, generating a motion trajectory, and extracting visual feature vectors, physical scale change vectors, and velocity change vectors; for sound signals, extracting sound feature vectors, and combining the motion trajectory, visual feature vectors, physical scale change vectors, and velocity change vectors, obtaining bird species identification results through multimodal feature joint modeling, and formulating a deterrent strategy. This effectively improves the classification accuracy of common airport birds and meets the high-precision bird strike prevention requirements of airports.

Description

Technical Field

[0001] This invention belongs to the field of multimodal data processing technology, and in particular relates to an intelligent management method, system, medium and equipment for bird strike prevention. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The rapid development of air transport has placed stringent requirements on airport operation safety. Bird strikes, as a high-frequency safety hazard in airport operations, can not only damage key components such as aircraft fuselages and engines, but in severe cases can also lead to major safety accidents such as loss of control, forced landings, or even crashes, causing huge economic losses to the air transport industry and posing a serious threat to the lives of crew members and passengers.

[0004] Currently, the mainstream bird strike prevention methods at airports mainly rely on manual patrols, fixed bird deterrent equipment, and traditional monitoring technologies. Some airports have gradually introduced bird detection systems based on machine vision, which has improved the intelligence level of bird strike prevention to some extent. However, for the complex application scenarios of airport airspace, existing technologies still have many technical shortcomings and cannot meet the high-precision, high-real-time, and high-robustness bird strike prevention requirements of airports. Specifically, these shortcomings are reflected in the following aspects:

[0005] Existing bird identification technologies mostly rely on single visual or auditory features, without combining bird flight behavior features for multi-dimensional discrimination. The complex background environment of airport airspace can easily interfere with visual feature extraction, while background noise such as aircraft engines, wind noise, and field operations can also mask bird call features. The limitations of single modal features lead to low accuracy in bird classification.

[0006] Airport airspace is an open and dynamically changing complex environment with various interference factors such as slow cloud drift, drastic changes in lighting (such as alternating sunshine and rain, and backlighting at dawn and dusk), camera noise, and atmospheric scattering. Traditional foreground segmentation techniques such as Gaussian mixture models (GMM) and background subtraction methods often adopt a global background update strategy, which is prone to generating a large number of false foregrounds due to cloud drift and sudden lighting disturbances. At the same time, they are not sensitive enough to detect small aerial targets with low contrast, such as birds, in airport airspace, making it difficult to effectively distinguish between real targets and false foregrounds, resulting in a significant increase in the misjudgment rate in subsequent tracking and recognition stages. Summary of the Invention

[0007] To address the technical problems mentioned above, this invention provides an intelligent management method, system, medium, and device for bird strike prevention. It integrates multi-dimensional features such as vision, sound, physical scale, flight speed, and motion trajectory. The joint modeling of multi-modal features compensates for the limitations of single modality, effectively resists the influence of airport background noise and complex environment on feature extraction, and effectively improves the classification accuracy of common airport birds, thus meeting the high-precision bird strike prevention requirements of airports.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] The first aspect of the present invention provides a smart management method for bird strike prevention, comprising:

[0010] Acquire sound signals from the target area and several consecutive frames of images captured by multiple cameras;

[0011] For each frame of image, a Gaussian mixture model is used to extract the foreground region, and feature points are extracted within the foreground region. The velocity variance and orientation variance of the feature points are calculated using the sparse optical flow method to eliminate false foreground. For the foreground region of each frame of image, several target detection boxes are obtained. For each target detection box, the target is located based on its position in images captured by different cameras at the same time to obtain the three-dimensional spatial position. For each target detection box, a label number is assigned using a greedy matching method. For target detection boxes with the same label number, the three-dimensional spatial positions at several consecutive time steps are stitched together to generate a motion trajectory.

[0012] For sound signals, the Mel frequency cepstral coefficients are used for feature extraction to obtain sound feature vectors;

[0013] For target detection boxes with the same label, extract the visual feature vector, physical scale change vector, and velocity change vector;

[0014] For all target detection boxes with marked numbers, bird species identification results are obtained through multimodal feature joint modeling based on motion trajectory, visual feature vector, physical scale change vector, and velocity change vector, combined with sound feature vector. Based on the bird species identification results, a driving strategy is formulated.

[0015] Furthermore, when extracting the foreground region, a flicker counter C(x,y,t) is set for each pixel (x,y); if the pixel state changes between frame t and frame (t-1), then C(x,y,t) = C(x,y,t-1) + 1; if the pixel state does not change between frame t and frame (t-1), then C(x,y,t) = max(0,C(x,y,t-1) - 0.5); when C(x,y,t) ≥ the counter threshold, the update of the Gaussian distribution parameters of pixel (x,y) is paused.

[0016] Furthermore, when executing the greedy matching method, for the t-th frame image, delayed decision tracking is triggered if any of the following conditions are met:

[0017] If the Euclidean distance between two or more target detection boxes is less than a preset distance threshold;

[0018] The same detection box matches multiple historical marker numbers, or multiple detection boxes compete for the same historical marker number.

[0019] Furthermore, the three-dimensional spatial position of a target detection box at time t is (AB, AC):

[0020] The actual straight-line distance between the target detection box and camera B ;

[0021] The actual straight-line distance between the target detection box and camera C ;

[0022] Wherein, the horizontal distance between the target detection box and camera B ; Horizontal distance between the target detection box and camera C The horizontal offset angles of the target detection bounding boxes relative to cameras B and C are respectively... and The vertical offset angles of the target detection bounding boxes relative to cameras B and C are respectively... and .

[0023] Furthermore, the horizontal offset angles of the target detection box relative to camera B and camera C are respectively:

[0024] ;

[0025] ;

[0026] The images captured by cameras B and C both have a resolution of W×H; the horizontal field of view of cameras B and C are both γ; the pixel coordinates of the center position of the target detection box in the image captured by camera B are (i B ,j B The pixel coordinates in the image captured by camera C are (i C ,j C ).

[0027] Furthermore, the vertical offset angles of the target detection box relative to camera B and camera C are respectively:

[0028] ;

[0029] ;

[0030] The images captured by cameras B and C both have a resolution of W×H; the vertical field of view of cameras B and C are both β; the pixel coordinates of the center position of the target detection box in the image captured by camera B are (i B ,j B The pixel coordinates in the image captured by camera C are (i C ,j C ).

[0031] Furthermore, for audio signals, before using Mel frequency cepstral coefficients for feature extraction, pre-emphasis, framing, Hanning windowing, and endpoint detection operations are performed sequentially.

[0032] A second aspect of the present invention provides a bird strike prevention intelligent management system, comprising:

[0033] The data acquisition module is configured to acquire sound signals from the target area and several consecutive frames of images captured by multiple cameras.

[0034] The trajectory generation module is configured as follows: for each frame of image, the foreground region is extracted using a Gaussian mixture model, feature points are extracted within the foreground region, and the velocity variance and orientation variance of the feature points are calculated using the sparse optical flow method to eliminate false foreground; for the foreground region of each frame of image, several target detection boxes are segmented; for each target detection box, target localization is performed based on its position in images captured by different cameras at the same time to obtain its three-dimensional spatial position; for each target detection box, a label number is assigned using a greedy matching method; for target detection boxes with the same label number, the three-dimensional spatial positions at several consecutive time steps are stitched together to generate a motion trajectory.

[0035] The sound processing module is configured to extract features from a sound signal using Mel-frequency cepstral coefficients to obtain a sound feature vector.

[0036] The feature extraction module is configured to extract visual feature vectors, physical scale change vectors, and velocity change vectors for target detection boxes with the same label number.

[0037] The classification module is configured to: for all labeled target detection boxes, based on motion trajectory, visual feature vector, physical scale change vector, and velocity change vector, combined with sound feature vector, obtain bird species identification results through multimodal feature joint modeling, and formulate a driving strategy based on the bird species identification results.

[0038] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the bird strike prevention intelligent management method described above.

[0039] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of the bird strike prevention intelligent management method described above.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] This invention integrates multi-dimensional features such as vision, sound, physical scale, flight speed, and motion trajectory. The joint modeling of multi-modal features makes up for the limitations of single modality, effectively resists the influence of airport background noise and complex environment on feature extraction, and effectively improves the classification accuracy of common airport birds (large birds of prey, small sparrows, waterbirds, etc.), meeting the high-precision bird strike prevention requirements of airports.

[0042] This invention significantly solves the problem of false foregrounds caused by interference from cloud drift, sudden changes in illumination, and camera noise in airport airspace by improving the fusion foreground detection method of Gaussian mixture model (GMM) and sparse optical flow tracking. On the one hand, the design of pixel-level flicker counter and nonlinear threshold model realizes adaptive control of background model update, effectively suppressing false foregrounds with frequent state switching (such as illumination disturbances and noise), and greatly improving the accuracy of foreground segmentation. On the other hand, motion consistency verification (dual indicators of velocity variance and orientation variance) specifically eliminates false targets caused by non-rigid deformation at cloud edges. Combined with the efficient computational characteristics of sparse optical flow, it has excellent detection capability for small-scale, low-contrast weak bird targets while meeting real-time requirements, significantly reducing missed detections and far exceeding the detection performance of traditional GMM, single background difference methods, and other techniques.

[0043] This invention can accurately eliminate quiet areas without bird calls and strong interference noise segments in airport environments, ensuring that subsequent MFCC feature extraction is only for valid bird call signals, thereby improving the discriminative power and anti-interference ability of sound features. Attached Figure Description

[0044] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0045] Figure 1 This is a flowchart of a bird strike prevention intelligent management method according to Embodiment 1 of the present invention;

[0046] Figure 2 This is a schematic diagram showing the offset between the target center position and the image center position in Embodiment 1 of the present invention;

[0047] Figure 3 This is a schematic diagram showing the distance between the target and the camera in Embodiment 1 of the present invention;

[0048] Figure 4 This is a schematic diagram of the dual-camera installation according to Embodiment 1 of the present invention;

[0049] Figure 5 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0051] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0052] Example 1

[0053] This embodiment provides an intelligent management method for bird strike prevention.

[0054] This embodiment provides an intelligent management method for bird strike prevention, such as... Figure 1 As shown, it includes the following steps:

[0055] Step 1: Acquire a series of consecutive frames of images (from at least two cameras) and audio signals of the target area.

[0056] For images of the target area, the acquisition parameters are: image frame rate ≥ 25fps, and resolution uniformly W×H (e.g., 1920×1080).

[0057] For the sound signal in the target area, the acquisition parameters are: sampling rate ≥ 44.1kHz, quantization bit depth 16bit.

[0058] The dual cameras are installed at a fixed distance of 200 meters, with their optical axes parallel and facing the same direction. Furthermore, adjacent cameras have at least half of their viewing angles overlapping. Figure 4 As shown.

[0059] Step 2: A foreground detection method that integrates an improved Gaussian mixture model (GMM) with sparse optical flow tracking.

[0060] To address the challenges of frequent cloud changes, complex lighting conditions, and small-scale, low-contrast aerial targets in airport airspace, this embodiment proposes a foreground detection method that integrates an improved Gaussian mixture model and sparse optical flow analysis. This method first uses the improved Gaussian mixture model to perform online modeling of the dynamic sky background, achieving preliminary segmentation of the foreground region. Based on this, sparse optical flow tracking is introduced to analyze the motion characteristics of candidate foreground regions. By evaluating the consistency of target motion direction, false foreground regions caused by slow cloud drift, background noise, or lighting disturbances are effectively eliminated. Simultaneously, the learning rate of the background model is dynamically adjusted based on the detected real target motion characteristics to suppress ghosting effects and improve the model's adaptability to background changes. This method achieves stable detection of weak aerial targets under complex sky background conditions, significantly improving real-time performance and anti-interference capabilities.

[0061] Step 201: Anti-interference background modeling.

[0062] For each frame of the image, GMM performs real-time background modeling.

[0063] (1) Pixel state judgment: The gray value of pixel (x,y) in the t-th frame is I(x,y,t), and the parameter of the k-th Gaussian distribution in the GMM is (μ k (x,y,t-1),σ k 2 (x,y,t-1),ω k (x,y,t-1)), where μ k (x,y,t-1),σ k 2 (x,y,t-1) and ω k Let (x, y, t-1) be the mean, variance, and weight of pixel (x, y) in frame t-1, respectively. Then, the matching condition between pixel I(x, y, t) and the Gaussian distribution is: |I(x, y, t) - μ k (x,y,t-1)∣≤2.5σ k (x,y,t-1); If a matching Gaussian distribution exists, the pixel state is determined to be background; otherwise, it is determined to be foreground.

[0064] (2) Flicker Counter Design: Set a flicker counter C(x,y,t) for each pixel, with an initial value of 0; if the pixel state changes between frame t and frame t-1 (foreground) If the background is not changed, then C(x,y,t) = C(x,y,t-1) + 1; if the pixel state is not changed, then C(x,y,t) = max(0,C(x,y,t-1)-0.5), and slowly decays.

[0065] (3) Background model update control: If the counter threshold is T (corresponding to frequent switching within t frames), when C(x, y, t) ≥ T, the background model update of this pixel is paused. That is, the Gaussian distribution parameters μ k (x, y, t), σ k 2 (x, y, t), ω k (x, y, t) and μ k (x, y, t - 1), σ k 2 (x, y, t - 1), ω k (x, y, t - 1) remain the same; when C(x, y, t) < T, the parameters are adjusted according to the conventional GMM update rule: μ k (x, y, t) = (1 - α)μ k (x, y, t - 1) + αI(x, y, t), σ k 2 (x, y, t) = (1 - α)σ k 2 (x, y, t - 1) + α(I(x, y, t) - μ k (x, y, t)) 2 , ω k (x, y, t) = (1 - α)ω k (x, y, t - 1) + αM; where α is the initial learning rate and M is the matching flag (M = 1 when matching, otherwise M = 0).

[0066] Among them, the counter threshold T is calculated based on the camera frame rate fps: , where k is the scene adaptation experience coefficient, is the frame rate attenuation coefficient, and round() is the rounding function. In this embodiment, combined with the non - linear characteristics of the dynamic changes of light / clouds in the airport airspace, the natural constant e is introduced to construct a non - linear threshold model to adapt to the non - uniform characteristics of pixel state switching.

[0067] In this embodiment, a "flicker counter" is established for each pixel. If a certain pixel frequently switches between the "foreground" and "background" states within T frames (such as noise or flash), and the counter exceeds the threshold, by "pausing the background model update of specific pixels", it is possible to avoid interference signals from misleading the background model judgment, thereby improving the accuracy of foreground segmentation and effectively suppressing false foregrounds generated by camera noise and sudden light changes.

[0068] Step 202: Motion consistency verification.

[0069] For each frame of the image, feature points p are extracted within the extracted foreground region; the motion vectors of the feature points are calculated using the sparse optical flow method, and the velocity variance and orientation variance of the motion vectors of all feature points within the foreground region are calculated.

[0070] (1) Calculation of optical flow vector: Let the coordinates of feature point p in frame t be (u t ,v t The coordinates of the (t+1)th frame are (u t+1 ,v t+1 If the optical flow vector is: .

[0071] (2) Calculation of speed and direction: Let the time interval between adjacent frames be Δt = 1 / fps, where fps is the camera frame rate, then the amplitude of the motion speed of the feature point is The direction of motion (the angle between the motion and the positive x-axis) is: , .

[0072] (3) Foreground region motion consistency assessment: Suppose a foreground region contains N feature points, calculate the motion consistency of the region:

[0073] Speed ​​variance (reflects speed consistency): ;in, Average speed;

[0074] Directional variance (reflecting directional consistency): ;in, The average direction.

[0075] (4) False foreground removal rule: Let the velocity variance threshold be... The directional variance threshold is ,but:

[0076] like and Items deemed false foreground (such as cloud edges) are removed.

[0077] Otherwise, it is determined to be a real foreground area and will proceed to subsequent tracking.

[0078] This embodiment utilizes the collinearity characteristics of optical flow vectors (low velocity variance + low orientation variance) to eliminate false foregrounds caused by non-rigid deformation at cloud edges.

[0079] Step 3: 3D positioning and trajectory generation of aerial targets.

[0080] Step 301: Target segmentation.

[0081] For the t-th frame image, based on the foreground region retained after optical flow filtering, several target detection boxes are obtained through the image segmentation model.

[0082] Step 302: Target Positioning.

[0083] Assume that the image resolutions captured by cameras B and C are both W×H, the horizontal field of view of cameras B and C are both γ and the vertical field of view of cameras C are both β, and the distance between cameras B and C is L; for a certain target detection box (hereinafter referred to as the target), the pixel coordinates of the center position of the target (the center position of the target detection box) in the image captured by camera B are (i B ,j B In the image captured by camera C, the pixel coordinates of the center position of the target are (i... C ,j C The center coordinates of the images captured by camera B and camera C are both... .

[0084] like Figure 2 As shown, taking camera B as an example, the offset between the target center position and the image center position is... , ;

[0085] like Figure 3 As shown, D is the projection point of the target on the ground. Calculate the horizontal offset angle of the target relative to camera B and camera C. , and vertical offset angle , :

[0086] ;

[0087] ;

[0088] ;

[0089] ;

[0090] in, Set the camera to an upward angle.

[0091] According to the Law of Sines:

[0092] Horizontal distance of the target from camera B: ;

[0093] Horizontal distance of the target from camera C: .

[0094] According to the cosine formula:

[0095] The actual straight-line distance between the target and camera B: ;

[0096] The actual straight-line distance between the target and camera C: ;

[0097] Finally, the actual straight-line distances from the target to the two cameras were obtained:

[0098] ;

[0099] .

[0100] Let (AB, AC) be the three-dimensional spatial position of the target detection box at time t.

[0101] Step 303: Data association.

[0102] (1) For the first frame image, cross-camera spatial geometric association matching is performed on the target detection boxes extracted from the images captured by multiple cameras. The interrelated target detection boxes are filtered by the parallax features and position mapping relationship of dual-camera imaging, and a unique global target ID is assigned to each group of related detection boxes to realize the unified identification of the same aerial target by multiple cameras.

[0103] Specifically, based on the installation parameters of parallel optical axes, consistent orientation, and fixed spacing of the two cameras, combined with the camera intrinsic parameters (resolution W×H, horizontal field of view γ, vertical field of view β), the theoretical mapping position of each detection box in camera B to the imaging plane of camera C is calculated, and the Euclidean distance between this theoretical mapping position and the center of all detection boxes in camera C is calculated; if a certain distance is less than the preset cross-camera matching distance threshold D cross If the detection box is found to be a cross-camera associated detection box for the same aerial target, then a unique global continuous number ID is assigned to each cross-camera associated detection box confirmed by geometric matching.

[0104] (2) For each target detection box in the t-th (t>1) frame image, the ID greedy matching method is used to assign an ID (identifier).

[0105] Specifically, for the i-th target detection box in the t-th (t>1) frame image, calculate its distance to all target detection boxes in the (t-1)-th frame image, prioritize matching the target detection box with the closest distance, and inherit the ID of the target detection box matched in the (t-1)-th frame image.

[0106] Step 304: Trajectory generation.

[0107] For target detection boxes with the same label, the three-dimensional spatial positions at several consecutive time points are stitched together to generate a motion trajectory.

[0108] Step 305: Intersection Judgment.

[0109] (1) For the t-th (t>1) frame of the image, delay-delayed tracking is triggered immediately if any of the following conditions are met:

[0110] When two or more target detection boxes are detected, the Euclidean distance between them is less than a preset distance threshold D. th ;

[0111] This can lead to ID contention, where the same detection frame matches multiple historical IDs or multiple detection frames compete for the same historical ID.

[0112] Step 306: Delayed decision tracking.

[0113] (1) Calculation of three predicted trajectories.

[0114] Trajectory 1: Generated using the state transition equation of Kalman filtering. Let the target state vector with ID A at time t-1 be... (Including the target's most accurate position and velocity information at time t-1), then the predicted position P at subsequent time k is... A (t+k) is:

[0115] ;

[0116] ;

[0117] Where F is the state transition matrix and H is the observation matrix; To represent the prior prediction of the target state at time t+k based on the observation information at time t-1; This represents the prediction result of the target state at time t+k-1 based on the observation information at time t-1.

[0118] Trajectory 2: Utilizing the average optical flow vector of all feature points within the target detection bounding box with ID A at time t-1. Perform linear inference to predict the position P at subsequent time k. B (t+k) is:

[0119] ;

[0120] Among them, P obs (t-1) represents the actual observed position of the target before the intersection. This represents the inter-frame time interval.

[0121] Trajectory 3: Weighted fusion of Trajectory 1 and Trajectory 2 to adapt to nonlinear motion, predicting the position P of the target with ID A at subsequent time k. C (t+k) is:

[0122] ;

[0123] in, For fusion weighting coefficients.

[0124] (2) Observation sequence matching and cost calculation.

[0125] During frames t to t+T (where T is the number of frames from the start of target convergence to target separation), acquire the observation positions P of all separated candidate targets. obs (k), calculate the matching cost Cost(k) between the candidate observation sequence and the three virtual trajectories (1, 2, 3).

[0126] The comprehensive cost function is defined as follows:

[0127] ;

[0128] Location Distance Cost D pos (k) is the calculation of the center P of the detection frame. obs (k) and virtual trajectory point P pred Euclidean distance (k):

[0129] ;

[0130] Among them, virtual trajectory point P pred (k) is a trajectory point in trajectory 1, trajectory 2 or trajectory 3.

[0131] Vector angle cost D dir (k) is the movement vector of the detection box. tangent direction of virtual trajectory cosine distance:

[0132] ;

[0133] Among them, w1 and w2 are weighting coefficients.

[0134] This embodiment introduces a delayed decision-making mechanism to solve the ID jump problem when targets intersect.

[0135] (3) ID final confirmation.

[0136] Calculate the cumulative cost of all possible matching combinations within T frames. The matching combination with the lowest cumulative cost is selected as the final identity assignment for the observed sequence.

[0137] Step 4: Use MFCC (Mel frequency cepstral coefficients) to extract features from the collected sound signal to obtain K-dimensional MFCC features, and then stitch together the MFCC features of M consecutive frames to obtain the sound feature vector.

[0138] (1) Sound signal preprocessing: For background interference such as aircraft engine noise and ambient wind noise in the airport environment, pre-emphasis, framing, Hanning windowing and endpoint detection are performed in sequence to obtain a pure target sound segment.

[0139] The endpoint detection steps include:

[0140] First, for each frame of audio signal after pre-emphasis, framing, and windowing, the short-time energy E is calculated. n With zero-crossing rate Z n .

[0141] Then, calculate the short-time energy mean E for the first 10-20 frames of the silent segment (assuming the initial frame is silent). avg With the mean zero-crossing rate Z avg Set energy threshold E th =k1·E avg k1 is the energy amplification factor, which adapts to the energy difference between bird calls and silence, and the zero-crossing rate has a low threshold Z. low =k2·Z avg High threshold for zero crossing rate Z high =k3·Z avg The coefficients k2<1 and k3>1 are used to distinguish between low-frequency noise and high-frequency strong interference, respectively.

[0142] Then, the threshold is updated every 50 frames. If a valid bird song segment is detected, the mean is recalculated using the noise energy / zero-crossing rate of the 10 frames before and after that segment, and E is corrected. th Z low Z high It adapts to the dynamic changes in environmental noise.

[0143] Then, each frame of signal is classified according to the following rules:

[0144] Silent Frame: E n <E th And Z n <Z low ;

[0145] High-noise frames: E n ≥E th And Z n >Z high , or E n <E th And Z n >Z high ;

[0146] Candidate bird song frame: E n ≥E th And Z low ≤Zn ≤Z high .

[0147] Then, segment-level filtering and smoothing are performed:

[0148] Continuous frame verification: Only candidate bird song frame sequences with a continuous length of ≥3 frames are retained and marked as valid bird song segments;

[0149] Edge extension: The effective bird song segment is padded with one frame before and after it to avoid truncating the beginning / end of the bird song;

[0150] Remove isolated segments: Remove isolated candidate frame sequences with a length of less than 3 frames, and finally output continuous and complete valid bird song segments.

[0151] (2) MFCC feature extraction: The preprocessed audio signal is frequency mapped by the Mel filter bank, and the feature correlation is removed by discrete cosine transform (DCT). The first K dimensions (including DC component) are retained as the core MFCC features.

[0152] (3) Feature vector construction: In order to capture the temporal continuity of bird calls, the MFCC features (including optional differential features) of M consecutive frames are spliced ​​in time order to form a sound feature vector with dimension K×M, ensuring that the features can fully represent the temporal change pattern of bird calls.

[0153] Step 5: A bird classification method based on multimodal information fusion and attention mechanism.

[0154] Step 501: Visual feature extraction.

[0155] For a target with ID A, the target region is segmented based on the detection box in all images, and then input into the ResNet50 model (feature extraction model) to extract visual feature vectors.

[0156] In this embodiment, a channel attention layer is inserted after each residual block output of ResNet50, enabling the network to adaptively enhance its attention to key discriminative features such as bird plumage distribution and morphological structure, and suppress the interference of complex backgrounds on classification results.

[0157] Step 502: Extraction of physical scale changes of the target.

[0158] For a target with ID A, calculate: the physical scale of target A in frame t = the number of pixels occupied by target A in the image of frame t × one pixel size / camera focal length;

[0159] By stitching together the physical scales of target A across all images in chronological order, we obtain the target's physical scale change vector.

[0160] Step 503, Flight speed time vector.

[0161] Based on the optical flow vector, the velocity of target A in each frame of the image can be calculated. After stitching the images in chronological order, the velocity change vector of target A can be obtained.

[0162] Step 504: Based on the motion trajectory, visual feature vector, physical scale change vector, and velocity change vector of all targets, combined with the sound feature vector, multimodal feature joint modeling is used to obtain bird species identification results. This effectively improves the accuracy and robustness of bird classification under complex environmental conditions and provides a reliable basis for the formulation of subsequent differentiated bird control strategies.

[0163] Step 6: Develop a bird deterrence strategy based on the bird species identification results, and carry out bird deterrence work based on the deterrence strategy.

[0164] For example, if the classification result is large birds of prey (such as eagles and hawks), activate a strong sonic bird repeller (frequency 2000-5000Hz) + a laser bird repeller (green laser, power 50mW); if the classification result is small birds of prey (such as sparrows and swallows), activate an ultrasonic bird repeller (frequency 20-40kHz) + flashing lights (frequency 5-10Hz).

[0165] Example 2

[0166] This embodiment provides a bird strike prevention intelligent management system, including:

[0167] The data acquisition module is configured to acquire sound signals from the target area and several consecutive frames of images captured by multiple cameras.

[0168] The trajectory generation module is configured as follows: for each frame of image, the foreground region is extracted using a Gaussian mixture model, feature points are extracted within the foreground region, and the velocity variance and orientation variance of the feature points are calculated using the sparse optical flow method to eliminate false foreground; for the foreground region of each frame of image, several target detection boxes are segmented; for each target detection box, target localization is performed based on its position in images captured by different cameras at the same time to obtain its three-dimensional spatial position; for each target detection box, a label number is assigned using a greedy matching method; for target detection boxes with the same label number, the three-dimensional spatial positions at several consecutive time steps are stitched together to generate a motion trajectory.

[0169] The sound processing module is configured to extract features from a sound signal using Mel-frequency cepstral coefficients to obtain a sound feature vector.

[0170] The feature extraction module is configured to extract visual feature vectors, physical scale change vectors, and velocity change vectors for target detection boxes with the same label number.

[0171] The classification module is configured to: for all labeled target detection boxes, based on motion trajectory, visual feature vector, physical scale change vector, and velocity change vector, combined with sound feature vector, obtain bird species identification results through multimodal feature joint modeling, and formulate a driving strategy based on the bird species identification results.

[0172] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0173] Example 3

[0174] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the bird strike prevention intelligent management method described in Embodiment 1 above.

[0175] Example 4

[0176] This embodiment provides a computer device, such as... Figure 5 As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and send data. When the processor 1001 executes the program, it implements the steps of the bird strike prevention intelligent management method described in Embodiment 1 above.

[0177] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A smart management method for bird strike prevention, characterized in that, include: Acquire sound signals from the target area and several consecutive frames of images captured by multiple cameras; For each frame of image, the foreground region is extracted using a Gaussian mixture model, feature points are extracted within the foreground region, and the velocity variance and orientation variance of the feature points are calculated using the sparse optical flow method to eliminate false foreground. For the foreground region of each frame of the image, several target detection boxes are obtained by segmentation; For each target detection box, target localization is performed based on its position in images captured by different cameras at the same time to obtain its three-dimensional spatial position; for each target detection box, a label number is assigned using a greedy matching method; for target detection boxes with the same label number, the three-dimensional spatial positions at several consecutive time moments are stitched together to generate a motion trajectory. For sound signals, the Mel frequency cepstral coefficients are used for feature extraction to obtain sound feature vectors; For target detection boxes with the same label, extract the visual feature vector, physical scale change vector, and velocity change vector; For all the target detection boxes with the marked numbers, based on the motion trajectory, visual feature vector, physical scale change vector and velocity change vector, combined with the sound feature vector, the bird species identification result is obtained through multimodal feature joint modeling, and the bird species identification result is formulated to drive away the bird. Specifically, when extracting the foreground region, a flicker counter C(x,y,t) is set for each pixel (x,y). If the pixel state changes between frame t and frame (t-1), then C(x,y,t) = C(x,y,t-1) + 1. If the pixel state does not change between frame t and frame (t-1), then C(x,y,t) = max(0,C(x,y,t-1) - 0.5). When C(x,y,t) ≥ the counter threshold, the update of the Gaussian distribution parameters of pixel (x,y) is paused.

2. The intelligent management method for bird strike prevention as described in claim 1, characterized in that, When executing the greedy matching method, for the t-th frame of the image, delayed decision tracking is triggered if any of the following conditions are met: If the Euclidean distance between two or more target detection boxes is less than a preset distance threshold; The same detection box matches multiple historical marker numbers, or multiple detection boxes compete for the same historical marker number.

3. The intelligent management method for bird strike prevention as described in claim 1, characterized in that, The three-dimensional spatial position of a target detection box at time t is (AB, AC): The actual straight-line distance between the target detection box and camera B ; The actual straight-line distance between the target detection box and camera C ; Wherein, the horizontal distance between the target detection box and camera B ; Horizontal distance between the target detection box and camera C The horizontal offset angles of the target detection bounding boxes relative to cameras B and C are respectively... and The vertical offset angles of the target detection bounding boxes relative to cameras B and C are respectively... and L represents the distance between camera BC and the camera.

4. The intelligent management method for bird strike prevention as described in claim 3, characterized in that, The horizontal offset angles of the target detection box relative to camera B and camera C are as follows: ； ； The images captured by cameras B and C both have a resolution of W×H; the horizontal field of view of cameras B and C are both γ; the pixel coordinates of the center position of the target detection box in the image captured by camera B are (i B ,j B The pixel coordinates in the image captured by camera C are (i C ,j C ).

5. The intelligent management method for bird strike prevention as described in claim 3, characterized in that, The vertical offset angles of the target detection box relative to camera B and camera C are as follows: ； ； The images captured by cameras B and C both have a resolution of W×H; the vertical field of view of cameras B and C are both β; the pixel coordinates of the center position of the target detection box in the image captured by camera B are (i B ,j B The pixel coordinates in the image captured by camera C are (i C ,j C ), Set the camera to an upward angle.

6. The intelligent management method for bird strike prevention as described in claim 1, characterized in that, For audio signals, before using Mel frequency cepstral coefficients for feature extraction, pre-emphasis, framing, Hanning windowing, and endpoint detection operations are performed sequentially.

7. A bird strike prevention intelligent management system, characterized in that, include: The data acquisition module is configured to acquire sound signals from the target area and several consecutive frames of images captured by multiple cameras. The trajectory generation module is configured to: for each frame of image, extract the foreground region through a Gaussian mixture model, extract feature points in the foreground region, calculate the velocity variance and orientation variance of the feature points using the sparse optical flow method, and remove false foreground; For the foreground region of each frame of the image, several target detection boxes are obtained by segmentation; For each target detection box, target localization is performed based on its position in images captured by different cameras at the same time to obtain its three-dimensional spatial position; for each target detection box, a label number is assigned using a greedy matching method; for target detection boxes with the same label number, the three-dimensional spatial positions at several consecutive time moments are stitched together to generate a motion trajectory. The sound processing module is configured to extract features from a sound signal using Mel-frequency cepstral coefficients to obtain a sound feature vector. The feature extraction module is configured to extract visual feature vectors, physical scale change vectors, and velocity change vectors for target detection boxes with the same label number. The classification module is configured to: for all labeled target detection boxes, based on motion trajectory, visual feature vector, physical scale change vector and velocity change vector, combined with sound feature vector, obtain bird species identification results through multimodal feature joint modeling, and formulate a driving strategy based on the bird species identification results; Specifically, when extracting the foreground region, a flicker counter C(x,y,t) is set for each pixel (x,y). If the pixel state changes between frame t and frame (t-1), then C(x,y,t) = C(x,y,t-1) + 1. If the pixel state does not change between frame t and frame (t-1), then C(x,y,t) = max(0,C(x,y,t-1) - 0.5). When C(x,y,t) ≥ the counter threshold, the update of the Gaussian distribution parameters of pixel (x,y) is paused.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the bird strike prevention intelligent management method as described in any one of claims 1-6.

9. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the bird strike prevention intelligent management method as described in any one of claims 1-6.

Images