Intelligent bird repelling device and method based on multi-modal recognition

Abstract

The invention discloses an intelligent bird repelling device and method based on multi-modal recognition, and relates to the technical field of power grid operation, a microphone array is arranged in a target monitoring area, and sound signals of birds are monitored in real time; setting a camera in the target monitoring area, and acquiring an image of the bird in the target monitoring area; according to the images of the birds in the target monitoring area and the sound signals of the birds, the initial emission state of a laser and the initial sound production state of an audio generator are determined, and the birds in the target monitoring area are repelled; acquiring behavior state information that the birds are driven, and adjusting the emission state of the laser and the sound production state of the audio generator according to the behavior state information; in this way, birds can be repelled in various modes of sound and images, behavior characteristics of different birds and actual requirements of a power grid can be flexibly met, the adaptability in different environments is good, the effective bird repelling effect is ensured, and safe operation of the power grid is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid operation, and in particular to an intelligent bird-repelling device and method based on multimodal recognition. Background Art

[0002] As the scale of the power grid continues to expand, the safe operation of power facilities faces increasingly complex challenges. Bird activity, particularly large-scale bird migration, has become a key factor affecting the stability and security of the power grid. Birds roosting, nesting, or flying on power facilities can cause short circuits, damage to equipment, or power outages. In severe cases, they can even trigger large-scale power outages. Therefore, power companies and relevant management departments urgently need to take effective measures to prevent birds from posing a threat to the safe operation of the power grid.

[0003] Traditional bird-repelling methods include installing bird spikes, reflectors, or electronic sound devices. Bird spikes deter birds from roosting and nesting, reflectors use sunlight to create bright light to scare birds away, and electronic sound devices simulate the sounds of predators to deter birds. However, these bird-repelling devices generally employ a single method, making them inflexible in addressing the behavioral characteristics of different birds and the actual needs of power grids. These traditional methods are not only poorly adaptable to different environments, resulting in ineffective bird repellent and failing to ensure the safe operation of power grids. Summary of the Invention

[0004] The purpose of the present invention is to solve the above-mentioned problems and provide an intelligent bird-repelling device and method based on multimodal recognition.

[0005] In a first aspect of the present invention, an intelligent bird-repelling method based on multimodal recognition is first proposed, the method comprising:

[0006] A microphone array is set up in the target monitoring area to monitor the sound signals of birds in real time. The beam direction of the microphone array is determined based on the initially collected sound signals to collect the sound signals of birds;

[0007] A camera is set up in the target monitoring area. The camera adjusts the shooting angle according to the beam direction of the microphone array to obtain images of birds in the target monitoring area.

[0008] Determine the initial emission state of the laser and the initial sounding state of the audio generator based on the image of birds in the target monitoring area and the sound signals of the birds, and drive away the birds in the target monitoring area;

[0009] Behavioral state information of the driven birds is obtained, and the emission state of the laser and the sounding state of the audio generator are adjusted according to the behavioral state information.

[0010] Optionally, the step of determining the beam direction of the microphone array based on the initially collected sound signal is:

[0011] For the preliminary collection of sound signals in each direction of the microphone array, the beam direction of the microphone array is preliminarily adjusted through the beamforming algorithm according to the signal strength and signal ratio;

[0012] Acquire bird sound signals in the target monitoring area collected by the adjusted microphone array, analyze the bird sound signals, and determine whether the beam direction of the microphone array needs to be adjusted again;

[0013] If necessary, the beam direction of the microphone array is adjusted again through the beamforming algorithm.

[0014] Optionally, the steps of obtaining bird sound signals in the target monitoring area collected by the adjusted microphone array, analyzing the bird sound signals, and determining whether the beam direction of the microphone array needs to be adjusted again are:

[0015] The bird sound signals are analyzed, including calculating the signal uniformity and beam focusing quality of the bird sound signals in the target monitoring area collected by the adjusted microphone array;

[0016] Obtaining a beam direction adjustment trigger value based on signal uniformity and beam focusing quality, and comparing the beam direction adjustment trigger value with a preset adjustment trigger value threshold;

[0017] If the adjustment trigger value is not less than the preset adjustment trigger value threshold, there is no need to adjust the beam direction of the microphone array again, and subsequent analysis is directly performed based on the bird sound signal obtained after the preliminary adjustment of the beam direction of the microphone array;

[0018] If the adjustment trigger value is less than the preset adjustment trigger value threshold, the beam direction of the microphone array needs to be adjusted again, and the beam direction of the microphone array is adjusted again through the beamforming algorithm until the adjustment trigger value is not less than the preset adjustment trigger value threshold.

[0019] Optionally, the steps for calculating the signal uniformity are:

[0020] Acquire the sound signal strength in each direction from the adjusted microphone array, perform normalization processing, and obtain a signal strength set;

[0021] Calculate the standard deviation and mean of the signal strength set, and calculate the skewness of the signal. The calculation formula is: Where γ is the skewness of the signal, N is the total number of signal strengths in the signal strength set, and S i represents the i-th signal strength in the signal strength set, σ and μ are the standard deviation and mean respectively;

[0022] The signal uniformity is calculated based on the standard deviation, mean and skewness of the signal. The calculation formula is: Where Dc is the signal uniformity.

[0023] Optionally, the steps for calculating the beam focusing quality are:

[0024] Acquire sound signals in each direction from the adjusted microphone array, perform Fourier transform on the sound signals in each direction, and obtain a spectrum of the sound signals in each direction; calculate the amplitude of each frequency on the spectrum, and add the amplitudes of each frequency to obtain the response amplitude in the corresponding direction;

[0025] Preliminarily adjust the beam direction of the microphone array and record it as the target direction. Calculate the response amplitude in the target direction to obtain the target response amplitude. Divide the target response amplitude by the response amplitudes in other directions to obtain the response amplitude ratio of the corresponding direction.

[0026] The response amplitude ratios greater than 1 among the response amplitude ratios are recorded as normal values, and the normal values are divided by the total number of response amplitude ratios to obtain the beam focusing quality.

[0027] Optionally, the steps of determining the emission state of the laser and the sound state of the audio generator based on the image of the birds in the target monitoring area and the sound signals of the birds, and driving away the birds in the target monitoring area are:

[0028] Extract the density, number and species of birds from the images of birds in the target monitoring area, and preliminarily determine the emission frequency and emission intensity of the laser according to the density and number of birds;

[0029] Retrieving multiple natural enemy sound libraries of each type of bird from the audio generator according to the species type, and obtaining the total number of successful bird repelling and the total number of participating bird repelling in the historical bird repelling records of each natural enemy sound library, and dividing the total number of successful bird repelling by the total number of participating bird repelling to obtain the bird repelling success ratio;

[0030] Obtain the average reaction time of birds leaving each natural enemy sound library in the historical bird repelling records and the continuous playback time of each natural enemy sound library during the bird repelling process, normalize the average reaction time and continuous playback time, and calculate the priority value of each natural enemy sound library based on the normalized average reaction time, continuous playback time and bird repelling success rate, priority value = bird repelling success rate - average reaction time - continuous playback time;

[0031] The natural enemy sound library with the largest priority value is used as the target natural enemy sound library for sound production, and the volume of the target natural enemy sound library is determined according to the density and number of birds.

[0032] Optionally, the steps of obtaining behavioral state information of the driven birds and adjusting the emission state of the laser and the sounding state of the audio generator according to the behavioral state information are:

[0033] After the birds in the target monitoring area are driven away by the initial emission state of the laser and the initial sounding state of the audio generator, the three-dimensional coordinates of each bird at each moment within the preset time are obtained to obtain the escape trajectory;

[0034] The distance between two adjacent sampling points is calculated based on the escape trajectory, and the distances are added together to obtain the escape distance. The escape distance is divided by the total sampling time to obtain the escape response time of the corresponding bird.

[0035] Get the unit vectors of two adjacent sampling points of the escape trajectory and calculate the direction change between the two adjacent unit vectors. The calculation formula is: Where θ a Represents the change in direction between the ath unit vector and the a+1th unit vector;

[0036] Calculate the sum of all direction changes to get the escape trajectory complexity value;

[0037] Get the total number of times each bird returns to the target monitoring area within the preset time, and add up the total number of times all birds return to the target monitoring area to get the escape return value;

[0038] Obtain the total number of birds after a preset time, and divide the total number of birds by the total number of birds before driving away to obtain the effective driving away ratio;

[0039] The emission state of the laser and the sound state of the audio generator are adjusted according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio.

[0040] Optionally, the step of adjusting the emission state of the laser and the sound state of the audio generator according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio is:

[0041] The steps for adjusting the emission state of the laser and the sound state of the audio generator according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio are as follows:

[0042] The escape response time, escape trajectory complexity, escape return value, and effective repelling ratio were normalized, dimensionless, and mapped to the 0-1 interval. The normalized escape trajectory complexity and effective repelling ratio were added together, and the escape return value and the escape response time were subtracted from the sum to obtain the repelling effect value of each bird.

[0043] Calculate the average of the repelling effect values of all birds to obtain the repelling effect index, and compare the repelling effect index with the preset repelling effect index threshold. If the repelling effect value is not less than the preset repelling effect index threshold, there is no need to adjust the emission state of the laser and the sound state of the audio generator. The birds in the target monitoring area are still repelled according to the initial emission state of the laser and the initial sound state of the audio generator until all birds are out of the target monitoring area.

[0044] If the repelling effect index is less than the preset repelling effect index threshold, it is necessary to increase the emission frequency and emission intensity of the laser; at the same time, increase the volume of the target natural enemy sound library again until all birds are out of the target monitoring area.

[0045] In a second aspect of the present invention, an intelligent bird-repelling device based on multimodal recognition is proposed, comprising:

[0046] Sound collection module: A microphone array is set up in the target monitoring area to monitor the sound signals of birds in real time. The beam direction of the microphone array is determined based on the initially collected sound signals to collect the sound signals of birds.

[0047] Image acquisition module: A camera is set up in the target monitoring area. The camera adjusts the shooting angle according to the beam direction of the microphone array to obtain images of birds in the target monitoring area;

[0048] Preliminary driving away module: determines the initial emission state of the laser and the initial sound state of the audio generator based on the image of birds in the target monitoring area and the sound signals of birds, and drives away the birds in the target monitoring area;

[0049] Second-drive module: obtains the behavioral state information of the birds being driven away, and adjusts the emission state of the laser and the sound state of the audio generator according to the behavioral state information.

[0050] Beneficial effects of the present invention:

[0051] The present invention proposes an intelligent bird-repelling device and method based on multimodal recognition. The device and method comprises the following steps: setting a microphone array in a target monitoring area to monitor the sound signals of birds in real time, determining the beam direction of the microphone array based on the initially collected sound signals, and collecting the sound signals of birds; setting a camera in the target monitoring area, adjusting the shooting angle of the camera according to the beam direction of the microphone array, and obtaining images of birds in the target monitoring area; determining the initial emission state of a laser and the initial sounding state of an audio generator based on the images of birds in the target monitoring area and the sound signals of birds, and repelling birds in the target monitoring area; obtaining behavioral state information of the repelled birds, and adjusting the emission state of the laser and the sounding state of the audio generator based on the behavioral state information; in this way, the device can provide multiple modes of sound and image to repel birds, can flexibly respond to the behavioral characteristics of different birds and the actual needs of the power grid, has good adaptability in different environments, ensures the effectiveness of the bird-repelling effect, and guarantees the safe operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The present invention will be further described below with reference to the accompanying drawings.

[0053] Figure 1 This is a flow chart of an intelligent bird-repelling method based on multimodal recognition;

[0054] Figure 2 This is a framework diagram of an intelligent bird-repellent device based on multimodal recognition. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0057] The embodiment of the present invention provides an intelligent bird-repelling method based on multimodal recognition. Figure 1 , Figure 1 A flowchart of an intelligent bird-repelling method based on multimodal recognition provided by an embodiment of the present invention. The method includes the following steps:

[0058] A microphone array is set up in the target monitoring area to monitor the sound signals of birds in real time. The beam direction of the microphone array is determined based on the initially collected sound signals to collect the sound signals of birds;

[0059] A camera is set up in the target monitoring area. The camera adjusts the shooting angle according to the beam direction of the microphone array to obtain images of birds in the target monitoring area.

[0060] Determine the initial emission state of the laser and the initial sounding state of the audio generator based on the image of birds in the target monitoring area and the sound signals of the birds, and drive away the birds in the target monitoring area;

[0061] Behavioral state information of the driven birds is obtained, and the emission state of the laser and the sounding state of the audio generator are adjusted according to the behavioral state information.

[0062] An intelligent bird-repellent method based on multimodal recognition provided by an embodiment of the present invention can provide multiple modes of sound and image to expel birds through the above-mentioned method. It can flexibly respond to the behavioral characteristics of different birds and the actual needs of the power grid, and has good adaptability in different environments, ensuring the effectiveness of the bird-repellent effect and guaranteeing the safe operation of the power grid.

[0063] In one embodiment, a microphone array is set up in the target monitoring area to monitor the sound signals of birds in real time, and the beam direction of the microphone array is adjusted based on the initially collected sound signals to collect the sound signals of birds;

[0064] Specifically, the steps of adjusting the optimal beam direction of the microphone array based on the initially collected sound signal are as follows:

[0065] Microphone Array Deployment: Within the power grid area, microphones are placed in multiple directions according to a pre-set monitoring range, creating a microphone array. Each microphone component achieves directional signal collection through its spatial configuration. The microphone array can monitor and capture ambient sound signals in real time, including bird calls and background noise.

[0066] Signal acquisition: The array receives sound signals from different directions and converts these signals into digital audio data for subsequent processing.

[0067] Signal Strength Analysis: Initially collected sound signals are analyzed based on their intensity. The intensity of a sound signal is generally related to the distance from its source, so the relative direction of the bird's chirping can be estimated based on signal strength.

[0068] Signal-to-noise ratio (SNR) calculation: The ratio of signal strength to background noise in each direction (i.e., the SNR) is calculated. A higher SNR indicates a greater difference between the bird's call signal and the noise, indicating better quality. This helps determine whether the bird's call is obscured by background noise and provides feedback for beam adjustment.

[0069] Noise source interference detection: Analyze other noise sources in the environment (such as wind noise, noise from power equipment, etc.); through the spatial configuration of the microphone array, the direction of the noise source can be inferred, so that when subsequently adjusting the beam direction, these interference sources can be avoided, thereby improving the recognition accuracy of bird sounds.

[0070] The beamforming algorithm adjusts the direction of the microphone array:

[0071] Beamforming: A beamforming algorithm adjusts the directionality of a microphone array to enhance signals from a specific direction (i.e., the direction of bird calls) while reducing noise from other directions. For example, assume a four-microphone array is deployed within a power grid area, oriented due north, due south, due east, and due west. The microphone array collects real-time sound signals from different directions, including bird calls from due east and low-frequency noise from power equipment due west. Signal strength analysis reveals that the birdsong from due east is stronger, while the power equipment noise from due west, while weaker, is persistent and interferes with signal acquisition. The system then calculates the signal-to-noise ratio (SNR) for each direction. The higher SNR for birdsong from due east indicates better bird sound quality in that direction, while the lower SNR for power equipment noise from due west indicates greater noise interference. Combined with noise source interference detection, the spatial configuration of the microphone array identifies the primary noise source from due west. Based on these analyses, the system uses a beamforming algorithm to adjust the beam direction of the microphone array so that it focuses on the east direction, enhancing the signal of bird calls while suppressing noise interference from the west direction; and at the same time, the east direction is used as the beam direction, making the collected bird sound signals more accurate and clear.

[0072] At the same time, in order to determine whether the beam direction of the microphone array needs to be adjusted again to achieve optimal bird sound signal collection, it is also necessary to judge whether further adjustment is needed based on the bird sound signals obtained by the microphone array after the initial adjustment. The specific steps are as follows:

[0073] For the preliminary collection of sound signals in each direction of the microphone array, the beam direction of the microphone array is preliminarily adjusted through the beamforming algorithm according to the signal strength and signal ratio;

[0074] Acquire bird sound signals in the target monitoring area collected by the adjusted microphone array, analyze the bird sound signals, and determine whether the beam direction of the microphone array needs to be adjusted again;

[0075] If necessary, the beam direction of the microphone array is adjusted again through the beamforming algorithm.

[0076] In one embodiment, the steps of obtaining bird sound signals in the target monitoring area collected by the adjusted microphone array, analyzing the bird sound signals, and determining whether the beam direction of the microphone array needs to be adjusted again are as follows:

[0077] The bird sound signals are analyzed, including calculating the signal uniformity and beam focusing quality of the bird sound signals in the target monitoring area collected by the adjusted microphone array;

[0078] Obtaining a beam direction adjustment trigger value based on signal uniformity and beam focusing quality, and comparing the beam direction adjustment trigger value with a preset adjustment trigger value threshold;

[0079] If the adjustment trigger value is not less than the preset adjustment trigger value threshold, there is no need to adjust the beam direction of the microphone array again, and subsequent analysis is directly performed based on the bird sound signal obtained after the preliminary adjustment of the beam direction of the microphone array;

[0080] If the adjustment trigger value is less than the preset adjustment trigger value threshold, the beam direction of the microphone array needs to be adjusted again, and the beam direction of the microphone array is adjusted again through the beamforming algorithm until the adjustment trigger value is not less than the preset adjustment trigger value threshold.

[0081] It should be noted that the preset adjustment trigger value threshold is set by professionals based on actual conditions and will not be limited or elaborated on in detail.

[0082] It should be noted that if the adjusted trigger value is no less than the preset threshold, the initially adjusted microphone array beam direction is already capable of effectively capturing bird calls, and the signal quality is good. At this point, the signal uniformity and beam focusing quality meet the expected requirements. Therefore, no additional adjustments are required, and subsequent analysis can be performed directly based on the bird sound signals collected in the current beam direction. At this point, the collected sound signal typically has a high signal-to-noise ratio, meaning that the bird calls stand out against the background noise, providing reliable data support for subsequent bird repellent measures (such as activating a laser or audio generator). Assume that within a power grid area, the microphone array beam has been adjusted to a direction that focuses on capturing bird calls in the east direction, and the adjusted trigger value is no less than the preset threshold, indicating excellent signal quality. In this case, further beam adjustment is determined to be unnecessary, and subsequent bird repellent actions (such as laser irradiation or playing bird repellent audio) are initiated based on the image of the target monitoring area to ensure effective bird repellent.

[0083] If the adjustment trigger value is less than the preset adjustment trigger value threshold, it means that the initially adjusted beam direction has not effectively improved the signal quality, or the signal quality improvement is not very good. There may be a lot of background noise interference or the bird sounds cannot be captured strong enough. At this time, the system will use the beamforming algorithm to optimize the beam direction of the microphone array again. The beamforming algorithm simulates different beam direction adjustment schemes based on the current collected signal quality feedback, and gradually selects the optimal direction to maximize the quality of bird sound collection. Through this iterative optimization method, the system can continuously improve the clarity of bird sound signals, thereby providing more precise control signals for bird repellent measures.

[0084] In one implementation method, through this precise sound signal collection and beam direction optimization, the chirping of birds can be effectively distinguished from background noise, ensuring that the bird repellent system can accurately locate the specific position of the birds, and at the same time obtain the bird's chirping more clearly, and more accurately analyze which specific bird it is, thereby reducing the efficiency of subsequent bird repelling, reducing the probability of the audio generator emitting the wrong calls of bird natural enemies during subsequent bird repelling, reducing the cost of bird repelling, and improving the efficiency and reliability of the bird repelling effect.

[0085] In one embodiment, the steps for calculating signal uniformity are:

[0086] Acquire the sound signal strength in each direction from the adjusted microphone array, perform normalization processing, and obtain a signal strength set;

[0087] Calculate the standard deviation and mean of the signal strength set, and calculate the skewness of the signal. The calculation formula is: Where γ is the skewness of the signal, N is the total number of signal strengths in the signal strength set, and S i represents the i-th signal strength in the signal strength set, σ and μ are the standard deviation and mean respectively;

[0088] The signal uniformity is calculated based on the standard deviation, mean and skewness of the signal. The calculation formula is: Where Dc is the signal uniformity.

[0089] It should be noted that signal uniformity is a metric that measures the balance of the sound signals collected by the adjusted microphone array across different directions. It is calculated based on the standard deviation, mean, and skewness of the signal strength to quantify the uniformity of the distribution of sound signal strength in different directions. A high signal uniformity means that the sound signal strength captured by the microphone array is relatively balanced across all directions, indicating that the array's beam direction is accurately focused on the target monitoring area (i.e., the direction of the bird's song). Conversely, a low signal uniformity may indicate weak signals in certain directions or the presence of strong interference signals, resulting in the microphone array's beam direction not being fully optimized. A higher signal uniformity generally indicates that the beamforming algorithm has better focused on the target signal, and the difference in signal strength received from all directions is small. This balanced signal distribution is important for subsequent analysis, as it ensures that the quality of the bird's song signal is not significantly affected by directional deviation or noise interference. A high signal uniformity indicates that the microphone array has successfully achieved optimized directionality. In this case, the beam direction adjustment trigger value will be large, and no further beam direction adjustment is required. Therefore, if the signal uniformity is high, the beam direction adjustment trigger value will also be high, indicating that the array beam has been accurately aligned with the bird calls in the target monitoring area, and the influence of interference and noise has been suppressed. The system can directly perform subsequent analysis based on the sound signal after preliminary adjustment without further adjustment of the beam direction. Conversely, if the signal uniformity is low, the trigger value will be small, indicating that the current beam direction may not be fully focused on the source of the bird calls, and the system needs to further adjust the beam direction to improve the quality and accuracy of signal acquisition.

[0090] In one implementation, signal uniformity is calculated primarily to assess whether the distribution of sound signals collected by the microphone array in different directions is balanced after adjustment. This calculation method combines the standard deviation, mean, and skewness of the signal strength to comprehensively reflect the concentration and deviation of the signal. The standard deviation measures the fluctuation range of the signal strength, the mean indicates the overall level of the signal, and the skewness reveals the asymmetry of the signal distribution. Through the comprehensive evaluation of these statistics, signal uniformity can provide accurate feedback on the quality of the microphone array direction adjustment. If the signal uniformity is high, it indicates that the beam direction of the microphone array has been effectively focused on the target monitoring area, reducing interference from irrelevant directions, and the signal collection is more balanced. Conversely, a low signal uniformity indicates that the microphone array may not be effectively aimed at the target signal source, and further adjustment of the beam direction may be required.

[0091] In one embodiment, the steps for calculating the beam focusing quality are:

[0092] Acquire sound signals in each direction from the adjusted microphone array, perform Fourier transform on the sound signals in each direction, and obtain a spectrum of the sound signals in each direction; calculate the amplitude of each frequency on the spectrum, and add the amplitudes of each frequency to obtain the response amplitude in the corresponding direction;

[0093] Preliminarily adjust the beam direction of the microphone array and record it as the target direction. Calculate the response amplitude in the target direction to obtain the target response amplitude. Divide the target response amplitude by the response amplitudes in other directions to obtain the response amplitude ratio of the corresponding direction.

[0094] The response amplitude ratios greater than 1 among the response amplitude ratios are recorded as normal values, and the normal values are divided by the total number of response amplitude ratios to obtain the beam focusing quality.

[0095] It should be noted that beam focusing quality measures the ability and accuracy of a microphone array to focus on a target sound signal after adjustment. The calculation process converts the sound signal into a spectrogram through Fourier transform, further analyzes the response amplitude in each direction, and compares the response amplitude ratios in the target direction with those in other directions to assess whether the microphone array has successfully focused on the target signal. In this process, beam focusing quality reflects the relative strength and clarity of the target sound signal. When the response amplitude in the target direction is significantly greater than that in other directions, it indicates that the microphone array has successfully aligned its beam direction with the target sound source. This is because the microphone array uses the beamforming algorithm to enhance the sound signal from the target direction while suppressing noise or interference signals from other directions. Because signal strength is closely related to the directivity of the microphone array, the beam focusing process increases the signal amplitude in the target direction while reducing the signal amplitude in other directions. Therefore, when the response amplitude in the target direction is significantly greater than that in other directions, it means that the microphone array has successfully focused on and effectively enhanced the acquisition quality of the target sound source, thereby improving signal clarity and accuracy. Therefore, the greater the beam focusing quality, the better the microphone array focusing effect, the higher the signal quality, and further adjustments are no longer necessary. The microphone array can be directly relied upon for subsequent analysis after the initial adjustment. If the beam focusing quality is low, it may mean that the array beam direction has not yet effectively focused on the target signal source, and the beam direction may need to be adjusted again to ensure the best signal acquisition effect.

[0096] In one implementation, the core purpose of the beam focusing quality metric is to evaluate whether the microphone array successfully focuses on the target sound source. When the microphone array is adjusted, its beam direction should prioritize enhancing the signal strength of the target sound source while suppressing noise and interference signals from other directions. Converting the signal into a spectrogram through Fourier transform allows for precise analysis of signal strength at different frequencies, further assessing the response amplitude of sound signals in each direction. The ratio of the target response amplitude to the response amplitudes in other directions effectively reflects whether the beam has been successfully aligned with the target sound source. A response amplitude ratio greater than 1 indicates that the target signal is significantly stronger than the interference signal, thus determining the quality of the beam focusing. A larger response amplitude ratio indicates better amplification and clarity of the target sound signal, and a higher beam focusing quality metric value reflects better directional acquisition performance of the microphone array. Therefore, this calculation method can intuitively evaluate the focusing effect of the microphone array, ensuring that the target sound signal can be captured clearly and accurately.

[0097] In one embodiment, the steps of obtaining a beam direction adjustment trigger value according to signal uniformity and beam focusing quality are as follows:

[0098] The signal uniformity and the beam focusing quality are weighted and summed to obtain the beam direction adjustment trigger value.

[0099] It should be noted that when calculating the beam adjustment trigger value, a weighted summation of signal uniformity and beam focusing quality is performed. This is primarily to comprehensively consider the microphone array's signal performance in different directions. Signal uniformity reflects whether signal acquisition is balanced across different directions, helping to determine whether signals in certain directions are overly concentrated or sparse, impacting overall acquisition quality. Beam focusing quality measures whether the microphone array successfully focuses the beam on the target sound source, effectively enhancing the quality of the target signal. A high signal uniformity indicates a relatively even signal distribution across all directions, minimizing the need for beam adjustment. A high beam focusing quality indicates that the microphone array is successfully aligned with the target sound source, and the target signal is prominent. Therefore, the weighted summation of these two factors yields a comprehensive beam adjustment trigger value, enabling a more rational decision on whether further adjustments to the microphone array's beam direction are necessary to ensure optimal acquisition of the target signal. This approach enables more precise beam adjustment, avoids unnecessary multiple adjustments, and ensures the accuracy and clarity of the target sound signal.

[0100] In one embodiment, a camera is set up in the target monitoring area, and the camera adjusts the shooting angle according to the beam direction of the microphone array to obtain images of birds in the target monitoring area;

[0101] It's important to note that within a target surveillance area, a dual monitoring approach combining a microphone array and a camera can significantly improve the accuracy of identifying and tracking targets (such as birds). First, the microphone array collects sound signals from various directions in real time. Using a beamforming algorithm, it determines the location of the target sound source and optimizes the array's beam direction to maximize signal strength and minimize noise interference. Once the microphone array determines the optimal beam direction, the system transmits this direction to the camera, instructing it to adjust its shooting angle. The camera then adjusts to align with the sound source, accurately capturing image information within the target surveillance area. This multimodal monitoring approach, combining sound and image data, not only helps the system locate the target but also verifies its behavior and characteristics through image data, enhancing the comprehensiveness and accuracy of monitoring. By continuously tracking the changes in the sound source's position, the system can dynamically adjust and optimize the shooting angle to ensure the clearest possible image of the target. This efficient adaptive adjustment mechanism provides the entire surveillance system with greater flexibility and accuracy in handling complex environments and rapidly changing targets, significantly enhancing the reliability and real-time performance of target monitoring.

[0102] In one embodiment, the bird sound signal obtained after adjusting the beam direction of the microphone array is subjected to noise reduction processing. Noise is removed through a digital filter (FIR / IIR) and a noise reduction algorithm (spectral subtraction or deep learning noise reduction model), and a pure audio signal (sampling rate ≥44.1kHz, 16-bit quantization) is output to provide a high-quality signal for subsequent sound classification and decision-making; and the pre-processed sound signal is classified and decided through a bird voiceprint classifier; if the classification confidence is ≥85% and the same signal is detected three times in a row, it is determined to be a valid target.

[0103] It should be noted that after adjusting the microphone array's beam direction, the collected bird sound signals often contain some background noise or interference. This noise may originate from other environmental sound sources, such as wind or electrical equipment. To improve the accuracy of subsequent processing, the sound signals must undergo noise reduction. Digital filters (such as FIR / IIR filters) used in noise reduction can effectively reduce high- and low-frequency noise components, thereby preserving the primary frequency characteristics of the bird sounds. Furthermore, spectral subtraction or deep learning noise reduction models (such as those based on convolutional neural networks) are used to perform more refined signal processing to remove excess noise and enhance the clarity of the bird sounds. Ultimately, the processed audio signals are output as high-quality audio data with a sampling rate of 44.1kHz or higher and a quantization bit rate of 16 bits. This ensures the fidelity of the sound signal and facilitates subsequent sound analysis and classification. Next, the system uses a bird voiceprint classifier to perform feature extraction and classification analysis on the processed sound signals. This classifier, based on deep learning technology and trained with extensive bird song data, can identify the sound characteristics of different bird species and provide classification results. If the confidence level of the classification result reaches 85% or more, and the same bird sound signal can be identified in three consecutive detections, the system determines it as a valid target. This method can be applied in real-world environments. For example, in a bird monitoring system near power facilities, the system uses a microphone array to adjust the beam direction, perform noise reduction, and classify. When the system detects the same bird call multiple times with high confidence, it can activate the corresponding early warning mechanism or repelling measures to ensure that the power facilities are not affected by birds and protect the bird habitat.

[0104] In one implementation, the microphone array can effectively capture sound signals from a specific direction through beamforming technology, which means that the chirping of birds can be collected first, while ignoring interfering noise from other directions, such as the running sound of wind turbines or background environmental noise. Secondly, through signal strength analysis and signal-to-noise ratio (SNR) calculation, the system can further optimize the signal quality, making the bird sounds clearer and more accurate during the collection process, reducing the risk of being overwhelmed by noise. By applying digital filters and noise reduction algorithms (such as spectral subtraction or deep learning noise reduction models), the system can remove background noise, retain the characteristic sounds of birds, and provide high-quality audio signals for subsequent analysis. This processing method ensures the high credibility of sound data, provides a reliable basis for the accurate identification and timely expulsion of birds, reduces the occurrence of false alarms and missed alarms, and improves the efficiency and response speed of the bird monitoring system.

[0105] In one embodiment, the steps of driving away birds from the target monitoring area include determining the initial emission state of the laser and the initial sounding state of the audio generator based on the image of the birds in the target monitoring area and the sound signals of the birds.

[0106] Extract the density, number and species of birds from the images of birds in the target monitoring area, and preliminarily determine the emission frequency and emission intensity of the laser according to the density and number of birds;

[0107] Retrieving multiple natural enemy sound libraries of each type of bird from the audio generator according to the species type, and obtaining the total number of successful bird repelling and the total number of participating bird repelling in the historical bird repelling records of each natural enemy sound library, and dividing the total number of successful bird repelling by the total number of participating bird repelling to obtain the bird repelling success ratio;

[0108] Obtain the average reaction time of birds leaving each natural enemy sound library in the historical bird repelling records and the continuous playback time of each natural enemy sound library during the bird repelling process, normalize the average reaction time and continuous playback time, and calculate the priority value of each natural enemy sound library based on the normalized average reaction time, continuous playback time and bird repelling success rate, priority value = bird repelling success rate - average reaction time - continuous playback time;

[0109] The natural enemy sound library with the largest priority value is used as the target natural enemy sound library for sound production, and the volume of the target natural enemy sound library is determined according to the density and number of birds.

[0110] It should be noted that image recognition algorithms (such as convolutional neural networks) are used to analyze images and extract bird density, number, and species. For example, the image analysis system may identify 50 pigeons in the target monitoring area and classify them as pigeons through an algorithm. Based on this information, the system adjusts the laser's emission frequency and intensity according to the bird density (50 pigeons) and number, such as using low frequency and low intensity for low density and high frequency and high intensity for high density. Based on the bird species, the system retrieves a library of sounds of specific natural enemies from the audio generator. For example, for pigeons, the system may select a library of sounds of birds of prey that prey on pigeons. The audio generator searches historical data to calculate the bird repellent success rate for this natural enemy sound library. Assuming 80 successful attempts out of 100, the bird repellent success rate for this sound library is 0.8. The system then obtains the average reaction time and duration of playback for this natural enemy sound library in historical bird repellent attempts. For example, in 80 successful attempts, the average bird reaction time was 3 seconds and the playback duration was 30 seconds. Then, these data are normalized, for example, the average reaction time is normalized to the range of 0-1 to get 0.3, and the playback time is normalized to 0.4, and the bird-repelling success rate is already 0.8. Finally, the priority value of the natural enemy sound library is calculated: 0.8 (bird-repelling success rate) - 0.3 (reaction time) - 0.4 (playing time), and the priority value is 0.1. The sound library with the largest priority value is selected as the target natural enemy sound library for playback. If the priority value of the natural enemy sound library is the largest and its volume is also adjusted based on the density and number of birds (for example, the volume is louder when the density is high), the audio generator plays the sound library to drive away the birds. In this way, the entire system dynamically adjusts the bird-repelling strategy based on real-time bird data and historical bird-repelling effects to ensure the efficiency and adaptability of the bird-repelling process.

[0111] In one embodiment, the steps of obtaining behavioral state information of birds being driven away and adjusting the emission state of the laser and the sound state of the audio generator according to the behavioral state information are as follows:

[0112] After the birds in the target monitoring area are driven away by the initial emission state of the laser and the initial sounding state of the audio generator, the three-dimensional coordinates of each bird at each moment within the preset time are obtained to obtain the escape trajectory;

[0113] The distance between two adjacent sampling points is calculated based on the escape trajectory, and the distances are added together to obtain the escape distance. The escape distance is divided by the total sampling time to obtain the escape response time of the corresponding bird.

[0114] Get the unit vectors of two adjacent sampling points of the escape trajectory and calculate the direction change between the two adjacent unit vectors. The calculation formula is: Where θ a Represents the change in direction between the ath unit vector and the a+1th unit vector;

[0115] Calculate the sum of all direction changes to get the escape trajectory complexity value;

[0116] Get the total number of times each bird returns to the target monitoring area within the preset time, and add up the total number of times all birds return to the target monitoring area to get the escape return value;

[0117] Obtain the total number of birds after a preset time, and divide the total number of birds by the total number of birds before driving away to obtain the effective driving away ratio;

[0118] The emission state of the laser and the sound state of the audio generator are adjusted according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio.

[0119] It should be noted that in the above calculation process, the data involved can be obtained in a variety of ways: First, the three-dimensional coordinate data of the birds is captured by a multi-camera system or a real-time positioning system (such as GPS, inertial sensors or optical tracking technology) in the target area. Secondly, the escape trajectory and behavior information of the birds can be recorded by a real-time tracking system to record the movement path of the birds, and the movement distance and direction change of each bird can be calculated in combination with high-frequency sampling data. In addition, the data of the escape return value and the effective repelling ratio are derived from the monitoring system's detection of whether the birds have re-entered the target area, which may rely on sensors or cameras set up around the target area.

[0120] In one embodiment, the steps of adjusting the emission state of the laser and the sound state of the audio generator according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio are as follows:

[0121] The escape response time, escape trajectory complexity, escape return value, and effective repelling ratio were normalized, dimensionless, and mapped to the 0-1 interval. The normalized escape trajectory complexity and effective repelling ratio were added together, and the escape return value and the escape response time were subtracted from the sum to obtain the repelling effect value of each bird.

[0122] Calculate the average of the repelling effect values of all birds to obtain the repelling effect index, and compare the repelling effect index with the preset repelling effect index threshold. If the repelling effect value is not less than the preset repelling effect index threshold, there is no need to adjust the emission state of the laser and the sound state of the audio generator. The birds in the target monitoring area are still repelled according to the initial emission state of the laser and the initial sound state of the audio generator until all birds are out of the target monitoring area.

[0123] If the repelling effect index is less than the preset repelling effect index threshold, it is necessary to increase the emission frequency and emission intensity of the laser; at the same time, increase the volume of the target natural enemy sound library again until all birds are out of the target monitoring area.

[0124] It should be noted that when the escape response time is shorter, the escape trajectory complexity value is greater, the escape return value is smaller, and the effective repelling ratio is higher, there is no need to adjust the laser emission state and the audio generator sound state. Birds in the target monitoring area will continue to be repelled according to the initial laser emission state and the initial audio generator sound state. This indicates that the birds react very quickly and diversely to the repelling measures, and most birds have successfully escaped the target area and have not returned, indicating a significant repelling effect. This means that the repelling operation has successfully repelled the birds from the target monitoring area, and their reaction speed and path complexity indicate that they have been effectively disturbed. Therefore, there is no need to increase the laser emission frequency or intensity or the audio generator volume, as the current repelling intensity is sufficient to deal with the birds in the target area. In this case, the repelling effect can be continued by maintaining the initial laser and audio emission states, avoiding unnecessary resource waste and excessive interference. For example, suppose a bird repelling system in a power grid area is in its initial operation. Cameras and sensors detect a flock of birds roosting on the power grid. Once the laser and audio generator are activated, the system begins tracking the escape trajectory of each bird. After a certain period of time, the system calculated that the escape response time of most birds was short and their escape trajectories were very tortuous, indicating that they reacted quickly and were highly resistant to the influence of the laser and the sounds of their natural enemies. In addition, only a very small number of birds returned during the expulsion process, indicating that they had completely left the target area and did not re-enter. Based on these calculations, the system determined that there was no need to adjust the expulsion intensity or frequency because the existing expulsion method was already achieving the desired effect. Continuing to maintain the current settings would ensure that all birds were successfully expelled, not only saving energy but also avoiding excessive interference with the environment or non-target species.

[0125] Based on the same inventive concept, the embodiment of the present invention also provides an intelligent bird-repelling device based on multimodal recognition. Figure 2 , Figure 2 A framework diagram of an intelligent bird-repelling device based on multimodal recognition provided by an embodiment of the present invention, the device comprising:

[0126] Sound collection module: A microphone array is set up in the target monitoring area to monitor the sound signals of birds in real time. The beam direction of the microphone array is determined based on the initially collected sound signals to collect the sound signals of birds.

[0127] Image acquisition module: A camera is set up in the target monitoring area. The camera adjusts the shooting angle according to the beam direction of the microphone array to obtain images of birds in the target monitoring area;

[0128] Preliminary driving away module: determines the initial emission state of the laser and the initial sound state of the audio generator based on the image of birds in the target monitoring area and the sound signals of birds, and drives away the birds in the target monitoring area;

[0129] Second-drive module: obtains the behavioral state information of the birds being driven away, and adjusts the emission state of the laser and the sound state of the audio generator according to the behavioral state information.

[0130] An intelligent bird-repellent device based on multimodal recognition provided by an embodiment of the present invention can provide multiple modes of sound and image to repel birds through the above-mentioned method. It can flexibly respond to the behavioral characteristics of different birds and the actual needs of the power grid, and has good adaptability in different environments, ensuring the effectiveness of the bird-repellent effect and guaranteeing the safe operation of the power grid. The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be used to artificially limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the application of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. An intelligent bird-repelling method based on multimodal recognition, characterized in that: The following steps are involved: A microphone array is set up in the target monitoring area to monitor the sound signals of birds in real time. The beam direction of the microphone array is determined based on the initially collected sound signals to collect the sound signals of birds; A camera is set up in the target monitoring area. The camera adjusts the shooting angle according to the beam direction of the microphone array to obtain images of birds in the target monitoring area. Determine the initial emission state of the laser and the initial sounding state of the audio generator based on the image of birds in the target monitoring area and the sound signals of the birds, and drive away the birds in the target monitoring area; Behavioral state information of the driven birds is obtained, and the emission state of the laser and the sounding state of the audio generator are adjusted according to the behavioral state information.

2. The intelligent bird-repelling method based on multimodal recognition according to claim 1, characterized in that: The steps for determining the beam direction of the microphone array based on the initially collected sound signal are as follows: For the preliminary collection of sound signals in each direction of the microphone array, the beam direction of the microphone array is preliminarily adjusted through the beamforming algorithm according to the signal strength and signal ratio; Acquire bird sound signals in the target monitoring area collected by the adjusted microphone array, analyze the bird sound signals, and determine whether the beam direction of the microphone array needs to be adjusted again; If necessary, the beam direction of the microphone array is adjusted again through the beamforming algorithm.

3. The intelligent bird-repelling method based on multimodal recognition according to claim 2, characterized in that: The steps of obtaining the bird sound signal in the target monitoring area collected by the adjusted microphone array, analyzing the bird sound signal, and determining whether the beam direction of the microphone array needs to be adjusted again are as follows: The bird sound signals are analyzed, including calculating the signal uniformity and beam focusing quality of the bird sound signals in the target monitoring area collected by the adjusted microphone array; Obtaining a beam direction adjustment trigger value based on signal uniformity and beam focusing quality, and comparing the beam direction adjustment trigger value with a preset adjustment trigger value threshold; If the adjustment trigger value is not less than the preset adjustment trigger value threshold, there is no need to adjust the beam direction of the microphone array again, and subsequent analysis is directly performed based on the bird sound signal obtained after the preliminary adjustment of the beam direction of the microphone array; If the adjustment trigger value is less than the preset adjustment trigger value threshold, the beam direction of the microphone array needs to be adjusted again, and the beam direction of the microphone array is adjusted again through the beamforming algorithm until the adjustment trigger value is not less than the preset adjustment trigger value threshold.

4. The intelligent bird-repelling method based on multimodal recognition according to claim 3, characterized in that: The calculation steps of the signal uniformity are: Acquire the sound signal strength in each direction from the adjusted microphone array and perform normalization processing to obtain a signal strength set; Calculate the standard deviation and mean of the signal strength set, and calculate the skewness of the signal. The calculation formula is: Where γ is the skewness of the signal, N is the total number of signal strengths in the signal strength set, and S i represents the i-th signal strength in the signal strength set, σ and μ are the standard deviation and mean respectively; The signal uniformity is calculated based on the standard deviation, mean and skewness of the signal. The calculation formula is: Where Dc is the signal uniformity.

5. The intelligent bird-repelling method based on multimodal recognition according to claim 3, characterized in that: The calculation steps of the beam focusing quality are: Acquire sound signals in each direction from the adjusted microphone array, perform Fourier transform on the sound signals in each direction, and obtain a spectrum of the sound signals in each direction; calculate the amplitude of each frequency on the spectrum, and add the amplitudes of each frequency to obtain the response amplitude in the corresponding direction; Preliminarily adjust the beam direction of the microphone array and record it as the target direction. Calculate the response amplitude in the target direction to obtain the target response amplitude. Divide the target response amplitude by the response amplitudes in other directions to obtain the response amplitude ratio of the corresponding direction. The response amplitude ratios greater than 1 among the response amplitude ratios are recorded as normal values, and the normal values are divided by the total number of response amplitude ratios to obtain the beam focusing quality.

6. The intelligent bird-repelling method based on multimodal recognition according to claim 1, characterized in that: The laser emission state and the audio generator sound state are determined based on the image of birds in the target monitoring area and the sound signals of the birds. The steps to drive away the birds in the target monitoring area are as follows: Extract the density, number and species of birds from the images of birds in the target monitoring area, and preliminarily determine the emission frequency and emission intensity of the laser according to the density and number of birds; Retrieving multiple natural enemy sound libraries of each type of bird from the audio generator according to the species type, and obtaining the total number of successful bird repelling and the total number of participating bird repelling in the historical bird repelling records of each natural enemy sound library, and dividing the total number of successful bird repelling by the total number of participating bird repelling to obtain the bird repelling success ratio; Obtain the average reaction time of birds leaving each natural enemy sound library in the historical bird repelling records and the continuous playback time of each natural enemy sound library during the bird repelling process, normalize the average reaction time and continuous playback time, and calculate the priority value of each natural enemy sound library based on the normalized average reaction time, continuous playback time and bird repelling success rate, priority value = bird repelling success rate - average reaction time - continuous playback time; The natural enemy sound library with the largest priority value is used as the target natural enemy sound library for sound production, and the volume of the target natural enemy sound library is determined according to the density and number of birds.

7. The intelligent bird-repelling method based on multimodal recognition according to claim 1, characterized in that: The steps of obtaining the behavioral state information of the driven birds and adjusting the emission state of the laser and the sound state of the audio generator according to the behavioral state information are as follows: After the birds in the target monitoring area are driven away by the initial emission state of the laser and the initial sounding state of the audio generator, the three-dimensional coordinates of each bird at each moment within the preset time are obtained to obtain the escape trajectory; The distance between two adjacent sampling points is calculated based on the escape trajectory, and the distances are added together to obtain the escape distance. The escape distance is divided by the total sampling time to obtain the escape response time of the corresponding bird. Get the unit vectors of two adjacent sampling points of the escape trajectory and calculate the direction change between the two adjacent unit vectors. The calculation formula is: Where θ a Represents the change in direction between the ath unit vector and the a+1th unit vector; Calculate the sum of all direction changes to get the escape trajectory complexity value; Get the total number of times each bird returns to the target monitoring area within the preset time, and add up the total number of times all birds return to the target monitoring area to get the escape return value; Obtain the total number of birds after a preset time, and divide the total number of birds by the total number of birds before driving away to obtain the effective driving away ratio; The emission state of the laser and the sound state of the audio generator are adjusted according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio.

8. The intelligent bird-repelling method based on multimodal recognition according to claim 7, characterized in that: The steps for adjusting the emission state of the laser and the sound state of the audio generator according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio are as follows: The steps for adjusting the emission state of the laser and the sound state of the audio generator according to the escape response time, the escape trajectory complexity value, the escape return value and the effective driving ratio are as follows: The escape response time, escape trajectory complexity, escape return value, and effective repelling ratio were normalized, dimensionless, and mapped to the 0-1 interval. The normalized escape trajectory complexity and effective repelling ratio were added together, and the escape return value and the escape response time were subtracted from the sum to obtain the repelling effect value of each bird. Calculate the average of the repelling effect values of all birds to obtain the repelling effect index, and compare the repelling effect index with the preset repelling effect index threshold. If the repelling effect value is not less than the preset repelling effect index threshold, there is no need to adjust the emission state of the laser and the sound state of the audio generator. The birds in the target monitoring area are still repelled according to the initial emission state of the laser and the initial sound state of the audio generator until all birds are out of the target monitoring area. If the repelling effect index is less than the preset repelling effect index threshold, it is necessary to increase the emission frequency and emission intensity of the laser; at the same time, increase the volume of the target natural enemy sound library again until all birds are out of the target monitoring area.

9. An intelligent bird-repelling device based on multimodal recognition, used to implement the intelligent bird-repelling method based on multimodal recognition according to any one of claims 1 to 8, characterized in that: The device comprises: Sound collection module: A microphone array is set up in the target monitoring area to monitor the sound signals of birds in real time. The beam direction of the microphone array is determined based on the initially collected sound signals to collect the sound signals of birds. Image acquisition module: A camera is set up in the target monitoring area. The camera adjusts the shooting angle according to the beam direction of the microphone array to obtain images of birds in the target monitoring area; Preliminary driving away module: determines the initial emission state of the laser and the initial sound state of the audio generator based on the image of birds in the target monitoring area and the sound signals of birds, and drives away the birds in the target monitoring area; Second-drive module: obtains the behavioral state information of the birds being driven away, and adjusts the emission state of the laser and the sound state of the audio generator according to the behavioral state information.

Images