A smart ultrasonic bird deterrent device and method based on deep learning bird sound recognition

Abstract

The application discloses an intelligent ultrasonic wave bird repelling device and method based on deep learning bird sound recognition, and belongs to the technical field of bird repelling of power transmission and transformation equipment. The intelligent ultrasonic wave bird repelling device constructs a bird sound sample collection system to collect bird sound samples of a substation; a bird sound recognition feature extraction model is constructed, including developing a convolutional encoder-decoder of bird sound spectrum features based on Conv-STFT and Conv-iSTFT, establishing spectrum subtraction of bird sound multi-window spectrum estimation, and extracting mel-frequency cepstral coefficient (MFCC) features to obtain acoustic features of bird sound from original bird sound samples; a bird sound recognition model is constructed based on a convolutional neural network (CNN), acoustic features extracted from a large amount of original voice data are taken as training data, the bird sound recognition model is trained, and a classification result of unknown bird sound is obtained; and an ultrasonic wave intelligent frequency conversion bird repelling module is constructed, the ultrasonic wave generator is started based on the bird classification result, and bird repelling frequency is learned in depth.

Description

Technical Field

[0001] This invention belongs to the field of power transmission line protection technology, and specifically relates to an intelligent ultrasonic bird deterrent device and method based on deep learning bird sound recognition. Background Technology

[0002] In recent years, with the improvement of the ecological environment, the harm posed by bird activity to the safety and stable operation of power transmission lines has become increasingly significant. In recent years, the power grid has vigorously promoted the construction of visualized power transmission lines, using visual and intelligent methods to improve the efficiency of transmission line inspections. Transmission channel visualization technology has played a crucial role in the timely detection and early warning of potential hazards, making outstanding contributions to the safety, reliability, and intelligent upgrading of transmission lines. Although transmission channel visualization technology and related artificial intelligence technologies have made some progress in recent years, they are still in their initial stages, and there is still a long way to go before they can meet the needs of power transmission operation and maintenance. To meet the requirements of safe, reliable, and intelligent transmission lines, it is necessary not only to study visual technologies but also to extensively research auditory, tactile, and olfactory technologies.

[0003] Bird activity is a significant factor contributing to power transmission line failures, leading to the implementation of increasingly stringent bird control measures, such as enhanced patrols and the use of bird deterrent devices. However, given the vast area affected by bird activity, the numerous power poles and towers, and the difficulty in identifying specific birds, these general measures often fall short of providing effective control. Accurate identification and image / audio retrieval of bird images and sounds are crucial methods for preventing bird-related damage in power transmission line environments.

[0004] In practical applications, bird call identification relies on the premise that the characteristics of the calls remain stable over time. Compared to human speech recognition, bird calls are more complex and varied, and differences between species or individuals are more difficult to understand. Furthermore, audio signals collected in real-world environments are noisy, including environmental noise, sounds from other organisms and human activities, as well as interference noise during signal transmission. These challenges make automatic bird identification a demanding task, urgently requiring innovative research in fundamental theories and key technologies to make a strong contribution to achieving efficient, non-destructive, low-interference, and large-scale monitoring.

[0005] Given the problems with traditional bird deterrent technology, there is a need for a more effective bird sound recognition and intelligent bird deterrent device that can significantly reduce the labor costs of inspection work and enhance the effectiveness of intelligent bird deterrents. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides an intelligent ultrasonic bird-repelling device and method based on deep learning bird sound recognition, which can effectively drive away birds and ensure the safe operation of power transmission lines.

[0007] To achieve the above objectives, the present invention provides an intelligent ultrasonic bird deterrent device based on deep learning bird sound recognition, comprising a power supply module for providing power to an ultrasonic generator; a sensor module for collecting bird sounds and transmitting signals to a control module; a control module for preprocessing the sounds and obtaining the acoustic features of the bird sounds through MFCC, recognizing the acoustic features and performing deep learning; an ultrasonic generator for generating ultrasonic signals; and a mounting bracket for mounting the device on a power transmission line.

[0008] Furthermore, this invention proposes an intelligent ultrasonic bird-repelling method based on deep learning bird sound recognition, the specific implementation steps of which include:

[0009] A bird sound sample collection system was constructed using sensor modules;

[0010] A bird sound recognition feature extraction model is constructed using the control module;

[0011] A bird sound recognition model was built based on a convolutional neural network (CNN).

[0012] An ultrasonic intelligent variable frequency bird deterrent module is constructed using an ultrasonic generator.

[0013] Furthermore, the sensor module includes a sound sensor, an operational amplifier, and an A / D conversion circuit. The sound sensor converts the changing waveform of the sound into a level signal, which is input to the operational amplifier. After amplification, an analog voltage signal of 0-10V is obtained and sent to the A / D conversion circuit. The A / D conversion circuit converts the voltage signal into a digital signal of 0-4096 to obtain the original bird sound sample.

[0014] Furthermore, the specific steps for constructing the bird sound recognition feature extraction model include:

[0015] Develop a convolutional codec based on the bird sound spectral characteristics of Conv-STFT and Conv-iSTFT.

[0016] A spectral subtraction method for multi-window spectral estimation of bird calls is established. The same bird call data sequence is segmented after STFT transformation. The power spectrum of each segment is calculated through multiple orthogonal data windows. Then, the average value of each segment is used as the spectral estimate, which can obtain a smaller estimation variance. Finally, the frequency domain is restored to the time domain through ISTFT to obtain the enhanced bird call data.

[0017] The enhanced bird call data was used to extract the acoustic features of the bird calls using Mel-frequency cepstral coefficients (MFCC).

[0018] Furthermore, the convolutional encoder-decoder transforms the data to the frequency domain using Conv-STFT, then adjusts the frequency band sharpness through spectral compression, and further extracts features through a complex encoder. It also extracts the phase difference between two different channels and inputs it into an angular phase encoder based on a complex convolutional network to further extract spatial information such as phase. After feature fusion, filtering is performed, and the filtered features are superimposed on the unfiltered features to enhance the effective features. Finally, the features are fed into the decoder, inversely transformed back to the frequency domain, and then transformed to the time domain using iSTFT.

[0019] Furthermore, the convolutional neural network (CNN) includes convolutional layers, recurrent layers, feedforward layers, and thresholding.

[0020] Furthermore, during the training process, the error in the bird sound recognition model gradually decreases until convergence as the number of learning iterations increases. To prevent overfitting of the bird sound recognition model due to multiple iterations, a portion of the bird sound training samples is used as a validation set during training to validate the network after each iteration. After the training process is completed, the acoustic features of the bird sound to be tested are input into the trained network model to obtain the classification result of the unknown bird sound.

[0021] Furthermore, the specific steps for constructing the ultrasonic intelligent variable frequency bird deterrent module include:

[0022] 1) The ultrasonic generator is activated based on the classification results of the bird sound recognition model for the signal of birds, and starts to work at a certain frequency.

[0023] 2) If the classification result signal does not change and still exists after a short working period, change the frequency of the ultrasound; repeat the change of different frequencies until the classification result signal changes or disappears.

[0024] 3) The memory stores the optimal frequency range for driving away the bird and feeds it back to the bird recognition model, so that when the bird is recognized, the ultrasonic waves can work in the optimal frequency range and automatically change the frequency within the range to avoid the bird's adaptation.

[0025] Furthermore, the control module also includes a remote communication module, which can transmit the operating status of the device to a remote monitoring center.

[0026] Furthermore, the ultrasonic generator includes multiple ultrasonic probes capable of emitting ultrasonic signals in different directions.

[0027] Furthermore, the ultrasonic generator can emit ultrasonic signals of various frequencies and intensities to suit different types of birds.

[0028] Furthermore, the ultrasonic generator can learn the optimal frequency range for repelling different birds and automatically change the frequency within the range to avoid the bird's adaptation.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] An intelligent ultrasonic bird deterrent device based on deep learning bird sound recognition has the advantages of accurate recognition and high level of intelligence. On the one hand, it frees substation inspection personnel from mechanical and tedious bird identification and statistics work, reduces the drawbacks of manual inspection, saves human resources and time costs, improves the bird identification rate, and avoids the problems of difficulty in bird identification and low targeting when the bird prevention area is large. On the other hand, it is also conducive to the protection of the ecological environment, enhances the sustainable development of the ecosystem, and provides timely and effective information for the prevention and control of bird damage, and has great practical value. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the implementation of an intelligent ultrasonic bird-repelling method based on deep learning bird sound recognition according to the present invention.

[0032] Figure 2 This is a diagram of the convolutional encoder-decoder structure in an intelligent ultrasonic bird-repelling method based on deep learning bird sound recognition according to the present invention.

[0033] Figure 3 This invention relates to the spectral subtraction and Mel-frequency cepstral coefficient feature extraction process in an intelligent ultrasonic bird-repelling method based on deep learning bird sound recognition.

[0034] Figure 4 This is a comparison of the spectrograms of bird sounds before and after enhancement in the intelligent ultrasonic bird-repelling method based on deep learning bird sound recognition according to the present invention.

[0035] Figure 5 This is a time-domain comparison diagram of bird sound denoising before and after multi-window spectrum estimation and spectral subtraction in an intelligent ultrasonic bird deterrence method based on deep learning bird sound recognition according to the present invention. Detailed Implementation

[0036] The following detailed description, in conjunction with the accompanying drawings, illustrates the specific implementation of an intelligent ultrasonic bird-repelling device and method based on deep learning bird sound recognition according to the present invention.

[0037] Specifically, this invention proposes an intelligent ultrasonic bird-repelling device based on deep learning bird sound recognition, characterized by comprising:

[0038] Power supply module, sensor module, control module, ultrasonic generator, and mounting bracket.

[0039] The power module provides power to the ultrasonic generator;

[0040] The sensor module is used to collect the sounds of birds and transmit the signals to the control module;

[0041] The control module is used to preprocess the sound and obtain the acoustic features of the bird calls through MFCC, and to identify and perform deep learning on the acoustic features.

[0042] The ultrasonic generator is used to generate ultrasonic signals;

[0043] The mounting bracket is used to install the intelligent ultrasonic bird deterrent device on the power transmission line.

[0044] The control module also includes a remote communication module, which can transmit the working status of the intelligent ultrasonic bird deterrent device to a remote monitoring center.

[0045] The ultrasonic generator includes multiple ultrasonic probes, which can emit ultrasonic signals in different directions;

[0046] The ultrasonic generator can emit ultrasonic signals of various frequencies and intensities to suit different types of birds.

[0047] The ultrasonic generator can learn the optimal frequency range for repelling different birds and automatically change the frequency within the range to avoid the bird's adaptation.

[0048] The sensor module includes a sound sensor, an operational amplifier, and an A / D conversion circuit. The sound sensor converts the changing waveform of the sound into a level signal, which is input to the operational amplifier. After amplification, an analog voltage signal of 0-10V is obtained and sent to the A / D conversion circuit. The A / D conversion circuit converts the voltage signal into a digital signal of 0-4096 to obtain the original bird sound sample.

[0049] Specifically, such as Figure 1As shown, the intelligent ultrasonic bird-repelling method of the intelligent ultrasonic bird-repelling device based on deep learning bird sound recognition of the present invention includes: constructing a bird sound sample acquisition system to collect bird sound samples from a substation; constructing a bird sound recognition feature extraction model, using a bird sound spectral feature convolutional encoder-decoder based on Conv-STFT and Conv-iSTFT and spectral subtraction for bird sound multi-window spectrum estimation to achieve bird sound sample denoising and enhancement, and using Mel-frequency cepstral coefficients (MFCC) to extract features from the denoised samples to obtain the acoustic features of the bird sounds; constructing a bird sound recognition model based on a convolutional neural network (CNN), using acoustic features extracted from a large amount of raw speech data as training data to train the bird sound recognition model, and using the trained bird sound recognition model to identify newly monitored bird sound samples to obtain the classification result of unknown bird sounds; constructing an ultrasonic intelligent frequency conversion bird-repelling module, where the classification result indicates that the ultrasonic generator is activated by birds, and the bird-repelling frequency is learned through deep learning.

[0050] Specifically, Figure 2 This paper details the convolutional encoder-decoder structure used for denoising in the bird sound recognition feature extraction model of this invention. After inputting audio, it is first transformed to the frequency domain using Conv-STFT, then frequency band clarity is adjusted through spectral compression, and features are further extracted using a complex encoder. The phase difference between two different channels is then extracted and input into an angular phase encoder based on a complex convolutional network to further extract spatial information such as phase. After feature fusion, filtering is performed, and the filtered features are superimposed on the unfiltered features to enhance the effective features. Finally, the features are fed into the decoder, inversely transformed back to the frequency domain, and then transformed to the time domain using iSTFT to obtain the denoised and enhanced bird sound sample.

[0051] Based on the encoder-decoder structure diagram, a quantization-encoder adversarial training loss function is designed, as shown in the following formula: Detailed Implementation

[0052] The following detailed description, in conjunction with the accompanying drawings, illustrates the specific implementation of an intelligent ultrasonic bird-repelling device and method based on deep learning bird sound recognition according to the present invention.

[0053] Specifically, this invention proposes an intelligent ultrasonic bird-repelling device based on deep learning bird sound recognition, characterized by comprising:

[0054] Power supply module, sensor module, control module, ultrasonic generator, and mounting bracket.

[0055] The power module provides power to the ultrasonic generator;

[0056] The sensor module is used to collect the sounds of birds and transmit the signals to the control module;

[0057] The control module is used to preprocess the sound and obtain the acoustic features of the bird calls through MFCC, and to identify and perform deep learning on the acoustic features.

[0058] The ultrasonic generator is used to generate ultrasonic signals;

[0059] The mounting bracket is used to install the intelligent ultrasonic bird deterrent device on the power transmission line.

[0060] The control module also includes a remote communication module, which can transmit the working status of the intelligent ultrasonic bird deterrent device to a remote monitoring center.

[0061] The ultrasonic generator includes multiple ultrasonic probes, which can emit ultrasonic signals in different directions;

[0062] The ultrasonic generator can emit ultrasonic signals of various frequencies and intensities to suit different types of birds.

[0063] The ultrasonic generator can learn the optimal frequency range for repelling different birds and automatically change the frequency within the range to avoid the bird's adaptation.

[0064] The sensor module includes a sound sensor, an operational amplifier, and an A / D conversion circuit. The sound sensor converts the changing waveform of the sound into a level signal, which is input to the operational amplifier. After amplification, an analog voltage signal of 0-10V is obtained and sent to the A / D conversion circuit. The A / D conversion circuit converts the voltage signal into a digital signal of 0-4096 to obtain the original bird sound sample.

[0065] Specifically, such as Figure 1 As shown, the intelligent ultrasonic bird-repelling method of the intelligent ultrasonic bird-repelling device based on deep learning bird sound recognition of the present invention includes: constructing a bird sound sample acquisition system to collect bird sound samples from a substation; constructing a bird sound recognition feature extraction model, using a bird sound spectral feature convolutional encoder-decoder based on Conv-STFT and Conv-iSTFT and spectral subtraction for bird sound multi-window spectrum estimation to achieve bird sound sample denoising and enhancement, and using Mel-frequency cepstral coefficients (MFCC) to extract features from the denoised samples to obtain the acoustic features of the bird sounds; constructing a bird sound recognition model based on a convolutional neural network (CNN), using acoustic features extracted from a large amount of raw speech data as training data to train the bird sound recognition model, and using the trained bird sound recognition model to identify newly monitored bird sound samples to obtain the classification result of unknown bird sounds; constructing an ultrasonic intelligent frequency conversion bird-repelling module, where the classification result indicates that the ultrasonic generator is activated by birds, and the bird-repelling frequency is learned through deep learning.

[0066] Specifically, Figure 2This paper details the convolutional encoder-decoder structure used for denoising in the bird sound recognition feature extraction model of this invention. After inputting audio, it is first transformed to the frequency domain using Conv-STFT, then frequency band clarity is adjusted through spectral compression, and features are further extracted using a complex encoder. The phase difference between two different channels is then extracted and input into an angular phase encoder based on a complex convolutional network to further extract spatial information such as phase. After feature fusion, filtering is performed, and the filtered features are superimposed on the unfiltered features to enhance the effective features. Finally, the features are fed into the decoder, inversely transformed back to the frequency domain, and then transformed to the time domain using iSTFT to obtain the denoised and enhanced bird sound sample.

[0067] Based on the encoder-decoder structure diagram, a quantization-encoder adversarial training loss function is designed, as shown in the following formula: Detailed Implementation

[0068] The following detailed description, in conjunction with the accompanying drawings, illustrates the specific implementation of an intelligent ultrasonic bird-repelling device and method based on deep learning bird sound recognition according to the present invention.

[0069] Specifically, this invention proposes an intelligent ultrasonic bird-repelling device based on deep learning bird sound recognition, characterized by comprising:

[0070] Power supply module, sensor module, control module, ultrasonic generator, and mounting bracket.

[0071] The power module provides power to the ultrasonic generator;

[0072] The sensor module is used to collect the sounds of birds and transmit the signals to the control module;

[0073] The control module is used to preprocess the sound and obtain the acoustic features of the bird calls through MFCC, and to identify and perform deep learning on the acoustic features.

[0074] The ultrasonic generator is used to generate ultrasonic signals;

[0075] The mounting bracket is used to install the intelligent ultrasonic bird deterrent device on the power transmission line.

[0076] The control module also includes a remote communication module, which can transmit the working status of the intelligent ultrasonic bird deterrent device to a remote monitoring center.

[0077] The ultrasonic generator includes multiple ultrasonic probes, which can emit ultrasonic signals in different directions;

[0078] The ultrasonic generator can emit ultrasonic signals of various frequencies and intensities to suit different types of birds.

[0079] The ultrasonic generator can learn the optimal frequency range for repelling different birds and automatically change the frequency within the range to avoid the bird's adaptation.

[0080] The sensor module includes a sound sensor, an operational amplifier, and an A / D conversion circuit. The sound sensor converts the changing waveform of the sound into a level signal, which is input to the operational amplifier. After amplification, an analog voltage signal of 0-10V is obtained and sent to the A / D conversion circuit. The A / D conversion circuit converts the voltage signal into a digital signal of 0-4096 to obtain the original bird sound sample.

[0081] Specifically, such as Figure 1 As shown, the intelligent ultrasonic bird-repelling method of the intelligent ultrasonic bird-repelling device based on deep learning bird sound recognition of the present invention includes: constructing a bird sound sample acquisition system to collect bird sound samples from a substation; constructing a bird sound recognition feature extraction model, using a bird sound spectral feature convolutional encoder-decoder based on Conv-STFT and Conv-iSTFT and spectral subtraction for bird sound multi-window spectrum estimation to achieve bird sound sample denoising and enhancement, and using Mel-frequency cepstral coefficients (MFCC) to extract features from the denoised samples to obtain the acoustic features of the bird sounds; constructing a bird sound recognition model based on a convolutional neural network (CNN), using acoustic features extracted from a large amount of raw speech data as training data to train the bird sound recognition model, and using the trained bird sound recognition model to identify newly monitored bird sound samples to obtain the classification result of unknown bird sounds; constructing an ultrasonic intelligent frequency conversion bird-repelling module, where the classification result indicates that the ultrasonic generator is activated by birds, and the bird-repelling frequency is learned through deep learning.

[0082] Specifically, Figure 2 This paper details the convolutional encoder-decoder structure used for denoising in the bird sound recognition feature extraction model of this invention. After inputting audio, it is first transformed to the frequency domain using Conv-STFT, then frequency band clarity is adjusted through spectral compression, and features are further extracted using a complex encoder. The phase difference between two different channels is then extracted and input into an angular phase encoder based on a complex convolutional network to further extract spatial information such as phase. After feature fusion, filtering is performed, and the filtered features are superimposed on the unfiltered features to enhance the effective features. Finally, the features are fed into the decoder, inversely transformed back to the frequency domain, and then transformed to the time domain using iSTFT to obtain the denoised and enhanced bird sound sample.

[0083] Based on the encoder-decoder structure diagram, the quantization encoding-decoder adversarial training loss function is designed as follows:

[0084] The reconstruction loss is used to measure the difference between the original input x and the reconstructed output. The similarity is used to optimize the encoder-decoder parameters, and the second quantization loss is to minimize the encoding vector. and the most recent codebook vector The distance between them is calculated, but the gradient in the codebook space is stopped first, and only the encoder parameters are updated. The last term, commitment loss, is only used to update the parameters in the codebook space, using exponential averaging instead of gradient information, thus avoiding the posterior collapse problem.

[0085] The alignment loss function is shown in the following equation:

[0086] in and It is an encoded representation of two sequentially consecutive inputs.

[0087]

[0088] in For the loss function sample set, Training samples for the loss function, Training samples for the decoder loss function, Let y be the training samples for the encoder loss function, z be the given label, and z be the codebook representation.

[0089] The total loss function for quantized adversarial training is:

[0090] The above formula uses , , Three loss functions for training data, among which Based on the loss function, To compare loss functions, To identify the loss function.

[0091] Specifically, the comparison function is shown in the following formula:

[0092] in, and The original data were generated from two different noisy samples, and then compared using a contrastive loss to bring them closer together. and and The distance ensures that the original sample distribution is correct when reconstructed, and continues to sample from that distribution, backdiffusion, and generate clean samples.

[0093] The reconstructed inverse diffusion function is shown in the following equation:

[0094] in , The samples were selected during the reverse diffusion process. It is a discriminator.

[0095] The target loss function of the diffusion generator is shown in the following equation:

[0096] in Based on the loss function, The sample generated after diffusion.

[0097] Specifically, Figure 3 This paper details the spectral subtraction and Mel-frequency cepstral coefficient (MFCC) feature extraction processes in the bird sound recognition feature extraction model of this invention. Paths A, B, C, and D correspond to the generation processes of four commonly used time-frequency spectrograms. Path A demonstrates the generation process of a linear grayscale spectrum. After simple preprocessing, the audio data is transformed into fixed-format sound segments, which are then subjected to Discrete and Fast Fourier Transform (DFT). After taking the modulus of the transform results, normalization is performed to obtain the final linear grayscale spectrum. Path B involves taking the modulus of the transform results and then performing a logarithmic transform again, followed by normalization to obtain the logarithmic grayscale spectrum. Paths C and D represent the generation processes of logarithmic Mel features and Mel-frequency cepstral coefficients, respectively. The difference in MFCC lies in the addition of a Discrete Cosine Transform (DCT) after the Mel filter.

[0098] In one embodiment, the bird sound recognition feature extraction model includes spectral subtraction of multi-window spectral estimation;

[0099] Let the noisy audio be x(n), and after windowing and frame division, it is... Calculate the amplitude spectrum after STFT transformation. and phase spectrum Smoothing is performed between adjacent frames, and the average amplitude spectrum is calculated. .

[0100]

[0101] Take M frames before and after i as the center, for a total of 2M+1 frames, and then average them.

[0102] Framed signal Perform multi-window spectrum estimation to obtain the multi-window spectrum power spectral density P(k,i) (i represents the i-th frame, k represents the k-th spectral line).

[0103]

[0104] In the formula, PMTM represents the multi-window power spectral density estimation.

[0105] Calculate smoothed power spectral density .

[0106]

[0107] The frame number corresponding to the silent preceding segment (noise) is NIS, from which the average power spectral density value of the noise can be calculated. .

[0108]

[0109] Calculate the gain factor using the spectral subtraction relation:

[0110] In the formula, a is the over-reduction factor; b is the gain compensation factor.

[0111] Through gain factor and average amplitude spectrum The amplitude spectrum after spectral subtraction can be obtained:

[0112] Amplitude spectrum after spectral subtraction Combined with the phase spectrum in the steps Perform iSTFT, frequency domain Restoring to the time domain yields the noise-reduced signal. .

[0113] Specifically, Figure 4 The spectrum comparison diagrams before and after bird sound enhancement are given. Figure (a) is the spectrum diagram of clean speech, Figure (b) is the spectrum diagram of noisy speech, Figure (c) is the spectrum diagram after Conv-STFT enhancement, and Figure (d) is the spectrum diagram after Conv-iSTFT enhancement.

[0114] Specifically, Figure 5 The time-domain comparison diagrams before and after bird sound denoising are presented. After spectral subtraction denoising, the time-domain waveform of the clean speech component is more prominent, the waveform distortion is reduced, and the enhancement effect of the speech signal is more obvious.

[0115] According to embodiments of this disclosure, the bird sound recognition model provided is an N-layer cascaded convolutional neural network (CNN), where N > 1. Each cascaded layer includes a feature extraction layer, a dual-path time-frequency joint modeling unit, a skip connection layer, and an auxiliary branch classifier connected sequentially. Preprocessed bird sound sample data is input into the N-layer cascaded bird model for training to obtain N classification results. Then, the bird classification result is obtained based on the N classification results. Specifically, by inputting the preprocessed bird sound sample data into the feature extraction layer to extract and output the first feature data, and inputting the first feature data into the dual-path time-frequency joint modeling unit, the feature extraction of local features of the time-frequency structure can be realized. At the same time, the global dependency relationship of time dependency and frequency dependency can be constructed to obtain the second feature data. The second feature data is input into the skip connection layer, and the third feature data output from the skip connection layer is input into the auxiliary branch classifier for classification and the next layer of the cascaded structure for feature extraction and classification processing. This fully considers the impact of classification results at different levels in the cascaded structure on the final bird classification result, and effectively improves the accuracy of the bird sound recognition model in recognizing the sound of the target bird.

[0116] Specifically, the feature extraction layer provided in this disclosure consists of two convolutional layers with a kernel size of 3*3 and a stride of 1, a batch normalization and activation function layer, and an average pooling layer stacked together. The activation function layer is a linear rectified function, and the average pooling layer is an adaptive average pooling layer. The feature extraction process of the feature extraction layer can be represented by the following equation:

[0117] Where x is the preprocessed bird sound sample data or the third feature output of the skip connection layer in the i-1 layer cascade structure, i≤N, N>1; This represents the output after processing by the first convolutional layer, batch normalization, and activation function layer. This represents the output after processing by the second convolutional layer, batch normalization, and activation function layer. This represents the first feature data after processing by the average pooling layer.

[0118] Subsequently, M Mel filters (M-dimensional triangular filters) are used to filter the signal energy spectrum. The equal-height Mel filter bank composed of 64 triangular filters can well mimic human auditory perception, with the filters concentrated in the low-frequency range and relatively sparse in the high-frequency range.

[0119] Specifically, standard MFCC only reflects the static characteristics of speech parameters, while first-order difference MFCC (ΔMFCC) is a dynamic parameter that reflects the dynamic characteristics of speech parameters and has better robustness. Based on first-order difference MFCC, second-order difference MFCC can be further calculated. Signal parameterization methods and advanced pattern recognition technology have enabled the automatic identification of bird sounds. The proposed automatic identification method uses MFCC as a sound feature and GMM as a classifier. The method achieves an average recognition rate of 98.95% when identifying multiple types of bird sounds, and the time required to identify a sound sample of about 1 second is about 300ms. It shows good performance in terms of recognition accuracy and recognition time.

[0120] Specifically, the bird sound recognition model includes:

[0121] Convolutional layers: The extracted acoustic features of bird calls are input into CNN layers with two-dimensional convolutional filters. For each CNN layer, after passing the feature map output through the ReLU activation function, non-overlapping max pooling is used to reduce the dimensionality of the data and provide more frequency invariance.

[0122] Recurrent layers: The output of the convolutional layers is used as a frame sequence. Feed to the loop layer. The loop layer consists of... It consists of stacked recursive layers, each layer calculating and outputting the hidden vector for each frame. .

[0123]

[0124]

[0125]

[0126] In the formula: function F can represent a Long Short-Term Memory (LSTM) unit, which has two inputs: the output of the current frame of the previous loop layer. and the previous frame output of the current layer .

[0127] Feedforward layer: Following the recurrent layer is a feedforward layer, which serves as the output layer of the network. The output of the feedforward layer is activated from the last recurrent layer. Obtained.

[0128]

[0129] In the formula: G represents a feedforward layer with sigmoid activation, which applies the same set of weights to the features extracted from each frame; This is the output of the loop layer; This is the output after passing through a feedforward layer.

[0130] Thresholding: Output of the feedforward layer Used to calculate the event activity probability for each class, when k = 1, 2, 3, ..., k:

[0131] In the formula: k represents the number of classes; Parameters representing all layers in the network composition; This represents the output of the feedforward layer. It represents the activity probability of event k in frame t. After a series of convolution and pooling operations in the neural network, the final output is the predicted probability value of the corresponding bird sound.

[0132] Specifically, the formula for constructing the Gaussian mixture model is as follows:

[0133] In the formula, Let be a D-dimensional random vector. Let be the density of each component, and let be a Gaussian function with degree D that varies over each component density, where i takes the values ​​1, 2, ..., M. For mixed weights, For parameters.

[0134] The density calculation formulas for each component are as follows:

[0135] In the formula, It is the mean vector. Let be the covariance matrix, and let the mixed weights satisfy the relationship D is a variable Gaussian function, and exp is an exponential function.

[0136] Specifically, the density of the Gaussian mixture model is composed of the mean vector, covariance matrix, and mixture weight parameterization of all components.

[0137] In one embodiment, a re-estimation algorithm is used during the iteration process to ensure that the mode likelihood value decreases monotonically.

[0138] The average mixed weight is calculated using the following formula:

[0139] The posterior probability of the sound category is calculated as follows:

[0140] In the formula, For parameters, For mixed weights, Let the density of the i-th component be... For the new weights, Let be the density of the k-th component, where i and k are values ​​in the sound sample.

[0141] Specifically, the ultrasonic intelligent variable frequency bird deterrent module includes:

[0142] The classification results from the bird sound recognition model are used to activate the ultrasonic generator for the bird signal, which then begins to operate at a certain frequency.

[0143] If the classification result signal remains unchanged and persists for a short period of time, change the frequency of the ultrasound; repeat the process of changing different frequencies until the classification result signal changes or disappears.

[0144] The memory stores the optimal frequency range for driving away the bird and feeds it back to the bird recognition model, so that when the bird is recognized, the ultrasonic waves can work in the optimal frequency range and automatically change the frequency within the range to prevent the bird from adapting.

[0145] Specifically, when sorting samples based on the learner's predictions, the samples with the "most likely" bird-repelling frequency predicted by the learner can be placed first, and the samples with the "least likely" bird-repelling frequency predicted by the learner can be placed last. By treating the samples as positive examples one by one in this order, the current bird-repelling rate and accuracy can be calculated.

[0146] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An intelligent ultrasonic wave bird repelling method based on deep learning bird sound recognition, characterized in that, The intelligent ultrasonic bird repelling method adopts an intelligent ultrasonic bird repelling device based on deep learning bird sound recognition to intelligently repel birds, and the intelligent ultrasonic bird repelling device specifically comprises a power module, a sensor module, a control module, an ultrasonic generator and a mounting bracket, the power module provides power for the ultrasonic generator; the sensor module is used to collect the sound of birds and transmit the signal to the control module; the control module is used to pre-process the sound and obtain the acoustic characteristics of the bird sound through MFCC, identify and deeply learn the acoustic characteristics; the ultrasonic generator is used to generate ultrasonic signals; and the mounting bracket is used to mount the intelligent ultrasonic bird repelling device on the power transmission line; The intelligent ultrasonic bird repelling method specifically comprises the following steps: S1: constructing a bird sound sample collection system through a sensor module; S2: constructing a bird sound recognition feature extraction model through a control module, specifically comprising: S21: developing a convolutional encoder-decoder for bird sound spectrum features based on Conv-STFT and Conv-iSTFT; The convolutional encoder-decoder is converted to the frequency domain through Conv-STFT, then the frequency band clarity is adjusted through spectral compression, and then the features are further extracted through a complex encoder; the phase difference of two different channels is extracted and input into an angular phase encoder based on a complex convolutional network to further extract the phase space information; after feature fusion, filtering is performed, and the filtered features are superimposed on the features before filtering to enhance the effective features; finally, the features are sent to a decoder for inverse transformation back to the frequency domain and then transformed to the time domain through iSTFT; S22: establishing a bird sound multi-window spectrum estimation spectral subtraction, segmenting the STFT-converted bird sound data sequence, calculating the power spectrum of each segment through multiple orthogonal data windows, then taking the average value of each segment as the spectrum estimation, so that a smaller estimation variance can be obtained, and finally restoring the frequency domain to the time domain through ISTFT to obtain enhanced bird sound data; S23: extracting the enhanced bird sound data using Mel-frequency cepstral coefficients (MFCC) to obtain the acoustic characteristics of the bird sound; S3: constructing a bird sound recognition model based on a convolutional neural network (CNN), which comprises a convolutional layer, a recurrent layer, a feedforward layer and a thresholding; During the training process of the bird sound recognition model, the error in the network gradually decreases with the increase of the number of learning times until convergence; to prevent overfitting of the bird sound recognition model caused by multiple iterations, a part of the bird sound training samples are taken as a validation set to verify the network after each iteration during the training process; after the training process is completed, the acoustic characteristics of the bird sound to be tested are input into the trained network model, and the classification result of the unknown bird sound can be obtained; S4: constructing an ultrasonic intelligent frequency conversion bird repelling module through an ultrasonic generator, specifically comprising the following steps: S41: starting the ultrasonic generator for a bird sound recognition model to start working at a certain frequency; S42: working short time, if the classification result signal does not change and still exists, change the frequency of ultrasonic wave; repeat changing different frequencies until the classification result signal changes or disappears; S43: the memory stores the best frequency interval of driving the bird, and feeds back to the bird identification model, so that the ultrasonic wave can work in the best frequency interval when the bird is identified, and the frequency can be automatically changed in the interval to avoid the adaptability of the bird.

2. The intelligent ultrasonic bird repelling method based on bird sound recognition of deep learning according to claim 1, characterized in that, The control module further comprises a remote communication module, which can transmit the working state of the intelligent ultrasonic bird repelling device to a remote monitoring center.

3. The intelligent ultrasonic bird repelling method based on bird sound recognition of deep learning according to claim 2, characterized in that, The ultrasonic wave generator comprises a plurality of ultrasonic wave probes, which can emit ultrasonic wave signals in different directions; The ultrasonic wave generator can emit ultrasonic wave signals of different frequencies and intensities to adapt to different types of birds; The ultrasonic wave generator can deeply learn the best frequency interval of driving different birds and automatically change the frequency in the interval to avoid the adaptability of the bird.

4. The intelligent ultrasonic bird repelling method based on bird sound recognition of deep learning according to claim 3, characterized in that, The sensor module comprises a sound sensor, an operational amplifier and an A / D conversion circuit; the sound sensor converts the change waveform of sound into a level signal input to the operational amplifier, obtains an analog voltage signal of 0-10V after amplification processing, and sends it to the A / D conversion circuit; the voltage signal is converted into a digital quantity signal of 0-4096 by the A / D conversion circuit to obtain the original bird sound sample.

Images