Lightweight high-robustness transmission line high-risk bird voiceprint recognition method

Abstract

The invention discloses a light-weight high-robustness transmission line high-risk bird voiceprint recognition method, which comprises the following steps: collecting birdsong audio in real time, obtaining an audio sample containing birdsong, uniformly preprocessing the obtained birdsong audio sample, and obtaining a bird voiceprint recognition result; then, Mel-frequency cepstrum coefficients, dynamic information changing along with time and linear prediction cepstrum coefficients corresponding to the audio signals are obtained through MFCC, Delta and LPCC methods respectively, and mixed features are obtained after splicing is carried out on the Mel-frequency cepstrum coefficients, the dynamic information changing along with time and the linear prediction cepstrum coefficients; and finally, inputting the mixed features into the constructed lightweight voiceprint classification network model, and outputting a classification result. The method has the advantages of being high in classification precision, wide in application range, low in occupied resource and high in environmental noise robustness, the network model is designed in a lightweight mode, the calculation speed is high, and the requirement for lightweight high-robustness twitter audio classification is met. According to the method, the automation level of bird audio recognition can be remarkably improved, manual intervention is reduced, and the method has wide practical application value in the field of bird voiceprint recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of power transmission line safety, and in particular relates to a lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines. Background Art

[0002] As electricity demand grows, the coverage of transmission lines continues to expand, especially in areas with complex ecological environments such as wetlands, forests and mountains, which are also habitats and migration routes for birds. The frequent activities of birds near transmission lines may affect their survival and migration, and pose a threat to the security of the power grid.

[0003] Faults caused by birds account for a considerable proportion of transmission line accidents. For example, bird nesting causes insulation pollution and line tripping; hitting high-voltage lines causes short circuits or disconnections; and excrement may cause equipment corrosion. Especially during the migration season, the incidence of such accidents increases, seriously threatening the stable operation of the power grid. Traditional power grid maintenance mainly relies on manual inspections, which is inefficient and difficult to detect potential threats in a timely manner, especially in remote areas. Existing monitoring mostly relies on image recognition, which takes up a lot of resources and has a limited range of bird recognition.

[0004] At present, bird voiceprint recognition technology has gradually attracted attention due to its all-weather monitoring and low-cost advantages, but the model's low resource consumption and adaptability to different environments face challenges. Therefore, a lightweight and highly robust technology for identifying bird voiceprints is needed to monitor and predict the threat of birds to the power grid in real time and ensure the safe operation of the power system. Summary of the invention

[0005] In view of the shortcomings of the existing technology in the recognition of high-risk bird voiceprints on power transmission lines, such as occupying too many resources, high cost and weak resistance to environmental noise, difficulty in processing diverse audio data formats, and difficulty in generalizing models when data is scarce, the present invention discloses a lightweight and highly robust bird voiceprint recognition method. This method achieves efficient and accurate bird voiceprint classification through audio preprocessing, audio data enhancement, audio multi-feature fusion strategy and a lightweight neural network model based on deep separable convolution and dense connection, and has the advantages of strong noise robustness, wide application range, and low computing resource requirements.

[0006] A lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines comprises: preprocessing the obtained bird call audio samples in the power transmission line environment, respectively obtaining the Mel frequency cepstral coefficients, dynamic information changing over time and linear prediction cepstral coefficients corresponding to the audio signal through MFCC feature extraction method, Delta method and LPCC feature extraction method, and then splicing the above three features to obtain the spliced ​​mixed features; finally, inputting the mixed features into the constructed lightweight voiceprint classification network model, and outputting the classification results.

[0007] The bird-singing audio samples are selected from the data set of high-risk bird-singing sounds on power transmission lines that have been collected, or collected in real time; when real-time bird-singing audio is collected, the presence of bird-singing sounds is determined by setting the device wake-up threshold. When bird-singing sounds are detected, the device starts timed long audio recording, and voice activity detection is used to remove silent segments from the recorded audio segments. The audio is then normalized and filled with zeros to unify the shape, and finally an audio sample with bird-singing sounds is obtained. During actual collection, in order to extend the life of the equipment and save power, storage and other resources, we need to set the device wake-up threshold (decibel) according to the power equipment environment in different regions to ensure that only the part containing valid voiceprint information is recorded. The recorded samples use voice activity detection to remove invalid segments to reduce irrelevant information, and then the signal is normalized and filled with zeros to ensure a uniform shape.

[0008] During the model training phase, we used the collected audio data set of birds at high risk of power transmission lines to perform one or more of the following generalization enhancement processing on the bird song audio samples: random time offset, random superposition of background noise, and linear mixing of two different audio samples; finally, we obtained the final training set samples. Later, the real-time collected audio data can also be saved to the corresponding folder after model classification as a supplementary transmission line high-risk bird sample set to improve the richness of the data set.

[0009] Specifically, the model training and usage process is as follows:

[0010] (1) Setting a threshold to wake up the device to collect bird song audio data in real time, cropping appropriate segments, removing silent segments, resampling, post-normalization, and padding;

[0011] (2) Randomly time-shift the preprocessed audio data, randomly superimpose background noise, and linearly mix two different audio samples;

[0012] (3) It is proposed to extract features from audio data by combining the static features of MFCC with the dynamic features of Delta and the temporal prediction features of LPCC;

[0013] (4) Construct a lightweight voiceprint classification network model based on deep separable convolution and dense connection, and set up a channel attention mechanism to effectively extract the global features of the audio through multi-layer convolution and statistical pooling, and finally perform classification through a fully connected layer;

[0014] (5) Apply the trained neural network to new audio data and output the voiceprint recognition results to achieve accurate classification of the target bird’s voiceprint.

[0015] Furthermore, the bird song audio samples were processed to enhance generalization as follows:

[0016] (1) Superimpose Gaussian noise on the bird song audio signal to generate noisy audio samples;

[0017] (2) randomly shifting the noisy frequency signal obtained in step (1) to obtain shifted noisy frequency samples;

[0018] (3) Use the Mixup method to randomly select two bird song audio samples and linearly mix them to obtain a new audio sample;

[0019] The original bird song audio samples and the audio samples obtained in steps (1) to (3) together constitute the training set samples.

[0020] Furthermore, the mixed feature Feature-Mix is ​​composed as follows:

[0021] Feature-Mix = [MFCC(n), Δ MFCC (t), LPCC(m)]

[0022] Where: MFCC(n) represents the nth Mel frequency cepstral coefficient obtained by MFCC feature extraction method, n = 1, 2, ..., N, N represents the total feature dimension extracted; Δ MFCC (t) represents the dynamic information of the audio signal changing with time at time t obtained by the Delta method; LPCC(m) represents the mth linear prediction cepstral coefficient obtained by the LPCC feature extraction method, and m represents the order of the current cepstral coefficient.

[0023] The TDNN network mainly includes a backbone network, a pooling layer, a linear fully connected layer, and a classification layer. The backbone network extracts features from time series through a series of time-delayed convolutions. Its structure is simple and suitable for processing time series signal data. However, with the application of the model in complex tasks, TDNN has certain limitations in capturing global features, reducing computational costs, and improving feature expression capabilities. This method adjusts the structure of TDNN, significantly improving the feature extraction capability of the network while reducing computational complexity. Specific improvements include the introduction of deep separable convolutions, which decomposes the standard convolution of TDNN into deep convolutions and point-by-point convolutions, which greatly reduces the number of network parameters and computational complexity, making the model more lightweight; at the same time, the outputs of different convolutional layers are spliced ​​by dense connections, effectively enhancing the network's ability to capture multi-scale features. In addition, in order to further improve the model's feature expression capabilities, a channel attention mechanism module is added. By dynamically adjusting the feature weights of different channels, the network's attention to important features and suppression of invalid features are significantly enhanced.

[0024] More specifically, the lightweight voiceprint classification network model adopts an improved TDNN network structure, which mainly includes a backbone network, a pooling layer, a linear fully connected layer and a classification layer. In the backbone network, the features of different time frames are extracted and processed layer by layer through multiple deeply separable convolutions, and the outputs of different convolutional layers are feature spliced ​​through dense connections. Finally, through the channel attention mechanism, the feature weights of different channels are dynamically adjusted to effectively extract the global features of the audio.

[0025] In order to reduce the amount of computation of the model, this method introduces depthwise separable convolution based on TDNN. Traditional convolution performs convolution operations on all input channels at each layer, while depthwise separable convolution divides this process into two steps: first, each channel is convolved independently (depthwise convolution), and then each channel is linearly combined through pointwise convolution (1×1 convolution). This not only reduces the computational complexity, but also maintains the effectiveness of feature extraction.

[0026] In addition, this method adopts a dense connection design, that is, the output features of the previous layer are concatenated with the output of the current layer. In this way, the model can reuse features from different layers, thereby improving the ability to capture multi-scale features and reducing the problem of information loss. In specific implementation, the output of the second layer of convolution is concatenated with the output of the first layer and then passed to the subsequent convolution layer. The dense connection design enables the model to better integrate features at different levels and improve classification accuracy.

[0027] Furthermore, the depthwise separable convolution includes depthwise convolution and pointwise convolution;

[0028] In each layer of the backbone network of the lightweight voiceprint classification network model, each channel is independently deep convolved, and then each channel is linearly combined through point-by-point convolution. Finally, the point-by-point convolution output is batch normalized to obtain the output features of this layer.

[0029] At the same time, a dense connection design is adopted, that is, the output features of the previous layer are concatenated with the output features of the current layer, and then passed to the subsequent convolutional layer as the input of the next layer;

[0030] Finally, a channel attention mechanism module is added: the output features of each channel in the last layer are recalibrated to obtain the final output of the backbone network.

[0031] The lightweight and highly robust method for identifying high-risk birds on power transmission lines disclosed in the present invention realizes efficient processing and classification of diversified audio data in response to the challenge of low resource occupation and high robustness of voiceprint identification. The method has high recognition accuracy, is applicable to a variety of environments, and has strong anti-interference ability to complex background noise. At the same time, the network structure is lightweight and the calculation speed is fast. It is suitable for efficient voiceprint identification tasks, significantly improves the overall performance of the voiceprint identification system, and reduces hardware requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the process flow of a high-risk bird voiceprint recognition method on lightweight and highly robust transmission lines. DETAILED DESCRIPTION

[0033] In order to describe the present invention in more detail, the technical solution of the present invention is described in detail below in conjunction with specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0034] The present invention provides a lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines, which achieves the goal of automatically and accurately classifying bird song audio under low resource occupation and different soundscape conditions. The method preprocesses the collected or real-time recorded high-risk bird song audio on power transmission lines to standardize and unify the audio data samples, and generates diversified audio data samples using noise enhancement, random offset and Mixup. The MFCC, Delta and LPCC features of the audio data are extracted to capture the spectral information, dynamic change information and timing information of the audio, and finally feature fusion is performed to greatly improve the richness of information. At the same time, the TDNN network structure is improved, the depth separable convolution is added to reduce the computational complexity, the dense connection module is set to enhance the multi-scale feature fusion, the channel attention mechanism is added to optimize the feature selection, and finally the global timing information is integrated through statistical pooling, and the bird song audio is classified.

[0035] like Figure 1As shown, a method for identifying high-risk bird voiceprints under complex background sounds includes the following steps:

[0036] (1) Collect audio data and perform preprocessing.

[0037] The device uses mono recording with a recording time of 3 seconds and a sampling rate of 16kHz. In order to extend the life of the device and save power, storage and other resources, we need to set the device wake-up threshold (decibel) according to the power equipment environment in different regions to ensure that only the part containing valid voiceprint information is recorded. The recorded samples use voice activity detection to remove invalid fragments, and then the signal is normalized and padded with zeros to ensure a uniform shape.

[0038] In the actual modeling process, the collected data sets of birdsongs at high risk of transmission lines can be used, or the real-time recorded audio data can be used: the real-time recorded data is classified by the model and saved in the folder of the corresponding category as a supplementary sample set of birds at high risk of transmission lines. The two can together form the training sample set of the model.

[0039] In actual detection, the collected data set of high-risk bird singing on transmission lines can be used as model input, or real-time recorded audio data can be used: by setting the device wake-up threshold to determine whether there is bird singing. When bird singing is detected, the device starts timed long audio recording, and uses voice activity detection to remove silent segments from the recorded audio clips. The audio is then normalized and padded with zeros to unify the shape. Finally, an audio sample with bird singing is obtained as a verification sample for the model.

[0040] (2) Enhance the generalization ability of audio data.

[0041] This step is mainly applicable to the modeling process. For the actual detection process, step (2) can be omitted and step (3) can be directly entered.

[0042] Gaussian noise is superimposed on the samples of the high-risk bird call data set of power transmission lines to generate noisy audio samples. The intensity of the noise is controlled by the signal-to-noise ratio and is set to a random value in the range of [10dB, 30dB]. Then, the noisy audio samples obtained above are randomly offset, and the maximum offset is set to 10% to adapt the model to different timing situations. Using the Mixup technology, two bird call audio samples are randomly selected for linear mixing to obtain new audio samples, which improves the generalization ability of the samples while making up for the scarcity of some bird audio. Using the above method, a data-enhanced sample set is finally obtained.

[0043] (3) Feature fusion extraction.

[0044] A single feature often cannot reflect the delicate characteristics of bird calls. By extracting features at multiple levels, the network can better learn the different dimensions of bird voiceprints, which helps improve the robustness and generalization ability of the model, especially when the audio quality is poor or the background is complex.

[0045] In order to transform the time-continuous audio signal into a high-dimensional feature representation and highlight the changing characteristics of bird sounds, and fully improve the utilization of audio information, we extract and fuse features using the following methods:

[0046] First, the MFCC method is used to extract the spectrum information of the audio signal: the audio after data enhancement (in real-time detection, the acquired bird audio samples are directly processed) is pre-emphasized with a pre-emphasis coefficient of 0.95, and then the signal is framed, and a Hamming window is applied to each frame for windowing processing with a window length of 400. The fast Fourier transform (FFT) of each frame of the framed signal is performed with a length of 512, and 26 groups of Mel filters are used to convert the frequency axis to Mel scale, and the logarithmic energy after filtering is calculated, and the first several cepstrum coefficients are extracted through discrete cosine transform (DCT). The calculation formula is as follows:

[0047]

[0048] In formula (1):

[0049] MFCC(n): represents the nth Mel-frequency cepstral coefficient, which is one of the extracted audio features.

[0050] DCT{} indicates discrete cosine transform.

[0051] S m : represents the weight of the mth Mel filter, reflecting the contribution to the spectrum on the Mel frequency scale.

[0052] Represents the spectral energy after processing by the Mel filter bank, usually the frequency energy obtained by filtering the FFT result.

[0053] M: The total number of Mel filters determines the number of filters used to process the spectrum. The value is usually determined according to application requirements.

[0054] N: represents the total feature dimension extracted, that is, the dimension of MFCC, which is usually set according to demand.

[0055] We set the number of filter banks M to 26 and the number of cepstral coefficients N to 13.

[0056] The spectral information of the audio signal is extracted through MFCC. The input audio has a duration of 3 seconds, a sampling rate of 16,000 Hz, a frame length of 25 ms, and a frame shift of 10 ms. The audio will be divided into N frames:

[0057]

[0058] Each frame contains 400 sampling points, and the frame shift is 160 sampling points. Each frame is windowed (Hamming window) to reduce edge effects. Then, each frame is FFTed to convert the time domain signal into a frequency domain signal. The FFT result is passed through the Mel filter bank to obtain the output energy of each Mel filter (S m ), take the logarithm of the output energy of each Mel filter Finally, after discrete cosine transform (DCT), we get 13-dimensional MFCC coefficients (i.e., cepstral coefficients).

[0059] Secondly, the Delta method is used to capture the dynamic information of the audio signal changing over time: the dynamic information is obtained by calculating the first-order difference of MFCC. The calculation formula is as follows:

[0060]

[0061] In formula (2), t represents the time position of the current frame, and N is the length of the difference window. We set the length of the difference window to 2. The dynamic feature is also 13-dimensional, and the duration remains the same.

[0062] MFCC(t+n) and MFCC(tn) represent the offset of the time frame and are used to calculate the first-order difference (dynamic features).

[0063] MFCC(t+n): represents the MFCC coefficients of the nth frame after the current frame t.

[0064] MFCC(tn): represents the MFCC coefficients of the nth frame before the current frame t.

[0065] The Delta method captures the dynamic information of the audio signal changing over time through these inter-frame differences.

[0066] Then the LPCC method is used to capture the cepstral features based on linear prediction analysis: In audio signal processing, Linear Prediction Cepstral Coefficient (LPCC) is a method mainly used to extract speech features. It processes the signal and extracts its features based on linear prediction analysis. The calculation formula is as follows:

[0067]

[0068] In formula (3):

[0069] LPCC[m]: represents the mth linear prediction cepstral coefficient, which is one of the final extracted audio features.

[0070] a m: represents the mth prediction coefficient in the linear prediction analysis, reflecting the weight of the audio signal in the prediction process.

[0071] ∑: The symbol represents the summation of k, from 1 to m-1, which is used to calculate the contribution of all previous prediction coefficients to the current feature.

[0072] m: represents the order of the current cepstral coefficient, which is the feature dimension we want to calculate.

[0073] k: is an auxiliary variable used to traverse all prediction coefficients in order to integrate previous results when calculating LPCC[m].

[0074] The audio after data enhancement is pre-emphasized with a pre-emphasis coefficient of 0.95. The audio frame length is still set to 25ms, the frame shift is 10ms, and the audio is divided into 298 frames. Each frame is windowed (Hamming window). For each frame, we use the signal after the window to calculate the autocorrelation function. Through the autocorrelation function, the Durand-Koersner (DK) algorithm is used to obtain the LPC coefficient a k Here k = 1, 2, ..., m. We set m = 12, the initial conditions are LPCC[0] = 1 and LPCC[1] = -a1, and then calculate LPCC[2] to LPCC

[12] in sequence to obtain 12-dimensional linear prediction cepstral coefficients (LPCC features).

[0075] We concatenate the above three features to obtain the mixed feature Feature-Mix:

[0076] Feature-Mix = [MFCC(n), Δ MFCC (t), LPCC(m)]

[0077] The concatenated matrix contains a combination of three different features. The total feature matrix size = (13+13+12)×298 = 38×298, and the dimension of the final mixed feature is (B, 38, 298).

[0078] These features describe the audio signal from different perspectives (frequency domain, time variation, and linear prediction), which makes it possible to capture the rich information of bird sounds in the audio even in complex environments.

[0079] (4) Lightweight voiceprint classification network.

[0080] The lightweight voiceprint classification network used in the present invention is based on the TDNN network structure. The backbone network of the TDNN network extracts features from time series through a series of time-delayed convolutions (standard convolutions). Its structure is simple and suitable for processing time series signal data. However, with the application of the model in complex tasks, TDNN has certain limitations in capturing global features, reducing computational costs, and improving feature expression capabilities. This method adjusts the structure of the backbone network of TDNN, significantly improving the feature extraction capability of the network while reducing computational complexity.

[0081] Similar to the TDNN network, the structure of the lightweight voiceprint classification network proposed in the present invention is also mainly composed of a backbone network, a pooling layer, a linear fully connected layer and a classification layer. The difference is that the backbone network introduces deep separable convolution, dense connection method and channel attention mechanism module, among which the backbone network consists of three convolutional layers.

[0082] Specific improvements include the introduction of deep separable convolution, which decomposes the standard convolution in the TDNN network into deep convolution and point-by-point convolution, which greatly reduces the number of network parameters and computation, making the model more lightweight; at the same time, the outputs of different convolutional layers are concatenated by dense connections, effectively enhancing the network's ability to capture multi-scale features. In addition, in order to further improve the model's feature expression capabilities, a channel attention mechanism module is added, which significantly enhances the network's attention mechanism by dynamically adjusting the feature weights of different channels.

[0083] (I) The following is a more detailed explanation of depthwise separable convolution and dense connections:

[0084] In order to reduce the amount of computation of the model, this method introduces depthwise separable convolution based on TDNN. Standard convolution performs convolution operations on all input channels at each layer, while depthwise separable convolution divides this process into two steps: first, each channel is convolved independently (depthwise convolution), and then each channel is linearly combined through pointwise convolution (1×1 convolution). This not only reduces the computational complexity, but also maintains the effectiveness of feature extraction.

[0085] In addition, this method adopts a dense connection design, that is, the output features of the previous layer are concatenated with the output of the current layer. In this way, the model can reuse features from different layers, thereby improving the ability to capture multi-scale features and reducing the problem of information loss. In specific implementation, the output of the second layer of convolution is concatenated with the output of the first layer and then passed to the subsequent convolution layer. The dense connection design enables the model to better integrate features at different levels and improve classification accuracy.

[0086] To accommodate the convolution operation, the input dimension needs to be converted to (B, 298, 38), where B is the batch size.

[0087] The first layer of deep convolution performs convolution operations on each input channel (38 channels in total) separately to keep the channels independent. (38 channels, each channel has an independent 3×1 convolution kernel convolved along the frame number: that is, the time dimension. Standard convolution uses, for example, a 3×3 convolution kernel covering three channels for convolution movement, so it will generate more parameters and calculations) Assume the convolution kernel size is 3×1, the stride is 1, and the padding is 1.

[0088] For each time step t:

[0089]

[0090] is the depth convolution output of the ith channel at time step t, is the convolution kernel of the ith channel, K=3 is the size of the convolution kernel. The output dimension after depthwise convolution is (B, 298, 38).

[0091] X (i) is the input of the i-th channel, and the input feature is the mixed feature obtained in step (3).

[0092] Point-by-point convolution fuses the channels of each time step through 1×1 convolution, which only focuses on the input channels and does not change the time and space dimensions. The 38 input channels are mapped to 64 channels. The formula is:

[0093] Y p (t) = W p *Y d (t),W p ∈R 64×38×1

[0094] Y p (t) is the output generated by the point-by-point convolution operation, which represents the fused features of all output channels (64 channels) at time step t.

[0095] Y d (t) is a summary of the outputs of all i, i.e. Represents the set of all channels.

[0096] W pis the weight of the point-by-point convolution, with a dimension of 64×38×1 (the weight dimension of the point-by-point convolution is 64×38×1, which means that we have 64 convolution kernels, each of which is 1×1 in size. Each convolution kernel will focus on 38 channels of the input at the same time, weight and fuse them to generate a new channel. After batch normalization of the output of the first layer of convolution, the output is still (B, 298, 64).

[0097] The second layer of convolution uses the same depthwise convolution, and then uses point-by-point convolution to combine the output of the depthwise convolution into 128 output channels. The output is then batch normalized, and the final output is (B, 298, 128).

[0098] Dense connections concatenate the output of each layer with the output of the previous layers. This helps to integrate features at multiple scales, prevent information loss, and enhance the expressiveness of the model. The formula is:

[0099]

[0100] It is the input of the nth convolution layer, which contains the output of the previous n-1 layers.

[0101] Before the third layer of convolution, the output of the first layer and the output of the second layer are concatenated. So the input channel is (B, 298, 192), the same depth convolution is performed on the input, and then the output of the depth convolution is combined into 256 output channels using point-by-point convolution. The output is then batch normalized, and the final output is (B, 298, 256).

[0102] The original TDNN has 2.3 million convolutional layer parameters and 10.31 trillion FLOPs of computation. The improved network has only 60,000 convolutional layer parameters after using deep separable convolution and dense connection, which is a reduction of about 97.4%. The computation is 30.60 million FLOPs, which is a reduction of about 99.7%.

[0103] (II) Setting up the channel attention module:

[0104] By modeling the global features of each channel, the importance of different channels can be dynamically adjusted, useful features can be strengthened, and useless features can be suppressed.

[0105] After passing through the depthwise separable convolution, the output feature shape of the third layer is (B, 298, 256). We adjust the feature shape to (B, 256, 298) and input it into the channel attention mechanism module for processing.

[0106] First, the input features are converted into a global feature vector.

[0107]

[0108] y: Output global feature vector, shape (B, 256), representing the global information of each channel.

[0109] B: Batch Size, which indicates the number of samples processed simultaneously.

[0110] H: The height of the feature map, which is 298 in this step.

[0111] z c [j,i]: An element of the input feature map, representing the eigenvalue of the cth channel at position (j,i), where c ranges from 1 to 256.

[0112] i: The width of the feature map, which is operated from 1 to 298 in this step.

[0113] c: channel index, representing different channels in the feature map, ranging from 1 to 256.

[0114] By averaging the spatial dimensions (H and W) of the feature map, the output result is a feature vector with 256 dimensions. The main function of this step is to extract the global information of each channel, reduce the dimension of the data, and retain the global context of the feature.

[0115] Then perform feature assignment:

[0116] z = σ(W2·ReLU(W1·y))

[0117] σ represents the sigmoid activation function, which maps the output to the (0,1) interval, thus enhancing the learnability of the network.

[0118] z: Output feature assignment, shape is (B, 256), used to recalibrate the channels of the input feature map, the value is between 0 and 1.

[0119] W1: weight matrix, shape (256, 128), used to map the global feature vector y to an intermediate 128-dimensional feature space.

[0120] W2: weight matrix, shape (128, 256), used to map the 128-dimensional feature space back to 256 dimensions. The learning of these two weight matrices helps the model adaptively adjust the importance of each channel.

[0121] ReLU: Linear rectifier unit activation function, responsible for nonlinear transformation of the features after W1 mapping.

[0122] Define two weight vectors (W1 and W2), whose dimensions are (256, 128) and (128, 256) respectively. The global feature vector y is multiplied by the weight W1, then processed by the ReLU activation function, and then multiplied by the weight W2 to finally output z. The key to this process is to readjust the importance of each channel through the learned weights, so that the network can adaptively enhance or suppress the information of each channel, thereby improving the performance of the model.

[0123] Finally, recalibrate:

[0124]

[0125] x: Input feature map, shape is (B, 256, 298), each channel contains features in height and width.

[0126] z: feature assignment

[0127] After the recalibration, the output shape of the feature map is (B, 256, 298), where the features of each channel are enhanced or suppressed accordingly.

[0128] In the recalibration phase, we use z calculated in the previous step to weight the input feature map to achieve adaptive recalibration of the feature map. Specifically, by applying z to each channel of the input feature map, the model can enhance certain features as needed and suppress unimportant features. The main purpose of this step is to use the channel attention mechanism to increase the network's sensitivity to key features, thereby ultimately improving the model's expressiveness and effectiveness.

[0129] (5) Classify bird sounds.

[0130] Through MFCC, Delta and LPCC methods, key feature matrices containing spectrum, dynamic information and linear prediction features are extracted from audio signals. The extracted feature matrices are concatenated and input into the lightweight voiceprint classification network as mixed features. The network model can automatically capture important time-dependent features in audio signals. First, the network extracts and processes features of different time frames layer by layer through a series of deep separable convolutions, and effectively reuses multi-scale features in combination with dense connections. Subsequently, the model's selectivity for important features is further enhanced through the channel attention mechanism. Finally, the features are compressed and integrated through statistical pooling, and then converted to the embedding space through the linear layer, and finally the category prediction results are output through the classification layer.

[0131] In summary, we have greatly reduced the number of model parameters and computational complexity through deep separable convolution. The design of dense connection and channel attention mechanism has greatly improved the robustness of the model (feature extraction capability). Experimental results show that we have applied it to classify 35 bird species related to bird damage in power grids. While ensuring ultra-lightweight and high efficiency, the classification accuracy has been improved by 1%.

[0132] The results show that this plan is feasible.

[0133] The entire classification process significantly reduces the amount of calculation through a lightweight network structure while ensuring the accuracy of audio classification. The classification results will be displayed in the system output so that users can view the analysis results of the audio signal in real time.

[0134] The invention has high classification accuracy, wide application range, strong anti-noise interference ability, lightweight network model design, fast calculation speed, and meets the needs of a bird audio classification model with low resource consumption and high robustness. This method can significantly improve the automation level of bird audio recognition and reduce manual intervention. It has a wide range of practical application value in the field of bird voiceprint recognition under complex backgrounds.

[0135] It should be noted that the above embodiments are only used to illustrate the working principle of the present invention and are not used to limit the protection scope thereof. Those skilled in the art should understand that any improvement, equivalent replacement, or change made under the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines, characterized in that: include: The obtained bird singing audio samples in the transmission line environment are preprocessed, and then the MFCC feature extraction method, Delta method and LPCC feature extraction method are used to obtain the Mel-frequency cepstral coefficients corresponding to the audio signal, the dynamic information changing over time and the linear prediction cepstral coefficients. Then the above three features are spliced ​​to obtain the spliced ​​mixed features; finally, the mixed features are input into the constructed lightweight voiceprint classification network model to output the classification results.

2. The lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines according to claim 1 is characterized in that: The bird singing audio samples are selected from a data set of high-risk bird singing on power transmission lines that has been collected, or are collected in real time. When real-time collection of bird singing audio is adopted, the presence of bird singing is determined by setting a device wake-up threshold. When bird singing is detected, the device starts timed long audio recording, and voice activity detection is used to remove silent segments from the recorded audio segments. The audio is then normalized and filled with zeros to form a unified shape, so as to finally obtain an audio sample with bird singing.

3. The lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines according to claim 1 is characterized in that: During the model training phase, the collected data set of high-risk birds on power transmission lines is used to perform one or more of the following generalization enhancement processes: random time offset, random superposition of background noise, and linear mixing of two different audio samples to obtain training set samples.

4. The lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines according to claim 3 is characterized in that: The following generalization enhancement processing is performed on the bird song audio sample: (1) Superimpose Gaussian noise on the bird song audio signal to generate noisy audio samples; (2) randomly shifting the noisy frequency signal obtained in step (1) to obtain shifted noisy frequency samples; (3) Use the Mixup method to randomly select two bird song audio samples and linearly mix them to obtain a new audio sample; The original bird song audio samples and the audio samples obtained in steps (1) to (3) together constitute the training set samples.

5. The lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines according to claim 1 is characterized in that: The mixed feature Feature-Mix is ​​composed as follows: Feature-Mix=[MFCC(n), ΔMFCC(t), LPCC(m)] Wherein: MFCC(n) represents the nth Mel-frequency cepstral coefficient obtained by the MFCC feature extraction method, n=1,2,…,N, N represents the total feature dimension extracted; ΔMFCC(t) represents the dynamic information of the audio signal changing with time at time t obtained by the Delta method; LPCC(m) represents the mth linear prediction cepstral coefficient obtained by the LPCC feature extraction method, and m represents the order of the current cepstral coefficient.

6. The lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines according to claim 1 is characterized in that: The lightweight voiceprint classification network model adopts an improved TDNN network structure, which mainly includes a backbone network, a pooling layer, a linear fully connected layer and a classification layer. In the backbone network, the features of different time frames are extracted and processed layer by layer through multiple deeply separable convolutions, and the outputs of different convolutional layers are feature spliced ​​through dense connections. Finally, the feature weights of different channels are dynamically adjusted through the channel attention mechanism to effectively extract the global features of the audio.

7. The lightweight and highly robust method for identifying high-risk bird voiceprints on power transmission lines according to claim 6 is characterized in that: The depth-wise separable convolution includes depth-wise convolution and point-wise convolution; In each layer of the backbone network of the lightweight voiceprint classification network model, each channel is independently deep convolved, and then each channel is linearly combined through point-by-point convolution. Finally, the point-by-point convolution output is batch normalized to obtain the output features of this layer. At the same time, a dense connection design is adopted, that is, the output features of the previous layer are concatenated with the output features of the current layer, and then passed to the subsequent convolutional layer as the input of the next layer; Finally, a channel attention mechanism module is added: the output features of each channel in the last layer are recalibrated to obtain the final output of the backbone network.

Images