Bird sound recognition method based on multi-model integration

Abstract

The invention discloses a bird sound recognition method based on multi-model integration. The method comprises the following steps: (1) obtaining a bird sound signal and preprocessing data; (2) extracting key features from the bird sound data by combining Mel-frequency cepstrum coefficient, short-time Fourier transform and decibel conversion feature extraction methods; (3) a plurality of improved pre-training convolutional neural network models are used for training; each convolutional neural network model has different structures or parameter settings so as to capture different features of bird sounds; (4) designing an integration module by using an exponential smoothing method and a weighted smoothing method, and integrating a plurality of trained convolutional neural network models; output results of all the models are fused; (5) carrying out comprehensive performance evaluation on the integrated model; according to the method, the overall accuracy of bird sound recognition is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of bird sound recognition, and in particular to a bird sound recognition method based on multi-model integration. Background Art

[0002] In the field of bird acoustic monitoring, analyzing bird calls is an important means to understand their behavior, health status and ecological environment changes, and is widely used in ecological research, protected area monitoring and bird behavior research. At present, technologies such as feature fusion and deep learning are gradually being applied to bird sound recognition tasks, such as multidimensional feature fusion algorithms and convolutional neural networks. Although the feature fusion method enhances the data representation ability in feature extraction, it may also introduce more noise and increase risks, resulting in a decrease in recognition accuracy. In the prior art, although the feature fusion method improves the data representation ability, it may introduce additional noise, resulting in a decrease in recognition accuracy. At the same time, a single deep learning model has limitations in processing the diversity and complexity of bird sounds, and it is difficult to fully capture the rich features of bird sounds, thus affecting the recognition performance. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a bird sound recognition method based on multi-model integration, which solves the problem of low recognition accuracy in existing bird sound recognition technology by introducing a multi-model integration strategy.

[0004] Technical solution: The bird sound recognition method based on multi-model integration described in the present invention comprises the following steps:

[0005] (1) Obtain bird sound signals and preprocess the data;

[0006] (2) Combine the Mel-frequency cepstral coefficients, short-time Fourier transform and decibel conversion feature extraction methods to extract key features from bird sound data;

[0007] (3) using multiple improved pre-trained convolutional neural network models for training respectively; each convolutional neural network model has a different structure or parameter setting to capture different characteristics of bird sounds;

[0008] (4) Use exponential smoothing method and weighted smoothing method to design an integration module to integrate multiple trained convolutional neural network models; fuse the output results of each model;

[0009] (5) Conduct a comprehensive performance evaluation on the integrated model.

[0010] Furthermore, in step (1), the preprocessing includes denoising, normalization and segmentation of the audio signal; the preprocessed data is divided into audio segments of appropriate length.

[0011] Furthermore, in step (2), the processed bird sound signal is transformed, and the data form is converted from a vector to a complex matrix, wherein the complex matrix represents the signal strength at different times and frequencies; the complex matrix is ​​convolved through a set of Mel filters; the logarithm of the Mel filter output is extracted, and the output is stacked together along the time axis to form a two-dimensional matrix; wherein the horizontal axis of the matrix represents time, the vertical axis represents the frequency of the Mel filter, and each element represents the energy value at the corresponding time and frequency; the two-dimensional matrix is ​​post-processed using DB conversion to convert the frequency value into a decibel value.

[0012] Furthermore, step (3) is specifically as follows: five improved pre-trained convolutional neural network models are used for training respectively, including: Tf_efficientnetv2_s_in21k model, Seresnext50_32x4d model with SE module, Cspdarknet53 model with CSPN structure, Eca_nfnet_l0 model with ECA module, and Resnet34 model with residual connection structure; wherein, before training these five pre-trained convolutional neural network models, the models are adjusted and improved.

[0013] Furthermore, the model was adjusted and improved as follows: first, the original bird sound audio was converted into a spectrogram through Mel spectrum conversion and DB conversion; the Mixup operation was introduced to increase data diversity; and then a learnable GeM pooling layer was added after the pre-trained model to capture and extract the features of the bird sound audio; the output of GeM was obtained by performing power average calculation on the elements on each channel of the feature map.

[0014] Furthermore, a Mixup operation was introduced to increase data diversity as follows: Mixup was applied inside the bird sound audio, and a single bird sound audio was divided into multiple sub-segments of equal length during the data preprocessing stage, and then these sub-segments were linearly interpolated; then, two different bird sound audios were randomly selected to perform the same Mixup operation; wherein the linear interpolation objects were sub-segments from two different bird sound audios.

[0015] Furthermore, step (4) is specifically as follows: the prediction module receives the outputs from the five bird sound recognition neural network models and integrates them into a set of data, which is then input into the exponential smoother; then the weighted averager receives the output of the exponential smoother as input and assigns weights according to the performance of the five models on the test set; finally, the output of the weighted averager is subjected to Softmax conversion to obtain the final recognition result.

[0016] The bird sound recognition system based on multi-model integration described in the present invention comprises:

[0017] Preprocessing module: used to obtain bird sound signals and preprocess the data, including denoising, normalization and segmentation of audio signals to ensure data quality and consistency. The preprocessed data will be divided into audio segments of appropriate length to ensure the validity of the data.

[0018] Extraction module: used to extract key features from bird sound data by combining Mel frequency cepstral coefficients, short-time Fourier transform and decibel conversion feature extraction methods;

[0019] Training module: used to use multiple improved pre-trained convolutional neural network models for training respectively, each convolutional neural network model has a different structure or parameter setting to capture different characteristics of bird sounds.

[0020] Integration module: used to design an integration module using exponential smoothing method and weighted smoothing method to integrate multiple trained convolutional neural network models; and to fuse the output results of each model;

[0021] Evaluation module: used to conduct comprehensive performance evaluation on the integrated model.

[0022] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, it implements any one of the bird sound recognition methods based on multi-model integration.

[0023] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the bird sound recognition methods based on multi-model integration.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: By adopting a strategy based on multi-model integration, the accuracy of bird sound recognition is significantly improved. Compared with the single model or feature fusion method in the prior art, the multi-model integration strategy of the present invention can make full use of the advantages of multiple models, integrate their recognition results, and overcome the limitations of a single model in dealing with the complexity and diversity of bird sounds. It has been verified that the multi-model integration method adopted by the present invention can effectively reduce errors in the recognition process, improve the stability and reliability of recognition, and thus significantly improve the overall accuracy of bird sound recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a framework diagram of the bird sound recognition method of the present invention;

[0026] Figure 2 is a feature extraction flow chart of the present invention;

[0027] Figure 3It is a structural diagram of a neural network model of the present invention;

[0028] Figure 4 is a schematic diagram of the multi-model integration strategy of the present invention;

[0029] Figure 5 This is a graph showing the change in model accuracy under different levels of noise of the present invention;

[0030] Figure 6 It is a confusion matrix diagram under the multi-model strategy of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0032] like Figure 1 As shown, an embodiment of the present invention provides a bird sound recognition method based on multi-model integration, comprising the following steps:

[0033] S1 Data preprocessing; specifically as follows: Convert the input audio sampling rate and adjust it to 32kHz. Then load the converted audio and process its length: if the length is less than 5 seconds, fill it with zeros; if the length is greater than 5 seconds, trim it to ensure that the trimmed audio length is not less than 5 seconds, and then fill it with zeros. After trimming, introduce background noise data between the audio to achieve data enhancement.

[0034] S2 extracts the characteristics of bird sounds; the details are as follows: Figure 2 As shown, the processed bird sound signal is transformed by formula (1), and the data form is converted from a vector to a complex matrix, which represents the signal strength at different times and frequencies.

[0035]

[0036] Among them, X(t,f) represents the result of short-time Fourier transform, which is a complex matrix; x(t) represents the input bird sound signal, which is a vector; ω(τ) represents the window function, and the window function used is the Hamming window; N represents the window size, which is set to 1024.

[0037] The real part of X(t,f) represents the component of positive frequency, and the imaginary part represents the component of negative frequency. In order to remove the phase information of the complex number and obtain the signal energy at the corresponding time and frequency, the real part and the imaginary part are squared and added to obtain the power spectrum matrix.

[0038] The power spectrum matrix is ​​convolved through a set of Mel filters to simulate the human auditory system's perception of the sound spectrum. The calculation formula of the Mel filter is:

[0039]

[0040] Among them, H m (f) represents the response of the mth Mel filter to frequency f; m represents the number of filters, which is set to 64; f m-1 、f m+1 They represent the lowest frequency and the highest frequency in the filter frequency range respectively, and their values ​​are the center frequencies f of two adjacent filters. m However, when calculating the Mel filter response, the frequency range is usually limited to a reasonable range. In the present invention, the minimum frequency is set to 50 and the maximum frequency is set to 14000; f m Represents the center frequency of the mth Mel filter, and its value is determined by the Mel frequency conversion formula:

[0041]

[0042] Take the logarithm of the Mel filter output to enhance the features of the low-frequency part. Stack these outputs together along the time axis to form a two-dimensional matrix. The horizontal axis of the matrix represents time, and the vertical axis represents the frequency of the Mel filter. Each element represents the energy value at the corresponding time and frequency.

[0043] The two-dimensional matrix is ​​post-processed using the DB conversion to convert the frequency values ​​into decibel values. The calculation formula for this conversion is:

[0044]

[0045] Where A is the power spectrum of the input signal, A ref is the reference power, A ref Set to 1.0.

[0046] S3 uses multiple improved pre-trained convolutional neural network models for training; each convolutional neural network model has a different structure or parameter setting to capture different characteristics of bird sounds; the details are as follows: First, in order to further extract the characteristics of bird sounds in the spectrum graph, this paper selects and uses five pre-trained models that perform well in the field of image processing. These pre-trained models have different advantages in the bird sound recognition task. The details are as follows:

[0047] The Tf_efficientnetv2_s_in21k model is pre-trained on a large-scale image dataset and has learned a rich representation of spectral features, providing high accuracy. The Seresnext50_32x4d model introduces the SE module to enhance the ability to capture key spectral features in bird sounds, effectively improving the recognition performance of complex patterns. The Cspdarknet53 model introduces the CSPN structure, which can better process the characteristics of bird sounds in different time periods. It is suitable for capturing the temporal nature of bird sounds and improving the perception ability of the model. The Eca_nfnet_l0 model introduces the ECA module, which helps to understand the global contextual information in bird sounds and focus on important frequencies in bird sounds. The Resnet34 model uses a residual connection structure to solve the gradient vanishing problem in deep neural networks, making the network easier to train and able to effectively capture the hierarchical structure of bird sounds. Before training these five pre-trained convolutional neural network models, the models are first adjusted and improved. The improved neural network model structure suitable for the bird sound recognition task is as follows: Figure 3 shown.

[0048] Then, the original bird sound audio is converted into a spectrogram through Mel spectrum conversion and DB conversion. In order to improve the generalization ability of the model, the Mixup operation is introduced to increase data diversity. First, Mixup is applied inside the bird sound audio, and a single bird sound audio is divided into multiple sub-segments of equal length in the data preprocessing stage, and then these sub-segments are linearly interpolated. Next, two different bird sound audios are randomly selected to perform the same Mixup operation, but this time the linear interpolation objects are sub-segments from two different bird sound audios. The calculation formula of Mixup is as follows:

[0049] X mix =ρX i +(1-ρ)X j (5)

[0050] Y mix =ρY i +(1-ρ)Y j (6)

[0051] weight mix =ρweight i +(1-ρ)weight j (7)

[0052] Among them, X mix Represents the input data X i , X j Mixture of Y mix Indicates the label Y corresponding to the input data i , Y j Mixture of Y mixFor mixed data X mix When Y i , Y j When it is a label of the same type of bird sound, Y mix is the label of the bird sound of this category, corresponding to data X i , X j Mixed data X mix , increasing data diversity. i , Y j When Y is a label for different types of bird sounds, mix The labels that are different from these two types of bird sounds correspond to the mixed data X mix , which helps the model learn the commonalities and differences between the sounds of different bird species. mix Represents sample weight i , weight j A mixture of mixed data X mix The value of the sample weight is directly related to the quality of the bird sound audio, which is divided into five levels, from 1 to 5. Among them, 1 represents poor audio quality and unclear content, while 5 represents good audio quality and high content clarity. The weight value is calculated by dividing the quality value of the current bird sound audio by the maximum value of the bird sound audio quality value in the training set, and its value range is between 0.1 and 1.0. ρ represents the mixing coefficient of Mixup, and the present invention uses Beta (1,1) distribution to assign ρ.

[0053] There are important spectral features in bird sound audio, and the adaptability of GeM pooling can help the network better capture these significant features. Therefore, the present invention adds a learnable GeM pooling layer after the pre-training model to better capture and extract the features of bird sound audio. The output of GeM is obtained by performing power average calculation on the elements on each channel of the feature map. The specific calculation formula is as follows:

[0054]

[0055] Where x is the input feature map, C is the number of channels, and p is a hyperparameter that controls the shape of the pooling. When p = 1, GeM pooling is equivalent to ordinary average pooling; when p → ∞, GeM pooling approaches maximum pooling, and the initial value of p is set to 3.

[0056] Finally, the extracted features are input into the improved model, and the training is performed and the results are saved.

[0057] S4 is used to design an integrated module using exponential smoothing method and weighted smoothing method to integrate multiple trained convolutional neural network models; the output results of each model are fused as follows: The designed integrated prediction module is mainly divided into two parts: exponential smoother and weighted smoother, and its structure is as follows Figure 4 shown.

[0058] The prediction module receives the outputs from the five bird sound recognition neural network models and integrates them into a set of data, which is then input into the exponential smoother. Bird sound signals are time series signals, and the exponential smoother is a method for processing time series data, which is often used to suppress random fluctuations. Its core principle is to give higher weights to recent observations, gradually reduce the weights of distant observations, and achieve smoothing by continuously updating the average value. The exponential smoothing calculation formula is as follows:

[0059] S t =α·Y t +(1-α)·S t-1 (9)

[0060] Among them, S t is the exponential moving average at time point t, Y t is the observed value at time point t, S t-1 is the exponential moving average at time point t-1, and α is a smoothing parameter, which takes a value between 0 and 1. In the present invention, the value of α is set to 0.5.

[0061] The weighted averager receives the output of the exponential smoother as input and assigns weights to the five models based on their performance on the test set. Models with superior performance receive higher weights, while models with poor performance receive lower weights. This process helps reduce the noise that may be introduced by a single model and increases the trust in the collaborative predictions of multiple models. The calculation formula is as follows:

[0062]

[0063] Among them, n represents the number of models, which is set to 5; xi represents the model prediction value; wi represents the model weight.

[0064] The final recognition result can be obtained by performing Softmax conversion on the output of the weighted averager.

[0065] S5 conducts a comprehensive performance evaluation on the integrated model, optimizes the model based on the evaluation results, and adjusts the model parameters and integration strategy to further improve the recognition performance.

[0066] The present invention selects 20 common Chinese bird species in the Beijing Birdsdata bird sound database as experimental objects, and uses accuracy as an evaluation index for its own model and other comparison models.

[0067] In order to verify the effectiveness of the Mixup and GeM pooling modules proposed in the present invention, the present invention uses the five models of Tf_efficientnetv2_s_in21k, Seresnext50_32x4d, Cspdarknet53, Eca_nfnet_l0 and Resnet34 as benchmark models, and compares the Mixup module plus five benchmark models and the Mixup module plus the GeM pooling module plus five benchmark models, respectively. The results are shown in Table 1.

[0068] Table 1 Ablation experiment results

[0069]

[0070]

[0071] After applying the Mixup module to the five benchmark models, the accuracy of these models increased by an average of 1.682%. This shows that Mixup effectively improves the diversity of training data by introducing data enhancement and generating mixed samples, so that the model can better adapt to the differences in bird sounds. In order to further verify the anti-interference ability of the Mixup module to noise, the present invention conducts noise experiments by adding different degrees of Gaussian white noise to the test set. The changes in the accuracy of each model under different noise levels are shown in Figure 2. Figure 5 shown.

[0072] As the noise level increases, the recognition accuracy of all 8 methods decreases. Compared with the CRNN method and the DSRN+DilatedSAM+BiLSTM method, the method of introducing the Mixup module at the front end of the model adopted by the present invention can effectively reduce the impact of noise. Under different degrees of noise background, the average fluctuation of the recognition accuracy of the pre-trained model introducing the Mixup module is small. This advantage is also reflected in the integrated model finally adopted by the present invention. As can be seen from the blue curve in the figure, the integrated model is least affected by noise, which is due to the fact that the Mixup method introduces the effect of data enhancement by mixing the input data during the training process. This helps the model to better generalize to unseen data samples, thereby enhancing the model's adaptability to noise and changes.

[0073] As shown in Table 3, after the introduction of the GeM module on the basis of the Mixup module, the accuracy of the model increased by an average of 1.152%. This shows that the GeM module further optimizes the feature extraction process by adaptively adjusting the weights of the feature map. The GeM module implements adaptive pooling in the spatial dimension and can effectively capture important information in the input feature map. This adaptability improves the performance of the model and makes it better adapt to the spatial distribution characteristics of different bird sounds.

[0074] In order to verify the effectiveness of the integrated method proposed in the present invention, the recognition effects of five single models, the best recognition effect in the integration of two models, the best recognition effect in the integration of three models, the best recognition effect in the integration of four models, and the integrated effect of five models are compared. The results are shown in Table 2.

[0075] Table 2 Experimental results comparing single model and ensemble method

[0076]

[0077]

[0078] The proposed method of integrating five models with different weight values ​​has a recognition accuracy of 97.93%, which is about 2.7% higher than that of a single model (Resnet34+Mixup+GeM); about 1.94% higher than that of the best performance in the integration of two models; about 1.35% higher than that of the best performance in the integration of three models; about 0.81% higher than that of the best performance in the integration of four models; and 0.89% higher than that of the average integration method. The introduction of the exponential smoother has achieved remarkable results in reducing the noise and fluctuation of the output of a single model, thereby significantly improving the stability of the overall prediction. This introduction not only plays a role in reducing the excessive sensitivity of the model to specific samples, but also successfully eliminates the unstable factors of the prediction results. On the other hand, the introduction of the weighted averager helps to reduce the error generated by a single model. Through reasonable weight distribution, the method effectively integrates each model, and this integration provides a more reliable prediction basis for the practical application of the model, thereby improving the recognition accuracy as a whole. During the experiment, the average inference time of a single test audio is as follows: Model ① is 18ms, Model ② is 14ms, Model ③ is 15ms, Model ④ is 20ms, and Model ⑤ is 25ms. The average inference time of the integrated method is 30ms. Based on this observation, although the integrated method improves the accuracy of bird sound recognition, it does not significantly increase the computational complexity.

[0079] In addition, the present invention also compares other bird sound recognition methods under the same data set conditions, and the comparison results are shown in Table 3.

[0080] Table 3 Comparative experimental results of other methods

[0081]

[0082]

[0083] Obviously, the method of the present invention has advantages in recognition accuracy: compared with the 3log-Mel+DSRN+DilatedSAM+BiLSTM method, it has increased by 1.28%; compared with the Bi-LSTM-DenseNet method, it has increased by 5.45%; compared with the BTNN method, it has increased by 1.04%; compared with the commonly used integrated strategy Boosting, it has increased by 3.72%. The bird sound recognition method of the present invention introduces Mixup data enhancement, which effectively enhances the noise resistance of a single model, thereby improving the noise resistance of the integrated method. By introducing generalized mean pooling to optimize the feature extraction process, the prediction results of multiple models are integrated using exponential smoothers and weighted averagers, which effectively reduces the prediction error of a single model. These multi-faceted noise processing methods significantly improve the accuracy of bird sound recognition and achieve better recognition results than other methods.

[0084] The confusion matrix of the bird sound recognition method proposed in the present invention is as follows: Figure 6 As shown. When dealing with bird sound recognition with a small number of samples, this method shows a relatively high accuracy rate, which is mainly due to the effective feature extraction method adopted. Through the feature extraction of this method, the key features of bird sounds can be accurately captured, and the accurate recognition of specific bird sounds can be achieved. Taking the Eurasian buzzard (marked as 'EN' in the figure) as an example, only 2 samples were misidentified in the test of 29 audio samples. Taking the bone crested chicken (marked as 'CTC' in the figure) as an example, only 1 sample was misidentified in the 46 test audios. In addition, even when the number of samples is very small, this method can still successfully identify some bird sounds. For example, for the gray partridge (marked as 'GWP' in the figure), a bird sound with only 3 samples in the test audio, it can still accurately identify one of the samples. This result once again verifies the generalization ability of this method.

[0085] The embodiment of the present invention further provides a bird sound recognition system based on multi-model integration, comprising:

[0086] Preprocessing module: used to obtain bird sound signals and preprocess the data, including denoising, normalization and segmentation of audio signals to ensure data quality and consistency. The preprocessed data will be divided into audio segments of appropriate length to ensure the validity of the data.

[0087] Extraction module: used to extract key features from bird sound data by combining Mel frequency cepstral coefficients, short-time Fourier transform and decibel conversion feature extraction methods;

[0088] Training module: used to use multiple improved pre-trained convolutional neural network models for training respectively, each convolutional neural network model has a different structure or parameter setting to capture different characteristics of bird sounds.

[0089] Integration module: used to design an integration module using exponential smoothing method and weighted smoothing method to integrate multiple trained convolutional neural network models; and to fuse the output results of each model;

[0090] Evaluation module: used to conduct comprehensive performance evaluation on the integrated model.

[0091] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, the computer program implements any one of the bird sound recognition methods based on multi-model integration.

[0092] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any one of the bird sound recognition methods based on multi-model integration.

Claims

1. A bird sound recognition method based on multi-model integration, characterized in that: The following steps are involved: (1) Obtain bird sound signals and preprocess the data; (2) Combine the Mel-frequency cepstral coefficients, short-time Fourier transform and decibel conversion feature extraction methods to extract key features from bird sound data; (3) using multiple improved pre-trained convolutional neural network models for training respectively; each convolutional neural network model has a different structure or parameter setting to capture different characteristics of bird sounds; (4) Use exponential smoothing method and weighted smoothing method to design an integration module to integrate multiple trained convolutional neural network models; fuse the output results of each model; (5) Conduct a comprehensive performance evaluation on the integrated model.

2. The bird sound recognition method based on multi-model integration according to claim 1 is characterized in that: In step (1), the preprocessing includes denoising, normalization and segmentation of the audio signal; the preprocessed data is divided into audio segments of appropriate length.

3. The bird sound recognition method based on multi-model integration according to claim 1 is characterized in that: In step (2), the processed bird sound signal is transformed, and the data form is converted from a vector to a complex matrix, wherein the complex matrix represents the signal strength at different times and frequencies; the complex matrix is ​​convolved through a set of Mel filters; the logarithm of the Mel filter output is extracted, and the outputs are stacked together along the time axis to form a two-dimensional matrix; wherein the horizontal axis of the matrix represents time, the vertical axis represents the frequency of the Mel filter, and each element represents the energy value at the corresponding time and frequency; the two-dimensional matrix is ​​post-processed using DB conversion to convert the frequency value into a decibel value.

4. The bird sound recognition method based on multi-model integration according to claim 1, characterized in that: Step (3) is as follows: use 5 improved pre-trained convolutional neural network models for training respectively, including: Tf_efficientnetv2_s_in21k model, Seresnext50_32x4d model with SE module, Cspdarknet53 model with CSPN structure, Eca_nfnet_l0 model with ECA module, Resnet34 model with residual connection structure; before training these 5 pre-trained convolutional neural network models, adjust and improve the models.

5. The bird sound recognition method based on multi-model integration according to claim 4 is characterized in that: The specific adjustments and improvements to the model are as follows: First, the original bird sound audio is converted into a spectrogram through Mel spectrum conversion and DB conversion; the Mixup operation is introduced to increase data diversity; and then a learnable GeM pooling layer is added after the pre-trained model to capture and extract the features of the bird sound audio; the output of GeM is obtained by performing power average calculation on the elements on each channel of the feature map.

6. The bird sound recognition method based on multi-model integration according to claim 5, characterized in that: The Mixup operation was introduced to increase data diversity. The details are as follows: Mixup is applied inside the bird sound audio, and a single bird sound audio is divided into multiple sub-segments of equal length during the data preprocessing stage, and then these sub-segments are linearly interpolated; then, two different bird sound audios are randomly selected to perform the same Mixup operation; among which, the linear interpolation objects are sub-segments from two different bird sound audios.

7. The bird sound recognition method based on multi-model integration according to claim 1, characterized in that: Step (4) is as follows: the prediction module receives the outputs from the five bird sound recognition neural network models and integrates them into a set of data, which is then input into the exponential smoother; then the weighted averager receives the output of the exponential smoother as input and assigns weights based on the performance of the five models on the test set; finally, the output of the weighted averager is subjected to Softmax conversion to obtain the final recognition result.

8. A bird sound recognition system based on multi-model integration, characterized in that: include: Preprocessing module: used to obtain bird sound signals and preprocess the data, including denoising, normalization and segmentation of audio signals to ensure data quality and consistency. The preprocessed data will be divided into audio segments of appropriate length to ensure the validity of the data. Extraction module: used to extract key features from bird sound data by combining Mel frequency cepstral coefficients, short-time Fourier transform and decibel conversion feature extraction methods; Training module: used to use multiple improved pre-trained convolutional neural network models for training respectively, each convolutional neural network model has a different structure or parameter setting to capture different characteristics of bird sounds. Integration module: used to design an integration module using exponential smoothing method and weighted smoothing method to integrate multiple trained convolutional neural network models; and to fuse the output results of each model; Evaluation module: used to conduct comprehensive performance evaluation on the integrated model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the method for bird sound recognition based on multi-model integration according to any one of claims 1 to 7 is implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a bird sound recognition method based on multi-model integration according to any one of claims 1 to 7 is implemented.

Images