A bird sound recognition method and device based on attention mechanism

Abstract

The present application discloses a method and device for bird sound recognition based on an attention mechanism, and relates to the technical field of bird sound recognition. In the training stage, the method extracts MFCC features and GFCC features from bird sound data samples respectively, and then performs feature fusion to obtain a fused feature vector for model training to obtain a basic recognition model. In the recognition application stage, the attention mechanism weight coefficient is added to the last fully connected classification layer in the basic recognition model according to the bird distribution statistical data of the application area, and a matching bird sound recognition model is obtained for bird sound recognition in the application area. The present application combines the high accuracy of MFCC features with the strong robustness of GFCC features to avoid the problem of high-frequency bird sound signal loss caused by a single feature. The attention mechanism weight coefficient added in the application stage can reduce the misrecognition of similar bird sound signals in different regions, thereby improving the accuracy of bird sound recognition.

Description

Technical Field

[0001] The present application relates to the technical field of bird sound recognition, and in particular to a method and device for bird sound recognition based on an attention mechanism. Background Art

[0002] my country is rich in bird resources. Birds are friends of humans and also part of the ecological environment. Birds can sensitively perceive small changes in the environment and are the most ideal ecological species for monitoring environmental changes. In the study of bird diversity, singing, as a typical feature, has the advantage of high recognizability and has become an important means of studying bird species diversity.

[0003] At present, most of the research on sound recognition and classification in China focuses on music and speech recognition, and there is a lack of research on animal and bird sound recognition. The key to bird sound recognition lies in the acquisition of sound signals, feature selection and classifier selection under complex conditions in the wild sound environment. The biggest challenge in the acquisition of sound signals is noise interference. In 2013, Chen Shasha studied a short-time spectrum audio enhancement algorithm based on dynamic estimation of noise power spectrum to process noisy bird sound signals, extract pure and effective singing signals as much as possible, and thus increase the accuracy of classification. In 2022, Han Pengfei also proposed a new method for bird sound noise reduction. It can be said that breakthrough progress has been made in sound classification and recognition, from the traditional artificially designed feature extractor to extract features, KNN, SVM and other classifiers for classification and recognition, to the rise of deep learning, especially the application of deep learning methods based on convolutional neural networks has become popular, and end-to-end recognition methods such as LSTM and RNN have promoted the improvement of sound recognition capabilities. Research related to sound recognition is also in a booming trend.

[0004] However, the rapid development of technology has not brought about a qualitative improvement in the recognition of bird sounds under complex conditions in real wild sound environments. This is mainly because in the classification and recognition of bird sounds, most methods are based on MFCC (Mel Frequency Cepstrum Coefficient) features. The Mel filter is designed based on the structural characteristics of the human ear and is insensitive to high-frequency signals. Many species of birds have high-frequency signals, which will lead to the loss of the extracted feature signals. In addition, there are many species of birds, and the differences in the species of some bird calls cannot be distinguished from the sound signals alone, resulting in low accuracy in the recognition of bird sound signals in complex environments. Summary of the invention

[0005] In response to the above problems and technical requirements, this application proposes a bird sound recognition method and device based on an attention mechanism. The technical solution of this application is as follows:

[0006] A bird sound recognition method based on an attention mechanism, the bird sound recognition method comprising:

[0007] Obtain an initial sample data set, the initial sample data set including bird song data samples of several different types of bird samples;

[0008] Extract MFCC features and GFCC features from the bird song data samples of each bird sample in the initial sample data set, and then perform feature fusion to obtain the fused feature vector of the bird sample;

[0009] The fused feature vector of each bird sample is used as input, and the species of the bird sample is used as output. The basic recognition model is obtained by model training based on the CNN network.

[0010] According to the bird distribution statistics of the application area, the attention mechanism weight coefficient is added to the last fully connected classification layer in the basic recognition model to obtain a bird sound recognition model that matches the application area.

[0011] The bird sound recognition model matched to each application area is used to perform bird sound recognition on the bird song data to be identified collected in the application area.

[0012] Its further technical solution is to add attention mechanism weight coefficients to the last fully connected classification layer in the basic recognition model based on the bird distribution statistics in the application area, including:

[0013] According to the frequency of occurrence of various species of birds in the application area indicated by the bird distribution statistics of the application area, a corresponding attention mechanism weight coefficient is added to each species of the last fully connected classification layer in the basic recognition model; the higher the frequency of occurrence of a species of bird in the application area, the greater the attention mechanism weight coefficient added to the corresponding species.

[0014] Its further technical solution is to extract MFCC features and GFCC features respectively and then perform feature fusion, including:

[0015] The extracted MFCC features with a two-dimensional size of t*n and the extracted GFCC features with a two-dimensional size of t*m are fused by feature splicing to obtain a fused feature vector with a two-dimensional size of t*(m+n).

[0016] Its further technical solution is that the fully connected classification layer includes a fully connected layer and a softmax layer, and the bird sound recognition model matched to each application area is used to perform bird sound recognition on the bird song data to be recognized collected in the application area, including:

[0017] Collect the bird song data to be identified in the application area, extract MFCC features and GFCC features from the bird song data to be identified, and then perform feature fusion to obtain the feature vector to be identified;

[0018] The feature vector to be identified is input into the bird sound recognition model that matches the application area. After the fully connected layer in the bird sound recognition model outputs the values ​​of each species, it is weighted calculated according to the weight coefficient of the attention mechanism of each species and then input into the softmax layer. The softmax layer outputs the predicted probability of each species, and the species with the largest predicted probability is used as the species recognition result corresponding to the bird song data to be identified.

[0019] A further technical solution is that the bird distribution statistics in different application areas are different, and the matching bird sound recognition models obtained by correcting the basic recognition model are also different.

[0020] A bird sound recognition device based on an attention mechanism, the bird sound recognition device comprising:

[0021] A sample data acquisition module is used to acquire an initial sample data set, where the initial sample data set includes bird song data samples of several different types of bird samples;

[0022] A feature extraction module is used to extract MFCC features and GFCC features from the bird song data samples of each bird sample in the initial sample data set, and then perform feature fusion to obtain a fused feature vector of the bird sample;

[0023] The model training module is used to take the fused feature vector of each bird sample as input and the species of the bird sample as output, and perform model training based on the CNN network to obtain a basic recognition model;

[0024] A model correction module is used to add an attention mechanism weight coefficient to the last fully connected classification layer in the basic recognition model according to the bird distribution statistics of the application area, so as to obtain a bird sound recognition model that matches the application area;

[0025] The recognition module is used to perform bird sound recognition on the bird song data to be recognized collected in the application area by using the bird sound recognition model matched to each application area.

[0026] Its further technical solution is a model correction module, which is used to add a corresponding attention mechanism weight coefficient for each species in the last fully connected classification layer in the basic recognition model according to the frequency of occurrence of each species of birds in the application area indicated by the bird distribution statistical data of the application area; the higher the frequency of occurrence of a species of bird in the application area, the greater the attention mechanism weight coefficient added to the corresponding species.

[0027] Its further technical solution is that the feature extraction module is used to fuse the extracted MFCC features with a two-dimensional size of t*n and the extracted GFCC features with a two-dimensional size of t*m in a feature splicing manner to obtain a fused feature vector with a two-dimensional size of t*(m+n).

[0028] Its further technical solution is that the fully connected classification layer includes a fully connected layer and a softmax layer;

[0029] The recognition module is used to collect bird song data to be identified in the application area, extract MFCC features and GFCC features from the bird song data to be identified, and then perform feature fusion to obtain a feature vector to be identified; the feature vector to be identified is input into a bird sound recognition model that matches the application area, and after the fully connected layer in the bird sound recognition model outputs the values ​​of each species, a weighted calculation is performed according to the weight coefficient of the attention mechanism of each species and then input into the softmax layer, which outputs the predicted probability of each species, and the species with the largest predicted probability is used as the species recognition result corresponding to the bird song data to be identified.

[0030] A further technical solution is that the bird distribution statistics in different application areas are different, and the matching bird sound recognition models obtained by correcting the basic recognition model are also different.

[0031] The beneficial technical effects of this application are:

[0032] The present application discloses a bird sound recognition method and device based on an attention mechanism, which fuses MFCC features and GFCC features to obtain a fused feature vector, which can combine the high accuracy of MFCC features with the strong robustness of GFCC features, avoid the limitation of missing effective features when recognizing based on a single feature, and make up for the problem of losing some high-frequency bird sound signals caused by the single use of MFCC features in the prior art. In the recognition application stage, according to the bird distribution statistical data in different application areas as the weight factor of the attention mechanism, the confidence weight coefficients of different categories of bird sounds are added to the basic recognition model based on the CNN network, which can effectively avoid the misrecognition problem caused by the similar characteristics of common bird sound signals in different application areas. Both of the above mechanisms can improve the accuracy of bird sound recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a method flow chart of a bird sound recognition method according to an embodiment of the present application.

[0034] Figure 2 It is a schematic diagram of the application of a bird sound recognition method in an example of the present application.

[0035] Figure 31 is a device structure diagram of a bird sound recognition device in one embodiment of the present application. DETAILED DESCRIPTION

[0036] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0037] This application discloses a bird sound recognition method based on attention mechanism, please refer to Figure 1 The bird sound recognition method includes the following steps:

[0038] Step 1: Get the initial sample data set.

[0039] The initial sample data set obtained includes bird song data samples of several different types of bird samples. The ways to obtain bird song data samples include real environment collection and database collection, and in order to ensure the recognition effect, the number of bird song data samples collected in the real environment is not less than the number of bird song data samples collected in the database.

[0040] Step 2: Extract MFCC features and GFCC features from the bird song data samples of each bird sample in the initial sample data set, and then perform feature fusion to obtain a fused feature vector of the bird sample.

[0041] In practical applications, considering that bird song data samples collected in real environments often contain ambient sounds and other noises, it is often necessary to preprocess the sound signals before feature extraction. This process can adopt various existing sound signal preprocessing methods, which will not be described in detail in this embodiment.

[0042] 1. Extract MFCC features from bird song data samples. The extracted MFCC features are two-dimensional feature vectors with a two-dimensional size of t*n, where t and n are parameters. MFCC features organically combine the auditory perception characteristics of the human ear with the generation mechanism of bird song data samples. The filter group is obtained by transforming the spectrum energy spectrum with a triangular filter, then taking the logarithm, and finally performing an inverse discrete cosine transform to obtain the MFCC features. The extraction method includes:

[0043] (1) Divide the bird song data samples into short frames and calculate the periodogram estimate of the power spectrum.

[0044] (2) Pass the power spectrum through a Mel filter bank and sum the energy of each filter.

[0045] (3) Take the logarithm of all filter bank energies.

[0046] (4) Perform discrete cosine transform (DCT) on the energy of the logarithmic filter bank, keep the DCT coefficients, discard the rest, and extract the MFCC features.

[0047] 2. Extract GFCC features from bird song data samples. The extracted GFCC features are two-dimensional feature vectors with a two-dimensional size of t*m, where m is also a parameter. GFCC features are extracted using a Gammatone filter, which is a filter based on the standard cochlear structure. The extraction method includes:

[0048] (1) Bird song data samples are passed through a Gammatone filter bank.

[0049] (2) Take the absolute value of each channel filter response.

[0050] (3) Take the logarithm of the absolute value of the above filter response.

[0051] (4) Perform DCT on the above logarithmic results to reduce the correlation between the features of each dimension and extract the GFCC features.

[0052] After extracting the MFCC features and GFCC features of the bird song data samples respectively, in order to avoid the limitation of missing effective features when recognizing based on a single feature, the high accuracy of the MFCC features and the strong robustness of the GFCC features are combined, and the MFCC features and GFCC features are fused to obtain a fused feature vector.

[0053] In one embodiment, the extracted MFCC features with a two-dimensional size of t*n and the extracted GFCC features with a two-dimensional size of t*m are fused by feature splicing to obtain a fused feature vector with a two-dimensional size of t*(m+n).

[0054] According to the above method, the corresponding fusion feature vector can be extracted from the bird song data sample of each bird sample in the initial sample data set.

[0055] Step 3: Take the fused feature vector of each bird sample as input and the species of the bird sample as output, and perform model training based on the CNN network to obtain a basic recognition model.

[0056] Step 4: Add the attention mechanism weight coefficient to the last fully connected classification layer in the basic recognition model according to the bird distribution statistics of the application area to obtain a bird sound recognition model that matches the application area.

[0057] During the application stage, the basic recognition model is not directly used for bird sound recognition. In order to solve the problem of misidentification caused by the similarity of bird song data of common bird species in different regions, this application will adaptively modify the basic recognition model according to the distribution types of birds in each application area, and obtain a bird sound recognition model that matches the distribution types of birds in the application area, thereby improving recognition accuracy.

[0058] The implementation method includes: adding a corresponding attention mechanism weight coefficient for each species in the final fully connected classification layer in the basic recognition model according to the frequency of occurrence of each species of birds in the application area indicated by the bird distribution statistics of the application area. The higher the frequency of occurrence of a species of bird in the application area, the greater the attention mechanism weight coefficient added to the corresponding species. The fully connected classification layer includes a fully connected layer and a softmax layer. The classic CNN network includes an input layer, a convolutional layer, a fully connected layer, and a softmax layer. The added attention mechanism weight coefficient is used to perform weighted calculation with the output of the fully connected layer and then input into the softmax layer.

[0059] Among them, the bird distribution statistical data of each application area is collected in advance, and the attention mechanism weight coefficients corresponding to different occurrence frequencies are pre-configured. In actual application, the occurrence frequencies of different types of birds can be divided into several frequency levels, and each frequency level corresponds to a preset attention mechanism weight coefficient.

[0060] Since different application areas often have different bird distribution statistics, the added attention mechanism weight coefficients are often different, so the matching bird sound recognition models obtained by modifying the basic recognition model are also different. Please refer to Figure 2 The schematic diagram shown takes the application of the basic recognition model in Yunnan and Henan regions as an example.

[0061] For example, in one example, the application area is Yunnan. According to the statistical data of the distribution of some birds in Yunnan, the frequency of occurrence of various bird species in Yunnan is divided into three frequency levels, namely high frequency, medium frequency and low frequency. The following table shows the types of high frequency and medium frequency birds, as well as the corresponding attention mechanism weight coefficients of each type. The frequency of occurrence of other bird species that have not appeared in Yunnan is low frequency and is not recorded in the table below, and the attention mechanism weight coefficient corresponding to the low frequency species is 0:

[0062]

[0063]

[0064] In another example, the application area is Henan Province. According to the statistical data of the distribution of some birds in Henan Province, the frequency of occurrence of various bird species in Henan Province is also divided into three frequency levels, which are recorded as high frequency, medium frequency and low frequency. The following table shows the types of high frequency and medium frequency birds, as well as the corresponding attention mechanism weight coefficients of each type. The frequency of occurrence of other bird species that have not appeared in Henan Province is low frequency and is not recorded in the table below, and the attention mechanism weight coefficient corresponding to the low frequency species is 0:

[0065]

[0066]

[0067] By comparing the above two examples, it can be seen that the bird distribution statistics in Yunnan and Henan are different, and the corresponding attention mechanism weight coefficients of each species are also different, resulting in different bird sound recognition models obtained in the end.

[0068] Step 5: Use the bird sound recognition model matched to each application area to perform bird sound recognition on the bird sound data to be recognized collected in the application area. This includes:

[0069] The bird song data to be identified is collected in the application area, and the MFCC features and GFCC features are extracted from the bird song data to be identified, and then feature fusion is performed to obtain the feature vector to be identified. The feature extraction method is similar to the feature extraction method for the bird song data samples in the model training stage, and the data preprocessing step of the bird song data to be identified is also omitted here.

[0070] The feature vector to be identified is then input into the bird sound recognition model that matches the application area. After the fully connected layer in the bird sound recognition model outputs the values ​​of each species, it is weightedly calculated according to the weight coefficient of the attention mechanism of each species and then input into the softmax layer. The softmax layer outputs the predicted probability of each species, and the species with the largest predicted probability is used as the species recognition result corresponding to the bird song data to be identified.

[0071] For example, in one instance, the bird song data of black-headed goldfinch and skylark are highly similar. When the bird song data of black-headed goldfinch is input, the basic recognition model will have a high probability of identifying black-headed goldfinch as skylark. For example, the fully connected layer may output the value of 3 for black-headed goldfinch and the value of 3 for skylark. If the structure of the basic recognition model is directly used to input the above values ​​into the softmax layer, the predicted probability of black-headed goldfinch output by the softmax layer is 0.44 and the predicted probability of skylark output is 0.50, which is mistakenly identified as skylark.

[0072] However, according to the method of the present application, based on the bird distribution statistics in Yunnan Province as exemplified above, it can be determined that the attention mechanism weight coefficient of the black-headed goldfinch is 2, while the skylark has not appeared and the attention mechanism weight coefficient is 0, and a bird sound recognition model for Yunnan Province is obtained. When the fully connected layer outputs the corresponding value of the black-headed goldfinch as 3 and the corresponding value of the skylark as 3.5, the bird sound recognition model for Yunnan Province is used to perform weighted calculation on the attention mechanism weight coefficient and the value of the fully connected layer and then input into the softmax layer. Then, the predicted probability of the black-headed goldfinch output by the softmax layer is 0.86, and the predicted probability of the skylark output is 0, which can be identified as a black-headed goldfinch.

[0073] This application also discloses a bird sound recognition device based on the attention mechanism, please refer to Figure 3 As shown in the device structure diagram, the bird sound recognition device includes a sample data acquisition module, a feature extraction module, a model training module, a model correction module and a recognition module:

[0074] The sample data acquisition module is used to acquire an initial sample data set, which includes bird song data samples of several different types of bird samples.

[0075] The feature extraction module is used to extract MFCC features and GFCC features from the bird song data sample of each bird sample in the initial sample data set, and then perform feature fusion to obtain a fused feature vector of the bird sample.

[0076] The feature extraction module is also used to fuse the extracted MFCC features with a two-dimensional size of t*n and the extracted GFCC features with a two-dimensional size of t*m in a feature splicing manner to obtain a fused feature vector with a two-dimensional size of t*(m+n).

[0077] The model training module is used to take the fused feature vector of each bird sample as input and the species to which the bird sample belongs as output, and perform model training based on the CNN network to obtain a basic recognition model.

[0078] The model correction module is used to add the attention mechanism weight coefficient to the last fully connected classification layer in the basic recognition model according to the bird distribution statistical data of the application area, so as to obtain a bird sound recognition model that matches the application area.

[0079] The model correction module is also used to add a corresponding attention mechanism weight coefficient for each species in the last fully connected classification layer in the basic recognition model according to the frequency of occurrence of each species of birds in the application area indicated by the bird distribution statistics of the application area; the higher the frequency of occurrence of a species of bird in the application area, the greater the attention mechanism weight coefficient added to the corresponding species.

[0080] The recognition module is used to perform bird sound recognition on the bird song data to be recognized collected in the application area by using the bird sound recognition model matched to each application area.

[0081] The recognition module is also used to collect bird song data to be identified in the application area, extract MFCC features and GFCC features from the bird song data to be identified, and then perform feature fusion to obtain a feature vector to be identified; the feature vector to be identified is input into a bird sound recognition model that matches the application area, and after the fully connected layer in the bird sound recognition model outputs the values ​​of each species, a weighted calculation is performed according to the weight coefficient of the attention mechanism of each species and then input into the softmax layer, which outputs the predicted probability of each species, and the species with the largest predicted probability is used as the species recognition result corresponding to the bird song data to be identified.

[0082] The specific implementation method of each module in the bird sound recognition device is similar to the above-mentioned bird sound recognition method, and will not be repeated in this embodiment.

[0083] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. A bird sound recognition method based on attention mechanism, characterized in that: The bird sound recognition method comprises: Acquire an initial sample data set, wherein the initial sample data set includes bird song data samples of a plurality of different types of bird samples; Extracting MFCC features and GFCC features from the bird song data samples of each bird sample in the initial sample data set, and then performing feature fusion to obtain a fused feature vector of the bird sample; Taking the fused feature vector of each bird sample as input and the species to which the bird sample belongs as output, a basic recognition model is obtained by model training based on a CNN network; According to the frequency of occurrence of various types of birds in the application area indicated by the bird distribution statistics of the application area, a corresponding attention mechanism weight coefficient is added to each type of the last fully connected classification layer in the basic recognition model to obtain a bird sound recognition model matching the application area; the higher the frequency of occurrence of a type of bird in the application area, the greater the attention mechanism weight coefficient added to the corresponding type; The bird sound recognition model matched to each application area is used to perform bird sound recognition on the bird song data to be recognized collected in the application area.

2. The bird sound recognition method according to claim 1, characterized in that: The feature fusion after extracting MFCC features and GFCC features respectively includes: The extracted MFCC features with a two-dimensional size of t*n and the extracted GFCC features with a two-dimensional size of t*m are fused by feature splicing to obtain a fused feature vector with a two-dimensional size of t*(m+n).

3. The bird sound recognition method according to claim 1, characterized in that: The fully connected classification layer includes a fully connected layer and a softmax layer, and the bird sound recognition model matched to each application area is used to perform bird sound recognition on the bird song data to be recognized collected in the application area, including: Collecting bird song data to be identified in the application area, extracting MFCC features and GFCC features from the bird song data to be identified, and then performing feature fusion to obtain a feature vector to be identified; The feature vector to be identified is input into the bird sound recognition model that matches the application area. After the fully connected layer in the bird sound recognition model outputs the values ​​of each species, it is weightedly calculated according to the weight coefficient of the attention mechanism of each species and then input into the softmax layer. The softmax layer outputs the predicted probability of each species, and the species with the largest predicted probability is used as the species recognition result corresponding to the bird call data to be identified.

4. The bird sound recognition method according to claim 1, characterized in that: The bird distribution statistics in different application areas are different, and the matching bird sound recognition models obtained by correction based on the basic recognition model are also different.

5. A bird sound recognition device based on attention mechanism, characterized in that: The bird sound recognition device comprises: A sample data acquisition module is used to acquire an initial sample data set, wherein the initial sample data set includes bird song data samples of several different types of bird samples; A feature extraction module is used to extract MFCC features and GFCC features from the bird song data sample of each bird sample in the initial sample data set, and then perform feature fusion to obtain a fused feature vector of the bird sample; A model training module is used to take the fused feature vector of each bird sample as input and the species to which the bird sample belongs as output, and perform model training based on the CNN network to obtain a basic recognition model; A model correction module is used to add a corresponding attention mechanism weight coefficient to each species of the last fully connected classification layer in the basic recognition model according to the frequency of occurrence of each species of birds in the application area indicated by the bird distribution statistics of the application area, so as to obtain a bird sound recognition model matching the application area; the higher the frequency of occurrence of a species of birds in the application area, the greater the attention mechanism weight coefficient added to the corresponding species; The recognition module is used to use the bird sound recognition model matched to each application area to perform bird sound recognition on the bird song data to be recognized collected in the application area.

6. The bird sound recognition device according to claim 5, characterized in that: The feature extraction module is used to fuse the extracted MFCC features with a two-dimensional size of t*n and the extracted GFCC features with a two-dimensional size of t*m in a feature splicing manner to obtain a fused feature vector with a two-dimensional size of t*(m+n).

7. The bird sound recognition device according to claim 5, characterized in that: The fully connected classification layer includes a fully connected layer and a softmax layer; The recognition module is used to collect bird song data to be identified in the application area, extract MFCC features and GFCC features from the bird song data to be identified, and then perform feature fusion to obtain a feature vector to be identified; the feature vector to be identified is input into a bird sound recognition model that matches the application area, and after the fully connected layer in the bird sound recognition model outputs the value of each species, a weighted calculation is performed according to the weight coefficient of the attention mechanism of each species and then input into the softmax layer, the softmax layer outputs the predicted probability of each species, and the species with the largest predicted probability is used as the species recognition result corresponding to the bird song data to be identified.

8. The bird sound recognition device according to claim 5, characterized in that: The bird distribution statistics in different application areas are different, and the matching bird sound recognition models obtained by correction based on the basic recognition model are also different.

Images