Bird sound recognition method, device, computer equipment and storage medium

Abstract

The present application relates to the field of classification models, and uses convolutional neural networks to realize the recognition of bird sound categories. Specifically disclosed is a method, device, computer equipment and storage medium for bird sound recognition, which includes: performing noise reduction processing on the acquired bird sound recording data according to the wavelet transform algorithm to obtain a noise-reduced bird sound signal; pre-emphasizing, framing and windowing the noise-reduced bird sound signal, performing fast Fourier transform and square processing on the obtained window data, and obtaining the spectral line energy corresponding to each window data; processing the energy of each spectral line based on the Mel filter group, performing cepstrum analysis on the Mel spectrum data corresponding to each window data, and performing first-order difference and second-order difference on the results of the cepstrum analysis, and obtaining the Mel cepstrum feature vector corresponding to each window data; based on the trained convolutional neural network, the noise-reduced bird sound signal is recognized according to the Mel cepstrum feature vector, and the bird recognition data corresponding to the noise-reduced bird sound signal is output.

Description

Technical Field

[0001] The present application relates to the technical field of sound recognition, and in particular to a method, device, computer equipment and storage medium for bird sound recognition. Background Art

[0002] According to the Red Data List of the International Union for Conservation of Nature, nearly 1,400 (more than 13%) bird species in the world may be endangered. An effective bird monitoring method is needed to assess species richness and the current achievements of ecological conservation. Methods for accurately estimating the size of bird populations require a lot of time and effort and are costly. Bird calls are often used to detect, monitor and quantify species because this method is effective even when individuals are out of sight. Humans are able to identify birds by hearing, and bird experts can identify thousands of bird species by sound.

[0003] At present, bioacoustic recording devices with long battery life and high storage capacity are more common. By setting up these recording devices in the wild on a large scale, it is possible to obtain a large amount of recording data in the wild environment for a long time. However, analyzing bird species from these data requires ornithologists to spend a lot of time and energy. Summary of the invention

[0004] The present application provides a bird sound recognition method, apparatus, computer equipment and storage medium to automatically detect which bird the bird call belongs to, with higher recognition efficiency and accuracy.

[0005] In a first aspect, the present application provides a method for identifying bird sounds, the method comprising:

[0006] Obtain bird sound recording data;

[0007] Performing noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal;

[0008] Performing pre-emphasis processing, frame processing and windowing processing on the noise-reduced bird sound signal to obtain a plurality of window data;

[0009] Performing fast Fourier transform processing and square processing on the plurality of window data to obtain spectral line energies corresponding to the plurality of window data;

[0010] Processing the energy of each spectral line based on a Mel filter bank to obtain Mel spectrum data corresponding to each of the plurality of window data;

[0011] Performing cepstrum analysis on each of the Mel frequency spectrum data and performing first-order difference and second-order difference on the results of the cepstrum analysis to obtain Mel cepstrum feature vectors corresponding to each of the plurality of window data;

[0012] Based on the trained convolutional neural network, the denoised bird sound signal is identified according to the plurality of Mel-cepstral feature vectors to output bird identification data corresponding to the denoised bird sound signal.

[0013] In a second aspect, the present application further provides a bird sound recognition device, the device comprising:

[0014] A bird sound acquisition module is used to acquire bird sound recording data;

[0015] A bird sound noise reduction module, used for performing noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal;

[0016] A preprocessing module is used to perform pre-emphasis processing, frame processing and window processing on the noise-reduced bird sound signal to obtain a number of window data;

[0017] A spectral line energy extraction module, used for performing fast Fourier transform processing and square processing on the plurality of window data to obtain spectral line energies corresponding to the plurality of window data;

[0018] A Mel filter module, used for processing the energy of each spectral line based on a Mel filter bank to obtain Mel spectrum data corresponding to each of the plurality of window data;

[0019] A feature vector extraction module, used for performing cepstrum analysis on each of the Mel frequency spectrum data and performing first-order difference and second-order difference on the results of the cepstrum analysis, so as to obtain Mel cepstrum feature vectors corresponding to each of the plurality of window data;

[0020] The feature recognition module is used to recognize the denoised bird sound signal according to the plurality of Mel-cepstral feature vectors based on the trained convolutional neural network, so as to output bird recognition data corresponding to the denoised bird sound signal.

[0021] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor;

[0022] The memory is used to store computer programs;

[0023] The processor is used to execute the computer program and implement the above-mentioned bird sound recognition method when executing the computer program.

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the bird sound recognition method as described above.

[0025] The present application discloses a bird sound recognition method, device, computer equipment and storage medium. Firstly, bird sound recording data is obtained, and noise reduction processing is performed on the bird sound recording data through a wavelet transform algorithm, so as to reduce the noise interference in the extraction of feature vectors; then, the noise-reduced bird sound signal is pre-emphasized, framed and windowed, and the obtained window data is fast Fourier transformed and squared to obtain the spectral line energy corresponding to each window data; secondly, the energy of each spectral line is processed based on a Mel filter group, and a cepstrum analysis is performed on the obtained Mel spectrum data, and the first-order difference and second-order difference are performed on the result of the cepstrum analysis, so as to avoid the loss of important features; finally, based on a trained convolutional neural network, the noise-reduced bird sound signal is recognized according to the Mel cepstrum feature vector, so as to effectively improve the recognition efficiency and accuracy when the noise-reduced bird sound signal is recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 is a schematic flow chart of a bird sound recognition method provided in an embodiment of the present application;

[0028] Figure 2 yes Figure 1 A flowchart showing the sub-steps of noise reduction processing for bird sound recording data;

[0029] Figure 3 yes Figure 2 A schematic flow chart of sub-steps of an implementation method for extracting noise segments and bird sound segments from a video recording medium;

[0030] Figure 4 yes Figure 2 A schematic flow chart of sub-steps of another implementation method for extracting noise segments and bird sound segments;

[0031] Figure 5 yes Figure 2 A schematic flow chart of the sub-steps of performing wavelet threshold denoising in;

[0032] Figure 6 This is a schematic diagram of the effect of wavelet threshold denoising on bird sound clips;

[0033] Figure 7 A schematic diagram of the denoising effect of bird sound clips according to different wavelet basis functions;

[0034] Figure 8 for Figure 1A schematic flowchart of the sub-steps of identifying noise-reduced bird sound signals;

[0035] Fig. 9 for Figure 8 A schematic flowchart of the sub-steps of generating a feature image in FIG.

[0036] Fig.10 for Figure 8 Schematic diagram of the sub-steps of convolution and pooling in ;

[0037] Fig.11 A schematic block diagram of a bird sound recognition device provided in an embodiment of the present application;

[0038] Fig.12 for Fig.11 Schematic block diagram of the bird sound noise reduction module;

[0039] Fig.13 for Fig.11 A schematic block diagram of a feature recognition module;

[0040] Fig.14 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0043] The embodiments of the present application provide a bird sound recognition method, apparatus, computer device and storage medium. The bird sound recognition method can be applied to a terminal or a server, and can quickly and accurately output bird recognition data based on the acquired bird sound recording data.

[0044] The terminal may be a mobile phone, tablet computer, laptop computer, desktop computer, personal digital assistant, wearable device, etc. The server may be an independent server or a server cluster.

[0045] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0046] See also Figure 1 , Figure 1 1 is a schematic flow chart of a bird sound recognition method provided in an embodiment of the present application. The bird sound recognition method is used to identify the bird species corresponding to the bird sound by analyzing the bird sound recording data, for example, to provide data support for ecological environment monitoring. Exemplarily, the bird sound recognition method is applied to a bird sound recognition device so that the bird sound recognition device can automatically detect which bird the bird sound belongs to.

[0047] In this embodiment, bird sound recognition is performed based on a convolutional neural network. A convolutional neural network is a biologically inspired variant of a multi-layer perceptron, which has been proven to be very effective in fields such as image recognition and image classification. The following is an example of a bird sound recognition method based on a convolutional neural network.

[0048] like Figure 1 As shown, the bird sound recognition method includes steps S10 to S70.

[0049] Step S10: Obtain bird sound recording data.

[0050] The bird sound recording data may be collected in the wild by a recording device, and at least one audio segment in the bird sound recording data contains bird sounds.

[0051] Exemplarily, after the recording device collects the bird sound recording data, the bird sound recording data is sent to the server through communication means so that the server can obtain the bird sound recording data; or after the recording device collects the bird sound recording data, the staff exports the bird sound recording data in the recording device to a computer, mobile phone or other terminal, and the terminal sends the bird sound recording data to the server so that the server can obtain the bird sound recording data.

[0052] In some embodiments, the bird sound recording data includes not only bird sounds, but also some other irrelevant noises, such as environmental noise. The bird sound recording data may include one or more bird sound segments, each of which may be from the same bird or from different birds.

[0053] Step S20: performing noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal.

[0054] Exemplarily, the bird sound recording data may be denoised using a wavelet threshold denoising method, a wavelet modulus maximum denoising method or a denoising method based on correlation between wavelet coefficient scales.

[0055] Step S30: performing pre-emphasis processing, frame processing and windowing processing on the noise-reduced bird sound signal to obtain a plurality of window data.

[0056] The Mel-Frequency Cepstral Coefficients (MFCC) algorithm is based on the human auditory mechanism and analyzes the spectrum of speech according to the results of human auditory experiments. It is usually used to extract features from human speech information. In this embodiment, the Mel-Frequency Cepstral Coefficients algorithm is applied to a series of processes on the noise reduction speech signal containing the bird sound segment, including: preprocessing, fast Fourier transform, calculation of spectral line energy, calculation of mel spectrum data obtained by mel filter through spectral line energy, and calculation of the cepstrum of the mel spectrum data to obtain a number of mel cepstrum feature vectors.

[0057] Specifically, the purpose of pre-emphasis processing on the noise-reduced bird sound signal is to compensate for the loss of high-frequency components of the noise-reduced bird sound signal and enhance the high-frequency components. The filter used for pre-emphasis is

[0058] H(z)=1-az -1 ;

[0059] In the formula, a is a constant, z represents z transformation, and z -1 Indicates that the filter H(z) is a first-order filter.

[0060] The frame division process is used to treat the quasi-steady-state signal as a steady-state signal. Specifically, the quasi-steady-state signal is divided into shorter frames, and the quasi-steady-state signal can be treated as a steady-state signal in each frame and processed by the method of processing a steady-state signal.

[0061] In an embodiment, the noise reduction bird sound signal X(n) can be processed by a method for processing a steady-state signal by performing frame processing on the noise reduction bird sound signal X(n). Exemplarily, the noise reduction bird sound signal X(n) is divided into a plurality of frames with a length of 20-30 milliseconds; in order to avoid excessive changes between two adjacent frames, an overlapping area can be provided between two adjacent frames.

[0062] Each frame signal after framing is subjected to windowing processing, which is achieved by a windowing function, such as a rectangular window, a Heining window or a Hamming window. The windowing function is used to reduce leakage in the frequency domain. For example, a frame signal is multiplied by a window function to obtain the window data corresponding to the frame. The i-th window data is represented by X i (n).

[0063] Step S40: Perform fast Fourier transform processing and square processing on the plurality of window data to obtain spectral line energies corresponding to the plurality of window data.

[0064] For example, for the i-th window data X i(n) Perform fast Fourier transform to convert the window data X i (n) Convert time domain data into frequency domain data X(i,k).

[0065] The spectral line energy corresponding to each window data is obtained by squaring the frequency domain signal corresponding to each window data. i (n) The corresponding spectral line energy E(i,k) is:

[0066] E(i,k)=[X(i,k)] 2 .

[0067] Step S50: Process the energy of each spectral line based on the Mel filter bank to obtain Mel spectrum data corresponding to each of the plurality of window data.

[0068] Specifically, the Mel filter bank includes several filters, for example, the number of filters is 30-50, where the mth filter is H m (k), 0≤m<M, M is the number of filters. Each filter has a triangular filtering characteristic, where the mth filter H m The center frequency of (k) is f(m). In the mel frequency range, these filters are of equal bandwidth. Filter H m The transfer function of (k) is:

[0069]

[0070] The spectral line energy E(i,k) is filtered by the filter in the Mel filter bank, that is, the spectral line energy E(i,k) is multiplied by the frequency domain response of each filter in the Mel filter and then superimposed to calculate the Mel spectrum data corresponding to the spectral line energy E(i,k), that is, the i-th window data X i (n) The corresponding Mel spectrum data.

[0071] Specifically, the i-th window data X i (n) The corresponding Mel spectrum data includes the filter energy corresponding to each filter in the Mel filter bank, which represents the energy spectrum of the window data in different Mel frequency intervals. i (n) The Mel spectrum data corresponding to the mth filter is:

[0072]

[0073] Specifically, the number of Mel filter banks determines the number of identification features carried by the Mel cepstral coefficients. In speech recognition, a Mel filter bank of 13 is commonly used, and the first-order and second-order differences of the Mel cepstral coefficients are performed, with a total of 39 dimensions of features. However, such a Mel filter bank extracts fewer features and cannot cover the effective frequency of bird calls, resulting in the loss of many important identification features in the process of calculating the Mel cepstral coefficients.

[0074] Therefore, in this embodiment, the number of filters is increased, and a Mel filter group with 30-50 filters is used to extract several Mel-cepstral feature vectors of the denoised bird sound signal, which can effectively avoid losing important identification features in the process of extracting several Mel-cepstral feature vectors of the denoised bird sound signal.

[0075] Step S60: performing cepstrum analysis on each of the Mel frequency spectrum data and performing first-order difference and second-order difference on the results of the cepstrum analysis to obtain Mel cepstrum feature vectors corresponding to each of the plurality of window data.

[0076] Specifically, according to the Mel-cepstral coefficient algorithm, the Mel-frequency spectrum data S(i,m) corresponding to each of the plurality of window data are subjected to cepstrum analysis. The cepstrum analysis includes taking logarithms and discrete cosine transform (DCT), that is, taking logarithms of the filter energy corresponding to each filter in the Mel-frequency spectrum data corresponding to the window data and performing discrete cosine transform to obtain the Mel-cepstral coefficients corresponding to each filter of the corresponding window data; each window data corresponds to the Mel-cepstral coefficients of the same number of filters in the Mel filter group.

[0077] The standard Mel-cepstral coefficients only reflect the static characteristics of speech parameters, and the dynamic characteristics of speech can be described by the differential spectrum of these static features. By performing first-order and second-order differentials on the Mel-cepstral coefficients corresponding to the corresponding window data, the Mel-cepstral coefficients of the first-order differentials and the Mel-cepstral coefficients of the second-order differentials can be obtained. In this embodiment, the Mel-cepstral coefficients corresponding to a certain window data, the Mel-cepstral coefficients of the first-order differentials, and the Mel-cepstral coefficients of the second-order differentials are constructed into a Mel-cepstral feature vector corresponding to the window data.

[0078] Specifically, the first-order difference is the difference between two consecutive terms in a discrete function. Specifically, the discrete function is defined as X(k), and the first-order difference of this discrete function X(k) is:

[0079] Y(k)=X(k+1)-X(k);

[0080] Similarly, the first-order difference of Y(k), that is, the second-order difference of the discrete function X(k), is:

[0081] Z(k)=Y(k+1)-Y(k)=X(k+2)-2×X(k+1)+X(k).

[0082] In this embodiment, a Mel filter group with 42 filters is used, and 42 Mel cepstral coefficients corresponding to each of the window data are obtained. After first-order differences and second-order differences, a total of 126 Mel cepstral coefficients are obtained; therefore, the dimension of the Mel cepstral feature vector corresponding to each of the window data is 126.

[0083] Step S70: Based on the trained convolutional neural network, the denoised speech signal is identified according to the plurality of Mel-cepstral feature vectors to output bird identification data corresponding to the denoised speech signal.

[0084] Convolutional neural network is a feedforward neural network. The artificial neurons in the convolutional neural network can respond to surrounding units and perform image processing.

[0085] The plurality of Mel-cepstral feature vectors can construct a matrix structure similar to an image, and a convolutional neural network commonly used in image recognition can be used to recognize the noise-reduced speech signal based on the plurality of Mel-cepstral feature vectors.

[0086] In this embodiment, a feature image of a preset size is first generated based on the several Mel-cepstral feature vectors, and then the feature image is convolved and pooled several times based on the trained convolutional neural network. The processed result is then fully connected and normalized to obtain the bird recognition data corresponding to the denoised speech signal.

[0087] Exemplarily, the bird identification data includes the type of bird corresponding to the noise-reduced speech signal. Exemplarily, the bird identification data may also include the probability when the noise-reduced speech signal corresponds to the corresponding type, that is, the probability that the noise-reduced speech signal corresponds to a certain bird.

[0088] See also Figure 2 In step S20, the bird sound recording data is subjected to noise reduction processing according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal. That is, the bird sound signal noise reduction process specifically includes:

[0089] Step S201: Detect the bird sound recording data according to a short-time energy analysis method or a sine wave detection algorithm, and extract a noise segment without bird sounds and at least one bird sound segment containing bird sounds from the bird sound recording data according to the detection result.

[0090] Exemplarily, step S201 detects the bird sound recording data according to the short-time energy analysis method, and extracts a noise segment without bird sounds and at least one bird sound segment containing bird sounds from the bird sound recording data according to the detection result, such as Figure 3 As shown, the following steps are included:

[0091] Step S2011a, performing frame processing on the bird sound recording data to obtain a plurality of frame data.

[0092] Specifically, the short-time energy analysis method first divides the bird sound recording data into frames, for example, processing the bird sound recording data into a plurality of frames of data with a duration of 10-20 milliseconds.

[0093] For example, let the time domain signal of the bird sound recording data be x(n), and the time domain signal of the i-th frame data obtained after the windowing function frame processing be y i (n), y i (n) Satisfy:

[0094] y i (n)=w(m)×x((i-1)×inc+n);

[0095] In the formula, w(m) represents the window function, which can be a rectangular window, a Hening window or a Hamming window, m represents the width of the window; n=1,2,···,L, i=1,2,···,fn, L represents the frame length of each frame signal, that is, the number of sampling points; inc represents the frame shift length; fn represents the number of frame data after framing.

[0096] Step S2012a, calculating the short-time energy of each frame data according to the short-time energy analysis method.

[0097] Exemplarily, the i-th frame data y is calculated i The formula for the short-time energy E(i) of (n) is:

[0098]

[0099] Step S2013a: if the short-term energy of a plurality of consecutive frames of data is greater than a preset energy amplitude, the plurality of frames of data are stored as bird sound segments.

[0100] Specifically, the preset energy amplitude is a specific value. For example, if the value range of the short-time energy of the frame data is 0-1, the preset energy amplitude may be 0.2.

[0101] If the short-time energy of a plurality of consecutive frame data is greater than a preset energy amplitude of 0.2, the plurality of consecutive frame data contain audio components of bird calls, and the plurality of frame data are stored as bird sound segments.

[0102] Step S2014a: if the short-term energy of a plurality of consecutive frames of data is less than a preset energy amplitude, the plurality of frames of data are stored as noise segments.

[0103] Specifically, if the short-time energy of a plurality of consecutive frame data is less than a preset energy amplitude of 0.2, it is considered that the plurality of consecutive frame data does not contain an audio component of a bird's cry, and the plurality of frame data is stored as a noise segment.

[0104] In another embodiment, noise segments and bird sound segments are extracted from the bird sound recording data according to a sine wave detection algorithm. The sine wave detection algorithm is similar to voice activity detection (VAD) in speech signal processing, and the sine wave detection algorithm can determine whether there is a bird sound in a segment of a data by detecting the presence of a preset waveform signal in the signal.

[0105] like Figure 4 As shown, in step S201, the bird sound recording data is detected according to the sine wave detection algorithm, and a noise segment without bird sounds and at least one bird sound segment containing bird sounds are extracted from the bird sound recording data according to the detection result, which specifically includes the following steps:

[0106] Step S2011b: Perform Fourier transform on the bird sound recording data to obtain a plurality of short-time amplitude spectrum signals corresponding to the bird sound recording data.

[0107] Specifically, the bird sound recording data is first divided into frames, for example, the bird sound recording data is processed into a plurality of frames of data with a duration of 10-20 milliseconds. For example, the time domain signal of the i-th frame data is y i (n).

[0108] Specifically, for the time domain signal y of the i-th frame data i (n) Perform fast Fourier transform, which transforms the signal y i (n) Convert the time domain signal into the frequency domain signal, where the short-time amplitude spectrum signal corresponding to the kth spectrum line is:

[0109]

[0110] Where L represents the frame length of each frame signal, and N represents the total number of frames. X(i,k) is the time domain signal y of the i-th frame data. i (n) The short-time amplitude spectrum signal corresponding to the kth spectral line obtained after fast Fourier transform.

[0111] Step S2012b: If a signal with a preset waveform exists in the short-time amplitude spectrum signal, the short-time amplitude spectrum signal is stored as a bird sound segment.

[0112] Specifically, to determine whether there is a signal of a preset waveform in the short-time amplitude spectrum signal, it is necessary to consider the spectrum amplitude shape near the short-time amplitude spectrum peak and the phase continuity near the spectrum peak. A sine wave exhibits phase continuity near adjacent spectrum peaks, while noise has a random phase. Therefore, the preset waveform may be a sine wave corresponding to a preset frequency range and a preset amplitude range.

[0113] Specifically, each spectrum peak is represented by a set of features extracted from the short amplitude spectrum. The spectrum peak can be regarded as a multivariate random variable, thereby capturing the spectrum amplitude shape near the spectrum peak. If the spectrum amplitude shape near the spectrum peak has a specific C-shape, and the specific C-shape is largely different from the shape exhibited by the noise peak, then the spectrum amplitude shape near the spectrum peak is a sine wave; and at the same time, the phases near adjacent spectrum peaks of the sine wave are continuous, then it is indicated that there is a signal of a preset waveform in the short-time amplitude spectrum signal, and the short-time amplitude spectrum signal is stored as a bird sound segment.

[0114] Step S2013b: If there is no signal of a preset waveform in the short-time amplitude spectrum signal, store the short-time amplitude spectrum signal as a noise segment.

[0115] If the spectrum amplitude shape near the spectrum peak does not have a specific C shape, the spectrum amplitude shape near the spectrum peak is a non-sinusoidal wave. At the same time, the phase near the adjacent spectrum peak has a random phase, which means that there is no signal with a preset waveform in the short-time amplitude spectrum signal, and the short-time amplitude spectrum signal is stored as a noise segment.

[0116] Step S202: performing wavelet transform on the extracted noise segment to obtain wavelet coefficients of the noise segment, and setting a noise threshold according to the wavelet coefficients of the noise segment.

[0117] Specifically, the wavelet coefficients of the noise fragments are obtained after the extracted noise fragments are subjected to wavelet transform. Since the noise fragments are signals that do not contain bird calls, they are equivalent to background noise in the bird sound fragments, so the background noise in the bird sound fragments can be removed according to the noise threshold determined by the noise fragments.

[0118] Specifically, after performing wavelet transform on the extracted noise segment, several wavelet coefficients of the noise segment are obtained, and a threshold λ, i.e., a noise threshold, can be set according to the several wavelet coefficients of the noise segment. Exemplarily, the setting of the noise threshold can be based on the following conditions: for all the wavelet coefficients of the noise segment, the noise threshold can filter out the various wavelet coefficients corresponding to the noise segment to the maximum extent. Therefore, when performing noise reduction processing on the bird sound recording data according to the noise threshold, the corresponding wavelet coefficients in the background noise of the bird sound segment can also be filtered out.

[0119] Exemplarily, the noise threshold is the maximum value among the absolute values ​​of several wavelet coefficients of the noise segment.

[0120] Step S203: performing wavelet threshold denoising on the at least one bird sound segment according to the noise threshold to obtain denoised bird sound signals corresponding to each of the bird sound segments.

[0121] Specifically, Figure 5 As shown, step S203 performs wavelet threshold denoising on the at least one bird sound segment according to the noise threshold to obtain a denoised bird sound signal corresponding to each of the bird sound segments, including the following steps:

[0122] Step S2031: performing wavelet transform on the bird sound segment according to a preset wavelet basis function to obtain wavelet coefficients corresponding to the bird sound segment.

[0123] Specifically, a preset wavelet basis function, such as a Meyer wavelet basis function, is used to perform wavelet transform on the bird sound segments in the bird sound recording data to obtain wavelet coefficients corresponding to the bird sound segments.

[0124] Assume that wavelet ψ(t) is L 2 (R) space function, wavelet ψ(t) is translated and stretched to obtain a wavelet basis function ψ a,b (t), as follows:

[0125]

[0126] Where a represents the scale factor, b represents the translation factor, a and b are real numbers and a is not zero.

[0127] Specifically, the wavelet transform may include a continuous wavelet transform or a discrete wavelet transform. Specifically, the continuous signal x(t) is expanded by the continuous wavelet transform in the form of:

[0128]

[0129] In the formula, ψ * (t) represents the complex conjugate of ψ(t). The discrete wavelet transform can reduce the redundancy of wavelet transform coefficients. When the continuous wavelet transform is discretized, the scale factor a and the translation factor b need to be discretized. Specifically, the commonly used discretization method is

[0130] a=2 -j ,b=k2 -j ,j,k∈Z;

[0131] Then the continuous wavelet function can be written as a discrete wavelet function:

[0132]

[0133] At the same time, the continuous wavelet transform can be rewritten as discrete wavelet transform:

[0134]

[0135] In the formula, j and k are real numbers, ψ * (2 j tk) is ψ(2 j tk). If the scale factor a is 2 j The discretization method is used, and the translation factor b still takes a continuous value. The resulting wavelet is called a binary wavelet. At this time, the discrete wavelet function can be rewritten as

[0136]

[0137] Let x(t),ψ(t)∈L 2 (R), then the dyadic wavelet transform of x(t) is

[0138]

[0139] If there are two constants A and B, and 0<A≤B<∞, so that the Fourier transform of the function ψ(t) ψ(2 j ω) Satisfaction

[0140]

[0141] Then the function ψ(t) is called a binary wavelet, and the above formula is called a stable condition; if A=B, then the above formula is called the most stable condition.

[0142] For the signal x(t)∈L 2 (R) To reconstruct, that is, to inverse transform the binary wavelet transform, it is necessary to find the dual framework of the discrete binary wavelet, that is, to find a binary dual wavelet So that the dual wavelet The Fourier transform of satisfies:

[0143]

[0144] In the formula, ψ * (ω) is the complex conjugate of ψ(ω), and ψ(ω) is the Fourier transform of ψ(t). For any signal x(t)∈L 2 (R), the inverse transform formula of the binary wavelet is

[0145]

[0146] In the formula, for The dual wavelet, WT(2 j ,b) is the dyadic wavelet transform of x(t).

[0147] Step S2032: thresholding the wavelet coefficients corresponding to the bird sound segment according to the noise threshold.

[0148] Thresholding processing is performed on the wavelet coefficients corresponding to the bird sound segment according to the noise threshold. Specifically, thresholding is performed on the wavelet coefficients of the bird sound segment according to the noise threshold to obtain thresholded wavelet coefficients.

[0149] Specifically, the noise threshold λ thresholds the wavelet coefficients of the bird sound segment through a hard threshold function or a soft threshold function. The formula of the hard threshold function is:

[0150]

[0151] In the formula, represents the size of the wavelet coefficients, Indicates the size of the wavelet coefficient after applying the threshold.

[0152] When the absolute value of a wavelet coefficient is less than the noise threshold λ, the wavelet coefficient is set to zero; when a wavelet coefficient is greater than the threshold λ, the wavelet coefficient is kept unchanged.

[0153] The formula of the soft threshold function is:

[0154]

[0155] In the formula, Represents a symbolic function.

[0156] Specifically, if Greater than 0, then Return 1; is equal to 0, then Return 0; Less than 0, then Returns -1. When the absolute value of a wavelet coefficient is less than the noise threshold λ, the wavelet coefficient is set to zero; when it is greater than the threshold λ, the wavelet coefficient is subtracted from the noise threshold λ.

[0157] Specifically, the hard threshold function is a simple zeroing method, while the soft threshold function "shrinks" the wavelet coefficients greater than the threshold, that is, subtracts the threshold, so that the input-output curve becomes continuous. The soft threshold function can ensure both large wavelet coefficients and smooth transition of coefficients. Therefore, in the embodiment, the soft threshold function is selected to threshold the wavelet coefficients of the bird sound segment.

[0158] The wavelet coefficients of the bird sound segment in the bird sound recording data are thresholded using a noise threshold to obtain the thresholded wavelet coefficients.

[0159] Step S2033: performing inverse wavelet transform on the wavelet coefficients after the thresholding process to obtain a noise-reduced bird sound signal corresponding to the bird sound segment.

[0160] Specifically, the inverse wavelet transform formula is:

[0161]

[0162] In the formula, for The dual wavelet, WT(2 j ,b) Binary wavelet transform. The reconstructed signal obtained after the inverse wavelet transform is the denoised bird sound signal X(n) after the bird sound recording data is denoised.

[0163] In order to remove the noise component in the bird sound segment to obtain the denoised bird sound signal, the bird sound segment in the bird sound recording data is first subjected to wavelet transform using a preset wavelet basis function to obtain the wavelet coefficients corresponding to the bird sound segment; then the wavelet coefficients of the bird sound segment are thresholded according to the noise threshold to obtain the thresholded wavelet coefficients; then the thresholded wavelet coefficients are subjected to inverse wavelet transform to obtain the denoised bird sound signal. The denoising effect is as follows: Figure 6 shown.

[0164] exist Figure 6 In the figure, (a), (c), and (e) represent the waveforms and time-frequency diagrams of three bird sound segments, and (b), (d), and (f) represent the waveforms and time-frequency diagrams of the denoised bird sound signals of the bird sound segments corresponding to (a), (c), and (e), respectively. After the wavelet threshold denoising process, the bird sound segments corresponding to (a), (c), and (e) retain most of the bird sound signals and filter out most of the noise components.

[0165] In this embodiment, the wavelet basis function selected when performing wavelet transform in step S202 and step S2031 can be Haar wavelet, Daubechies wavelet, Symlet wavelet, Biorthogonal wavelet, Morlet wavelet, Mexican hat wavelet, Coiflet wavelet, Meyer wavelet, Battle-lemarie wavelet or Stromberg wavelet, etc.

[0166] like Figure 7 The figure shows the denoising effect of several different wavelet basis functions on a bird sound segment.

[0167] exist Figure 7In the figure, (a) is the waveform and time-frequency diagram of the original bird sound segment, (b), (c), (d), and (e) are the waveform and time-frequency diagram of the denoised bird sound signal obtained by performing wavelet threshold denoising on the original bird sound segment using Haar wavelet basis function, Db6 wavelet basis function, Db10 wavelet basis function, and Meyer wavelet basis function, respectively.

[0168] When using Meyer wavelet for noise reduction, the background noise in the bird sound segment is eliminated most cleanly; at the same time, since the spectrum edge leakage of Meyer wavelet is the least, the frequency leakage generated when reconstructing the signal is also the least. Therefore, in the embodiment, Meyer wavelet basis function is selected for wavelet transform and wavelet threshold noise reduction.

[0169] See also Figure 8 , Figure 8 is a schematic diagram of an embodiment of the present application providing a method for identifying the denoised bird sound signal based on a trained convolutional neural network according to the plurality of Mel-cepstral feature vectors to output bird identification data corresponding to the denoised bird sound signal, such as Figure 8 As shown, the specific steps include:

[0170] Step S701: Generate a feature image corresponding to the denoised bird sound signal according to the Mel-cepstral feature vectors corresponding to each of the plurality of window data.

[0171] In some embodiments, Fig. 9 As shown, generating a feature image corresponding to the noise-reduced bird sound signal according to the Mel-cepstral feature vectors corresponding to each of the plurality of window data includes the following steps:

[0172] Step S7011: matrix the plurality of Mel-cepstrum feature vectors to obtain feature matrices corresponding to the plurality of Mel-cepstrum feature vectors.

[0173] Specifically, the plurality of Mel-cepstrum feature vectors are used to generate an N×M feature matrix, where N represents the dimension of the Mel-cepstrum feature vector corresponding to each window data, for example, N=126; M represents the number of window data corresponding to the noise reduction bird sound signal, and the feature matrix is ​​an N×M matrix including the plurality of Mel-cepstrum feature vectors.

[0174] Step S7012: convert the feature matrix into an image of preset length and width, so as to store the image as a feature image corresponding to the plurality of Mel-cepstral feature vectors.

[0175] Specifically, the feature matrix of the plurality of Mel-cepstral feature vectors is converted into an image of preset length and width. Exemplarily, the height of the image corresponds to N rows of the feature matrix, the width of the image corresponds to M columns of the image matrix; the value of each pixel in the image, such as the gray value, corresponds to the element at the same position in the feature matrix.

[0176] Exemplarily, assuming that the number of rows of the feature matrix is ​​N=126 and the number of columns is M=256, the feature matrix is ​​converted into a 126×256 image. The image is stored as a feature image corresponding to the plurality of Mel-cepstral feature vectors, that is, the 126×256 image is stored as a 126×256 feature image corresponding to the plurality of Mel-cepstral feature vectors.

[0177] Step S702: performing several convolution and pooling processes on the feature image based on the trained convolutional neural network.

[0178] Specifically, feature extraction is performed on the feature image through a convolution operation. In addition, the feature image after the convolution operation is processed by downsampling, such as pooling, to prevent overfitting.

[0179] Specifically, pooling is to replace a certain area of ​​the image with a value, such as replacing it with the maximum value or the average value. If the maximum value is used, it is called maximum pooling (Max-pooling); if the average value is used instead, it is called mean pooling (Mean-pooling). The pooling operation can reduce the image size and achieve translation and rotation invariance. This is because the output value is calculated from a region of the image and is not sensitive to translation and rotation. In this embodiment, maximum pooling is used for pooling.

[0180] Exemplarily, the convolutional neural network in this embodiment includes 5 convolutional layers, such as Fig.10 As shown, step S702 performs several convolution and pooling processes on the feature image based on the trained convolutional neural network. The specific processing steps are as follows:

[0181] Step S7021: Perform a first convolution process and a first pooling process on the feature image.

[0182] Specifically, the 126×256 pixel feature image is first convolved using 64 convolution kernels, each with a size of 5×5 and a convolution step of 2×1; the output of the convolution process is 64 feature images, each with a pixel size of 126×126; then the 64 feature images output by the convolution layer are subjected to maximum pooling processing with a pooling window of 2×2 and a stride of 2×2. After processing, 64 feature images are output, each with 63×63 pixels.

[0183] Step S7022: Perform a second convolution process and a second pooling process on the feature image after the first pooling process.

[0184] Specifically, the 64 feature images of 63×63 pixels output in step S7021 are first subjected to convolution layer processing, using 64 convolution kernels, each of which has a size of 5×5 and a step size of 1×1; the output of the convolution processing is 64 feature images, each of which has a pixel size of 63×63; then the 64 feature images output by the convolution layer are subjected to maximum pooling processing, with a pooling window of 2×2 and a stride of 2×2, and 64 feature images are output after processing, each of which has 32×32 pixels.

[0185] Step S7023: Perform a third convolution process and a third pooling process on the feature image after the second pooling process.

[0186] Specifically, the 64 32×32 pixel feature images output in step S7022 are first subjected to convolution layer processing, using 128 convolution kernels, each of which has a size of 5×5 and a step size of 1×1; the output of the convolution processing is 128 feature images, each of which has a pixel size of 32×32; then the 128 feature images output by the convolution layer are subjected to maximum pooling processing, with a pooling window of 2×2 and a stride of 2×2, and 128 feature images are output after processing, each of which has 16×16 pixels.

[0187] Step S7024: perform a fourth convolution process and a fourth pooling process on the feature image after the third pooling process.

[0188] Specifically, the 128 16×16 pixel feature images output in step S7023 are first subjected to convolution layer processing, using 256 convolution kernels, each with a size of 5×5 and a step size of 1×1; the output of the convolution processing is 256 feature images, each with a pixel size of 16×16; then the 256 feature images output by the convolution layer are subjected to maximum pooling processing, with a pooling window of 2×2 and a stride of 2×2, and 256 feature images are output after processing, each with 8×8 pixels.

[0189] Step S7025: perform a fifth convolution process and a fifth pooling process on the feature image after the fourth pooling process.

[0190] Specifically, the 256 8×8 feature images output in step S7024 are first subjected to convolution layer processing, using 256 convolution kernels, each of which has a size of 3×3 and a stride of 1×1; the output of the convolution processing is 256 feature images, each of which has 8×8 pixels; then the 256 feature images output by the convolution layer are subjected to maximum pooling processing, with a pooling window of 2×2 and a stride of 2×2, and 256 feature images are output after processing, each of which has 4×4 pixels.

[0191] Step S703: Based on the fully connected layer in the convolutional neural network, perform fully connected processing on the feature vectors output by the multiple convolution and pooling processes.

[0192] Specifically, the 256-pixel 4×4 feature image processed by the convolution layer and the maximum pooling is fully connected according to the convolutional neural network.

[0193] The fully connected layers (FC) act as a "classifier" in the entire convolutional neural network. The fully connected layers are used to connect all the features of the previous layer and send the output values ​​to the normalization layer.

[0194] For example, the trained convolutional neural network can recognize the calls of 8 kinds of birds, so the feature vector V output by the connection layer includes 8 elements, for example, the 8 elements V1-V8 are -1, -3, 2, 1, 3, -2, 0, 4 respectively.

[0195] Step S704: Based on the normalization layer in the convolutional neural network, the result of the full connection processing is normalized to output the bird recognition data corresponding to the noise-reduced bird sound signal.

[0196] According to the normalization layer in the convolutional neural network, the feature vector output by the fully connected layer is normalized. Normalization is a softmax regression classifier. The normalization layer receives an arbitrary real-valued element and compresses the real-valued element to a relative probability. The relative probability value corresponding to the element is between 0 and 1, and the sum of the relative probabilities corresponding to all real-valued elements is 1.

[0197] Exemplarily, the softmax regression classifier is implemented by a softmax function.

[0198] In this embodiment, the feature vector V output by the fully connected layer is normalized according to the softmax function to obtain each element V in the feature vector V. i The corresponding relative probability S i Specifically, the expression of the softmax function is

[0199]

[0200] In the formula, V represents the feature vector output by the fully connected layer, V i represents the i-th element in the feature vector V, i represents the category index of the bird; T represents the number of identifiable birds, S i represents the i-th element V in the feature vector V i The relative probability obtained after normalization is the relative probability that the noise-reduced bird sound signal corresponds to the i-th bird.

[0201] Exemplarily, when T=8, the feature vector V output by the fully connected layer includes elements V1-V8 of -1, -3, 2, 1, 3, -2, 0, 4 respectively.

[0202] After normalization by the softmax function, the vector output by the normalization layer in the convolutional neural network includes the relative probabilities corresponding to the 1st to 8th birds: 0.004, 0.001, 0.085, 0.030, 0.232, 0.002, 0.015, 0.631, that is, the relative probabilities corresponding to S1-S8 are 0.004, 0.001, 0.085, 0.030, 0.232, 0.002, 0.015, 0.631. Therefore, the noise-reduced bird sound signal has a probability of 0.004 that it belongs to the first bird, a probability of 0.232 that it belongs to the fifth bird, and a probability of 0.631 that it belongs to the eighth bird. Therefore, the bird identification data corresponding to the noise-reduced bird sound signal includes the eighth bird, and can also include the relative probability of 0.631.

[0203] The bird sound recognition method provided in the above embodiment performs noise reduction processing on the original recording through the wavelet transform algorithm to reduce the noise interference in the extraction of feature vectors; performs cepstrum analysis and differentiation on the Mel spectrum data obtained by the Mel filter group to avoid the loss of important features; thereby improving the recognition efficiency and accuracy when performing feature recognition on the noise-reduced bird sound signal based on the trained convolutional neural network.

[0204] See also Fig.11 , Fig.11 The embodiment of the present application also provides a schematic block diagram of a bird sound recognition device, which is used to execute the above-mentioned bird sound recognition method. The bird sound recognition device can be configured in a server or a terminal.

[0205] like Fig.11 As shown, the bird sound recognition device 80 includes: a bird sound acquisition module 801, a bird sound noise reduction module 802, a preprocessing module 803, a spectral line energy extraction module 804, a Mel filter module 805, a feature vector extraction module 806 and a feature recognition module 807.

[0206] The bird sound acquisition module 801 is used to acquire bird sound recording data.

[0207] The bird sound noise reduction module 802 is used to perform noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal.

[0208] The pre-processing module 803 is used to perform pre-emphasis processing, frame processing and windowing processing on the noise-reduced bird sound signal to obtain a plurality of window data.

[0209] The spectral line energy extraction module 804 is used to perform fast Fourier transform processing and square processing on the plurality of window data to obtain the spectral line energy corresponding to each of the plurality of window data.

[0210] The Mel filter module 805 is used to process the energy of each of the spectral lines based on a Mel filter bank to obtain Mel spectrum data corresponding to each of the plurality of window data.

[0211] The feature vector extraction module 806 is used to perform cepstrum analysis on each of the Mel frequency spectrum data and perform first-order difference and second-order difference on the results of the cepstrum analysis to obtain Mel frequency cepstrum feature vectors corresponding to each of the plurality of window data.

[0212] The feature recognition module 807 is used to recognize the denoised bird sound signal according to the plurality of Mel-cepstral feature vectors based on the trained convolutional neural network, so as to output bird recognition data corresponding to the denoised bird sound signal.

[0213] In one embodiment, Fig.12 As shown, the bird sound noise reduction module 802 includes: a detection submodule 8021, a threshold setting submodule 8022 and a noise reduction submodule 8023.

[0214] The detection submodule 8021 is used to detect the bird sound recording data according to the short-time energy analysis method or the sine wave detection algorithm, and extract a noise segment without bird calls and at least one bird sound segment containing bird calls from the bird sound recording data.

[0215] The threshold setting submodule 8022 is used to perform wavelet transform on the extracted noise segment to obtain the wavelet coefficient of the noise segment, and set the noise threshold according to the wavelet coefficient of the noise segment.

[0216] The denoising submodule 8023 is configured to perform wavelet threshold denoising on the at least one bird sound segment containing bird sounds using the noise threshold to obtain a denoised bird sound signal corresponding to the bird sound segment.

[0217] In one embodiment, Fig.13 As shown, the feature recognition module 807 includes: a feature image generation submodule 8071, a convolution and pooling submodule 8072, a fully connected submodule 8073 and a normalization submodule 5044.

[0218] The feature image generating submodule 8071 is used to generate a feature image corresponding to the noise-reduced bird sound signal according to the Mel-cepstral feature vectors corresponding to each of the plurality of window data.

[0219] The convolution and pooling submodule 8072 is used to perform several convolution and pooling processes on the feature image based on the trained convolutional neural network.

[0220] The fully connected submodule 8073 is used to perform fully connected processing on the feature vectors output by the several convolution and pooling processes based on the fully connected layer in the convolutional neural network.

[0221] The normalization submodule 8074 is used to normalize the result of the full connection processing based on the normalization layer in the convolutional neural network to output the bird recognition data corresponding to the noise-reduced bird sound signal.

[0222] The above-mentioned device can be implemented in the form of a computer program. Fig.14 Runs on the computer device shown.

[0223] See also Fig.14 , Fig.14 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server or a terminal.

[0224] See also Fig.14 The computer device includes a processor and a memory connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0225] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0226] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any bird call recognition method.

[0227] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0228] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0229] Acquire bird sound recording data; perform noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal; perform pre-emphasis processing, frame processing and windowing processing on the noise-reduced bird sound signal to obtain a plurality of window data; perform fast Fourier transform processing and square processing on the plurality of window data to obtain spectral line energy corresponding to each of the plurality of window data; process each of the spectral line energies based on a Mel filter group to obtain Mel spectrum data corresponding to each of the plurality of window data; perform cepstrum analysis on each of the Mel spectrum data and perform first-order difference and second-order difference on the result of the cepstrum analysis to obtain Mel cepstrum feature vectors corresponding to each of the plurality of window data; identify the noise-reduced bird sound signal according to the plurality of Mel cepstrum feature vectors based on a trained convolutional neural network to output bird identification data corresponding to the noise-reduced bird sound signal.

[0230] In one embodiment, the processor obtains the noise-reduced bird sound signal to implement:

[0231] The bird sound recording data is detected according to a short-time energy analysis method or a sine wave detection algorithm, and a noise segment without bird sounds and at least one bird sound segment containing bird sounds are extracted from the bird sound recording data according to the detection result; a wavelet transform is performed on the extracted noise segment to obtain the wavelet coefficient of the noise segment, and a noise threshold is set according to the wavelet coefficient of the noise segment; and a wavelet threshold denoising is performed on the at least one bird sound segment according to the noise threshold to obtain a denoised bird sound signal corresponding to each of the bird sound segments.

[0232] In one embodiment, the processor detects the bird sound recording data according to the short-time energy analysis method, and extracts a noise segment without bird sounds and at least one bird sound segment containing bird sounds from the bird sound recording data according to the detection result, so as to implement:

[0233] The bird sound recording data is framed to obtain a plurality of frame data; the short-time energy of each frame data is calculated according to the short-time energy analysis method; if the short-time energy of a plurality of consecutive frame data is greater than a preset energy amplitude, the plurality of frame data are stored as bird sound segments; if the short-time energy of a plurality of consecutive frame data is less than a preset energy amplitude, the plurality of frame data are stored as noise segments.

[0234] In one embodiment, the processor detects the bird sound recording data according to a sine wave detection algorithm, and extracts a noise segment without bird sounds and at least one bird sound segment containing bird sounds from the bird sound recording data according to the detection result, so as to implement:

[0235] Fourier transform is performed on the bird sound recording data to obtain a plurality of short-time amplitude spectrum signals corresponding to the bird sound recording data; if a signal with a preset waveform exists in the short-time amplitude spectrum signal, the short-time amplitude spectrum signal is stored as a bird sound segment; if no signal with a preset waveform exists in the short-time amplitude spectrum signal, the short-time amplitude spectrum signal is stored as a noise segment.

[0236] In one embodiment, the processor performs wavelet threshold denoising on the at least one bird sound segment according to the noise threshold to obtain denoised bird sound signals corresponding to each of the bird sound segments, for implementing:

[0237] The bird sound segment is subjected to wavelet transform according to a preset wavelet basis function to obtain wavelet coefficients corresponding to the bird sound segment; the wavelet coefficients corresponding to the bird sound segment are subjected to threshold processing according to the noise threshold; and the wavelet coefficients after threshold processing are subjected to inverse wavelet transform to obtain a noise-reduced bird sound signal corresponding to the bird sound segment.

[0238] In one embodiment, the processor recognizes the denoised bird sound signal based on the trained convolutional neural network according to the plurality of Mel-cepstral feature vectors to output bird identification data corresponding to the denoised bird sound signal, so as to realize:

[0239] A feature image corresponding to the denoised bird sound signal is generated according to the Mel-cepstral feature vectors corresponding to each of the plurality of window data; the feature image is subjected to a plurality of convolution and pooling processes based on a trained convolutional neural network; based on a fully connected layer in the convolutional neural network, a fully connected process is performed on the feature vectors output by the plurality of convolution and pooling processes; based on a normalization layer in the convolutional neural network, a normalization process is performed on the result of the fully connected process to output the bird identification data corresponding to the denoised bird sound signal.

[0240] In one embodiment, the processor generates a feature image corresponding to the noise-reduced bird sound signal according to the Mel-cepstral feature vectors corresponding to each of the plurality of window data, so as to achieve:

[0241] The plurality of Mel-cepstrum feature vectors are matrixed to obtain feature matrices corresponding to the plurality of Mel-cepstrum feature vectors; and the feature matrix is ​​converted into an image of preset length and width to store the image as a feature image corresponding to the plurality of Mel-cepstrum feature vectors.

[0242] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement any one of the bird sound recognition methods provided in the embodiments of the present application.

[0243] The computer-readable storage medium may be an internal storage unit of the computer device described in the above embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital card (Secure Digital, SD), a flash card (Flash Card), etc. equipped on the computer device.

[0244] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for identifying bird sounds, characterized in that: include: Obtain bird sound recording data; Performing noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal; Performing pre-emphasis processing, frame processing and windowing processing on the noise-reduced bird sound signal to obtain a plurality of window data; Performing fast Fourier transform processing and square processing on the plurality of window data to obtain spectral line energies corresponding to the plurality of window data; Processing the energy of each spectral line based on a Mel filter bank to obtain Mel spectrum data corresponding to each of the plurality of window data; Performing cepstrum analysis on each of the Mel frequency spectrum data and performing first-order difference and second-order difference on the results of the cepstrum analysis to obtain a plurality of Mel cepstrum feature vectors corresponding to each of the plurality of window data; Based on the trained convolutional neural network, the denoised bird sound signal is identified according to the plurality of Mel-frequency cepstrum feature vectors to output bird identification data corresponding to the denoised bird sound signal; The step of performing noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal comprises: Performing frame processing on the bird sound recording data to obtain a plurality of frames of data; Detecting a plurality of frames of the bird sound recording data according to a short-time energy analysis method or a sine wave detection algorithm, and extracting a noise segment without bird calls and at least one bird sound segment containing bird calls from the plurality of frames of the bird sound recording data according to the detection result; Performing wavelet transform on the extracted noise segment to obtain wavelet coefficients of the noise segment, and setting a noise threshold according to the wavelet coefficients of the noise segment; Performing wavelet transformation on the bird sound segment according to a preset wavelet basis function to obtain wavelet coefficients corresponding to the bird sound segment; Based on a hard threshold function or a soft threshold function, thresholding the wavelet coefficients corresponding to the bird sound segment according to the noise threshold; The wavelet coefficients after the thresholding process are subjected to inverse wavelet transformation to obtain a noise-reduced bird sound signal corresponding to the bird sound segment.

2. The bird sound recognition method according to claim 1, characterized in that: The method of detecting a plurality of frames of the bird sound recording data according to the short-time energy analysis method, and extracting a noise segment without bird sounds and at least one bird sound segment containing bird sounds from the plurality of frames of the bird sound recording data according to the detection result, comprises: Calculate the short-time energy of each frame data according to the short-time energy analysis method; If the short-term energy of a plurality of consecutive frames of data is greater than a preset energy amplitude, the plurality of frames of data are stored as bird sound segments; If the short-term energy of a plurality of consecutive frames of data is less than a preset energy amplitude, the plurality of frames of data are stored as noise segments.

3. The bird sound recognition method according to claim 1, characterized in that: The method of detecting a plurality of frames of the bird sound recording data according to a sine wave detection algorithm, and extracting a noise segment without bird sounds and at least one bird sound segment containing bird sounds from the plurality of frames of the bird sound recording data according to the detection result, comprises: Performing Fourier transform on a plurality of frames of the bird sound recording data to obtain a plurality of short-time amplitude spectrum signals corresponding to the bird sound recording data; If there is a signal with a preset waveform in the short-time amplitude spectrum signal, storing the short-time amplitude spectrum signal as a bird sound segment; If there is no signal of a preset waveform in the short-time amplitude spectrum signal, the short-time amplitude spectrum signal is stored as a noise segment.

4. The bird sound recognition method according to any one of claims 1 to 3, characterized in that: The method of identifying the denoised bird sound signal based on the trained convolutional neural network according to the plurality of Mel-cepstral feature vectors to output bird identification data corresponding to the denoised bird sound signal includes: Generate a feature image corresponding to the noise-reduced bird sound signal according to the Mel-cepstral feature vectors corresponding to each of the plurality of window data; Performing several convolution and pooling processes on the feature image based on a trained convolutional neural network; Based on the fully connected layer in the convolutional neural network, performing fully connected processing on the feature vectors output by the multiple convolution and pooling processes; Based on the normalization layer in the convolutional neural network, the result of the full connection processing is normalized to output the bird recognition data corresponding to the noise-reduced bird sound signal.

5. The bird sound recognition method according to claim 4, characterized in that: The step of generating a feature image corresponding to the noise-reduced bird sound signal according to the Mel-cepstral feature vectors corresponding to the plurality of window data includes: Matrixing the plurality of Mel-cepstrum feature vectors to obtain feature matrices corresponding to the plurality of Mel-cepstrum feature vectors; The feature matrix is ​​converted into an image with a preset length and width, so as to store the image as a feature image corresponding to the plurality of Mel-cepstral feature vectors.

6. A bird sound recognition device, characterized in that: include: A bird sound acquisition module is used to acquire bird sound recording data; A bird sound noise reduction module, used for performing noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal; A preprocessing module is used to perform pre-emphasis processing, frame processing and window processing on the noise-reduced bird sound signal to obtain a number of window data; A spectral line energy extraction module, used for performing fast Fourier transform processing and square processing on the plurality of window data to obtain spectral line energies corresponding to the plurality of window data; A Mel filter module, used for processing the energy of each spectral line based on a Mel filter bank to obtain Mel spectrum data corresponding to each of the plurality of window data; A feature vector extraction module, used for performing cepstrum analysis on each of the Mel frequency spectrum data and performing first-order difference and second-order difference on the results of the cepstrum analysis, so as to obtain a plurality of Mel frequency cepstrum feature vectors corresponding to each of the plurality of window data; A feature recognition module, used for recognizing the denoised bird sound signal according to the plurality of Mel-cepstral feature vectors based on a trained convolutional neural network, so as to output bird recognition data corresponding to the denoised bird sound signal; The step of performing noise reduction processing on the bird sound recording data according to a wavelet transform algorithm to obtain at least one noise-reduced bird sound signal comprises: Performing frame processing on the bird sound recording data to obtain a plurality of frames of data; Detecting a plurality of frames of the bird sound recording data according to a short-time energy analysis method or a sine wave detection algorithm, and extracting a noise segment without bird calls and at least one bird sound segment containing bird calls from the plurality of frames of the bird sound recording data according to the detection result; Performing wavelet transform on the extracted noise segment to obtain wavelet coefficients of the noise segment, and setting a noise threshold according to the wavelet coefficients of the noise segment; Performing wavelet transformation on the bird sound segment according to a preset wavelet basis function to obtain wavelet coefficients corresponding to the bird sound segment; Based on a hard threshold function or a soft threshold function, thresholding the wavelet coefficients corresponding to the bird sound segment according to the noise threshold; The wavelet coefficients after the thresholding process are subjected to inverse wavelet transformation to obtain a noise-reduced bird sound signal corresponding to the bird sound segment.

7. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the bird sound recognition method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the bird sound recognition method according to any one of claims 1 to 5.

Images