Animal species identification method based on microphone array and sound identification model

Abstract

The invention belongs to the technical field of biodiversity observation, and relates to an animal species recognition method based on a microphone array and a sound recognition model, which comprises the following steps of: 1) acquiring three-dimensional space environment sound in real time by using the microphone array; 2) processing the collected three-dimensional space environment sound to obtain a sound signal with improved quality; and 3) based on the quality-improved sound signal, an edge-arranged convolutional attention neural network sound recognition model is adopted to carry out animal species recognition. Accurate recognition of animal sound in a complex acoustic environment is realized, so that animal species categories can be effectively detected and recognized.

Description

Technical Field

[0001] The invention belongs to the technical field of biodiversity observation and relates to an animal species identification method, and in particular to an animal species identification method based on a microphone array and a sound recognition model. Background Art

[0002] In biodiversity monitoring, manual monitoring usually requires a lot of manpower and time investment, and the monitoring results may be affected by the experience and subjective judgment of the observer. Although traditional visual recognition methods are effective, they have limitations in some cases. For example, the effect of visual recognition is limited at night, in bad weather or in dense vegetation. At the same time, the traditional single microphone acquisition method is often difficult to effectively separate when facing multiple sound sources, which greatly affects the recognition accuracy; moreover, the traditional animal sound recognition method requires manual design of feature extraction algorithms and does not have the ability of automatic feature learning and self-optimization.

[0003] Therefore, a new animal species identification method is urgently needed to ensure the accuracy of animal species identification. Summary of the invention

[0004] In order to overcome the defects of the prior art, the present invention proposes an animal species identification method based on a microphone array and a sound recognition model, which realizes the accurate recognition of animal sounds in a complex acoustic environment, thereby being able to effectively detect and identify the animal species category.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] An animal species identification method based on a microphone array and a sound recognition model, characterized in that it comprises the following steps:

[0007] 1) Use microphone array to collect three-dimensional environmental sound in real time;

[0008] 2) Processing the collected three-dimensional space environment sound to obtain a sound signal with improved quality;

[0009] 3) Based on the quality-improved sound signal, an edge-arranged convolutional attention neural network sound recognition model is used to identify animal species.

[0010] Preferably, the step 3) specifically includes:

[0011] 31) preprocessing the sound signal with improved quality to obtain a preprocessed sound spectrum;

[0012] 32) using the convolutional layer, feedforward layer and residual network layer of the convolutional attention neural network sound recognition model to perform noise reduction processing on the preprocessed sound spectrum to obtain an enhanced sound signal of the calling animal of interest;

[0013] 33) using the multi-layer convolutional layers of the convolutional attention neural network sound recognition model to perform local feature clustering and separation on each frame of the enhanced sound signal of the animal of interest to obtain clustered local features;

[0014] 34) using the attention coding layer of the convolutional attention neural network sound recognition model to extract the inter-frame global features of the enhanced sound signal of the animal of interest, so as to obtain the inter-frame contextual correlation features;

[0015] 35) using the attention coding layer of the convolutional attention neural network sound recognition model to perform feature aggregation on the clustered local features and the previous and next correlation features between frames to obtain aggregated features;

[0016] 36) Using the decoding layer of the convolutional attention neural network sound recognition model, the aggregated features are decoded into the family, genus and species of the animal to obtain the family, genus and species information of the animal.

[0017] Preferably, the step 31) is specifically as follows: after sampling the sound signal with improved quality and dividing it into fixed time slices, Fourier transform is performed to convert it into a spectrum signal, and Mel spectrum transform is performed on the spectrum signal with preset coefficients to obtain a preprocessed sound spectrum.

[0018] Preferably, the microphone array in step 1) is a spatial spherical microphone array, and the radius R and the number of microphone array elements N of the spatial spherical microphone array satisfy: Among them, d max is the maximum spacing between microphone array elements and d min is the minimum spacing between microphone array elements and C is the speed of sound, f max is the highest frequency picked up by the microphone array element, f min It is the lowest frequency picked up by the microphone array element.

[0019] Preferably, the Cartesian coordinates of the i-th microphone array element in the spatial spherical microphone array are:

[0020] x i =R×sin(φ i )×cos(θ i )

[0021] y i =R×sin(φi )×sin(θ i )

[0022] z i =R×cos(φ i )

[0023] Among them, φ i is the polar angle of the i-th microphone array element and θ i is the azimuth angle of the i-th microphone array element and δ is the golden angle and

[0024] Preferably, the step 2) specifically includes:

[0025] 21) Using adaptive audio noise reduction technology to perform real-time noise reduction on the collected three-dimensional space environment sound;

[0026] 22) Using multi-sound source real-time positioning technology to locate multiple sound sources in the three-dimensional space environment sound after noise reduction processing;

[0027] 23) Use multi-sound source dynamic tracking technology to simultaneously track the positions of multiple located sound sources;

[0028] 24) Use multi-sound source real-time separation technology to separate multiple sound sources tracked in position to obtain a clear single sound source signal;

[0029] 25) Using sound enhancement technology, each separated single sound source signal is enhanced in real time to obtain a sound signal with improved quality.

[0030] Preferably, in step 2.1), the collected three-dimensional space environment sound is subjected to real-time noise reduction processing using a least mean square LMS iterative algorithm or a recursive least squares RLS iterative algorithm;

[0031] In the step 22), for the three-dimensional environment sound after the noise reduction process, the independent component analysis ICA algorithm is first used to separate multiple sound sources, and then the TDOA algorithm is used to locate the sound source for each separated sound source;

[0032] In the step 23), the Kalman filter algorithm is used to simultaneously track the positions of the multiple located sound sources;

[0033] In the step 24), beamforming technology is used to separate multiple sound sources tracked in azimuth to obtain a clear single sound source signal;

[0034] In the step 25), each separated single sound source signal is enhanced in real time using a minimum mean square error estimation algorithm to obtain a sound signal with improved quality.

[0035] In addition, the present invention also provides an animal species identification system based on a microphone array and a sound recognition model, which is characterized by comprising:

[0036] A microphone array is used to collect three-dimensional ambient sound in real time;

[0037] A sound processing module, which is used to process the collected three-dimensional space environment sound to obtain a sound signal with improved quality;

[0038] An edge-arranged convolutional attention neural network sound recognition model for identifying animal species based on the sound signals with improved quality.

[0039] Moreover, the present invention also provides an animal species identification device based on a microphone array and a sound recognition model, characterized in that it comprises:

[0040] one or more processors;

[0041] A memory for storing one or more programs;

[0042] When the one or more programs are executed by the one or more processors, the one or more processors implement the animal species identification method based on the microphone array and the sound recognition model as described above.

[0043] Finally, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the animal species identification method based on a microphone array and a sound recognition model as described above are implemented.

[0044] Compared with the prior art, the animal species identification method based on microphone array and sound recognition model of the present invention has one or more of the following beneficial technical effects:

[0045] 1. The present invention realizes the real-time positioning, tracking, separation of multiple sound sources and accurate identification of animal species.

[0046] 2. The present invention expands the methods, time and environmental scope of monitoring the diversity of vocal animals, significantly improving the accuracy and reliability of recognition, especially when faced with complex and changeable background noise in the wild, overlapping sounds of multiple species and a large number of species categories. Through continuous learning and optimization, the convolutional attention neural network sound recognition model can continuously improve its recognition ability and adapt to new acoustic data and recognition needs.

[0047] 3. The present invention realizes real-time sound collection and recognition at the field monitoring site, without relying on the network to transmit data to a remote server for analysis and recognition. This not only greatly improves the work efficiency of biodiversity monitoring, but also ensures the quality and timeliness of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the animal species identification method based on microphone array and sound recognition model of the present invention.

[0049] Figure 2 It is a schematic diagram of the structure of the animal species identification system based on microphone array and sound recognition model of the present invention. DETAILED DESCRIPTION

[0050] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components described below or illustrated in the following figures. The present invention can have other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used here are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and its variations herein is intended to cover the items and their equivalents and additional items displayed below. Unless otherwise specified or limited, the terms "install", "connect", "support" and "couple" and their variations are widely used and cover direct installation and indirect installation, connection, support and connection. In addition, "connect" and "couple" are not limited to physical or mechanical connections or connections.

[0051] Furthermore, in the disclosure of the present invention, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0052] In view of the defects of the prior art, the present invention provides an animal species identification method based on a microphone array and a sound recognition model. The method uses microphone array technology to efficiently collect sounds in a three-dimensional space environment, and ensures the clarity of sound signals through real-time noise reduction technology. Subsequently, multiple sound source real-time positioning technology and multiple sound source dynamic tracking technology are used to accurately locate and track the positions and dynamic changes of multiple sound sources. Moreover, in order to further improve the recognition accuracy, multiple sound source real-time separation technology is used to separate sound signals from different sound sources, and sound enhancement technology is used to improve signal quality. Finally, the convolutional attention neural network sound recognition model is used to identify the processed sound signal, thereby effectively detecting and identifying the animal species category. This method achieves accurate recognition of animal sounds in complex acoustic environments by integrating multiple advanced technologies.

[0053] Figure 1 FIG. 4 is a flow chart showing an animal species identification method based on a microphone array and a sound recognition model of the present invention. Figure 1 As shown, the animal species identification method based on microphone array and sound recognition model of the present invention comprises the following steps:

[0054] 1. Use microphone array to collect three-dimensional environmental sound in real time.

[0055] The present invention uses a spatial spherical microphone array to collect three-dimensional space environment sound in real time. Furthermore, an omnidirectional microphone array element is used to pick up sounds in all directions. Furthermore, a high-sensitivity microphone array element is used to improve the pickup of distant weak sounds. Compared with the use of a single microphone, the use of microphone array technology can achieve efficient collection of sounds in a three-dimensional space environment.

[0056] The radius R and the number of microphone elements N of the spherical microphone array are determined by the following method:

[0057] 1. According to the Nyquist sampling theorem, in order to avoid spatial aliasing, the distance between microphone array elements should be less than half of the sound wave wavelength λ, and the relationship between wavelength λ and sound speed c and frequency f is Therefore, based on the highest frequency f that the microphone array element can pick up max , the minimum spacing d between microphone array elements can be calculated min ,Right now, Based on the lowest frequency f that the microphone array can pick up min , the maximum spacing d between microphone array elements can be calculated max ,Right now, Where c is the speed of sound, usually 343 m / s (in air at 20°C).

[0058] 2. Select the spherical radius R of the spherical microphone array.

[0059] The radius R should meet the spacing requirements between microphone array elements, that is,

[0060] 3. Calculate the number of microphone array elements N.

[0061] For a sphere with radius R, the number of microphones N can be estimated by the following formula:

[0062] Therefore, based on the sphere radius R selected in step 2, the formula in step 3 is used to calculate d in step 1. min Determine the number of microphones N, That is, the calculation result is rounded.

[0063] For the position of each microphone array element, the Fibonacci method can be used to calculate the approximate uniform distribution coordinates of the microphone array element on the sphere, specifically:

[0064] 1. Calculate the golden angle δ:

[0065] 2. Calculate the polar coordinates of each microphone array element:

[0066] Among them, the polar angle φ i :

[0067] Azimuth θ i :

[0068] Where N is the number of microphone array elements, and i is the i-th microphone array element.

[0069] 3. Convert the polar coordinates of each microphone array element to Cartesian coordinates:

[0070] x i =R×sin(φ i )×cos(θ i )

[0071] y i =R×sin(φ i )×sin(θ i )

[0072] z i =R×cos(φ i ).

[0073] With the radius R of the spherical microphone array, the number of microphone array elements N and the Cartesian coordinates of each microphone array element, the spherical microphone array can be specifically configured.

[0074] 2. Process the collected three-dimensional space environment sound to obtain a sound signal with improved quality.

[0075] The present invention uses a variety of processing technologies to process the collected three-dimensional space environment sound to improve the accuracy of animal species identification, specifically including:

[0076] 1. Use adaptive audio noise reduction technology to perform real-time noise reduction on the collected three-dimensional space environment sound.

[0077] For the original audio data of each channel picked up by the microphone array, different adaptive filtering algorithms are used to perform noise reduction processing according to the specific conditions of computing resources.

[0078] (1) When computing resources are limited, the least mean squares (LMS) iterative algorithm is used. The process is as follows:

[0079] a. Initialization.

[0080] Set the Nth-order initial weight vector of the least mean square filter w(0) = [ω0(0), ω1(0), ..., ω N-1 (0)] T ,ω0(0),ω1(0),…,ω N-1 (0) is the Nth-order initial weight vector of the least mean square filter, and all elements in the vector w(0) are set to zero; the learning rate parameter μ is selected, and its value is usually between 0<μ<1.

[0081] b. Input sound signal.

[0082] Assume an N-order input signal x(n) and an expected signal d(n), where x(n) is a noisy signal, d(n) is an expected noise-free signal, and n is the current iteration step.

[0083] c. Calculate the nth iteration filter output.

[0084] That is, calculate the N-th order filter output signal y(n):

[0085] y(n)=w T (n) x(n),

[0086] where x(n) = [x(n), x(n-1), ..., x(n-N+1)] T , is the sampling value of N sound signals before the current time n obtained from the microphone array element.

[0087] d. Calculation of the nth iteration error.

[0088] That is, calculate the Nth-order error signal e(n):

[0089] e(n)=d(n)-y(n).

[0090] e. Weight update for the nth iteration.

[0091] That is, the filter weights are updated according to the error signal:

[0092] w(n+1)=w(n)+μ·e(n)·x(n),

[0093] Among them, w(n+1) is the weight vector of the next iteration, and the remaining parameters are shown in the above steps.

[0094] F. Iteration.

[0095] Repeat steps c to e until a predetermined error level is reached, thereby obtaining a noise-reduced sound output signal.

[0096] (2) When computing resources are sufficient, the recursive least squares (RLS) iterative algorithm is used. The process is as follows:

[0097] a. Initialization.

[0098] Set the Nth-order initial weight vector w(0) = [ω0(0), ω1(0), ..., ω N-1 (0)] T ,ω0(0),ω1(0),…,ω N-1 (0) is the N-order initial weight vector of the recursive least squares RLS filter, and all elements in the vector w(0) are set to zero; the initial N-order inverse covariance matrix P(0) = δ -1 I, where δ is a small positive number 10 -6 , I is the N-order unit matrix; the forgetting factor λ is selected, and its value is usually between 0<λ<1.

[0099] b. Input sound signal.

[0100] Assume an N-th order input signal x(n) and a desired sound signal d(n), where x(n) is a signal containing noise, d(n) is a desired noise-free signal, and n is the current iteration step.

[0101] c. Calculate the sound output of the nth iteration filter.

[0102] That is, calculate the N-order filter sound output signal y(n):

[0103] y(n)=w T (n) x(n),

[0104] where x(n) = [x(n), x(n-1), ..., x(n-N+1)] T It is the sampling value of N sound signals before the current time n obtained from the microphone array element.

[0105] d. Calculation of the nth iteration error.

[0106] That is, calculate the Nth-order error signal e(n):

[0107] e(n)=d(n)-y(n).

[0108] e. Calculation of gain vector for the nth iteration.

[0109] That is, calculate the Nth-order gain vector k(n):

[0110]

[0111] Among them, P(n-1) is the inverse covariance matrix of the previous iteration, its initial value is shown in step a, the update formula is shown in step g, and the remaining parameters are shown in the above steps.

[0112] f. Weight update for the nth iteration.

[0113] That is, calculate the updated filter weights:

[0114] w(n)=w(n-1)+k(n)·e(n),

[0115] Among them, w(n-1) is the weight of the previous iteration, its initial value is shown in step a, and the remaining parameters are shown in the above steps.

[0116] g. Update the inverse covariance matrix for the nth iteration.

[0117] That is, calculate the updated inverse covariance matrix:

[0118]

[0119] Wherein, P(n-1) is the inverse covariance matrix of the previous iteration, its initial value is shown in step a, and the remaining parameters are shown in the above steps.

[0120] h. Iteration.

[0121] Repeat steps c to g until a predetermined error level is reached, thereby obtaining a noise-reduced sound output signal.

[0122] 2. Use multi-sound source real-time positioning technology to locate multiple sound sources in the three-dimensional space environment sound after noise reduction processing.

[0123] The multi-sound source real-time positioning technology is used to locate the multiple sound sources in the three-dimensional space environment sound after noise reduction, that is, direction recognition, so that the simultaneous collection of multi-source sounds in the monitoring scenario can be realized.

[0124] In the present invention, for the three-dimensional space environmental sound after the above-mentioned noise reduction processing, the independent component analysis ICA (Independent Component Analysis) algorithm is first used to separate multiple sound sources, and then the time difference positioning TDOA (Time Difference of Arrival) algorithm is used to locate the sound source of each separated sound source.

[0125] (1) The ICA algorithm is used to separate multiple sound sources. The process is as follows:

[0126] a. Data preparation.

[0127] Input sound signal matrix: The sound signals collected by N microphone array elements form an observation sound signal matrix X, in which each row is the sampling value of the sound signal of a microphone array element.

[0128] Centering: The sampling values ​​of the sound signal are centralized so that the mean of each row of the signal is zero. Specifically, for each sound signal sampling value x in the i-th column, i , minus the average sample value of the row Get the processed sample value x i ',Right now,

[0129] Whitening: Whiten the sound signal so that the covariance matrix of the signal is the unit matrix.

[0130] b. Model assumptions.

[0131] Assume that the microphone array observation sound signal matrix X is a linear mixture of multiple sound source signal matrices S, that is, X=AS.

[0132] Among them, A is the unknown mixing matrix, and S is the multi-source signal matrix that needs to be estimated.

[0133] c. Maximize independence.

[0134] The independence metric negative entropy is selected to evaluate the independence of the signal. The unmixing matrix W is adjusted by the gradient descent of the iterative optimization algorithm to maximize the negative entropy of the output multi-source separation signal matrix Y=WX.

[0135] d. Iterative optimization.

[0136] Initialization: Randomly initialize the unmixing matrix W.

[0137] Iterative update: Update W using nonlinear logarithmic function and gradient descent method,

[0138] W←W+η(I+g′(Y)Y T )W,

[0139] Where η is the learning rate, g′(·) is the derivative of the nonlinear function, and I is the identity matrix.

[0140] Orthogonalization: After each update, the unmixing matrix W is orthogonalized to maintain its orthogonality.

[0141] e. Convergence judgment.

[0142] When the change of W is less than the predetermined threshold ∈, the iteration is stopped. Specifically, the norm change of the unmixing matrix W can be calculated,

[0143] ||ΔW||<∈,

[0144] Here, ∈ is a small positive threshold.

[0145] f. Output.

[0146] The separated multi-sound source signal matrix is ​​Y=WX, where each row represents an independent sound source signal.

[0147] (2) The TDOA algorithm is used to locate the separated sound sources. The process is as follows:

[0148] a. System settings.

[0149] Microphone array: There are N microphone array elements, and the position m of the i-th microphone array element in three-dimensional space is known i =(x i ,y i ,z i ).

[0150] Speed ​​of sound: Set the speed of sound in air c to 343m / s.

[0151] b. Signal acquisition.

[0152] Sampling: Audio signals are collected from all microphone array elements at the same time, with a sampling frequency of f s .

[0153] c. Time difference calculation.

[0154] Cross-correlation: Calculate the cross-correlation R of the signal of each pair (i-th and j-th) of microphone array elements in N microphones ij (τ), to determine the time delay,

[0155] R ij (τ)=∑ n x i (n)·x j (n+τ),

[0156] Among them, x i (n) and x j (n) are the signals from microphone elements i and j at the nth sampling moment, respectively, and τ is the time delay.

[0157] Peak detection: In cross-correlation R ij Find the peak value in (τ) to determine the time difference Δt ij .

[0158] Calculate the time difference Δt ij ,

[0159]

[0160] d. Sound source position estimation.

[0161] Geometric localization: Using the known positions of microphone array elements and the calculated time difference, a geometric method is used to estimate the location of the sound source.

[0162] Establish the equation system: For each pair of microphone array elements (i, j), establish the equation based on the time difference,

[0163] ||pm i ||-||pm j ||=c·Δt ij ,

[0164] Where p = (x, y, z) is the location of the sound source, m i =(x i ,y i ,z i ) is the position of the i-th microphone array element in the three-dimensional environment space, m j =(x j ,y j ,z j ) is the position of the j-th microphone array element in the three-dimensional environment space, and the remaining parameters are shown in the above steps.

[0165] Solve the system of equations: Use the least squares method to solve the system of equations and obtain the sound source position p.

[0166] Set the error function:

[0167]

[0168] Minimize the error function E(p) to obtain the optimal sound source position p.

[0169] 3. Use multi-sound source dynamic tracking technology to simultaneously track the positions of multiple sound sources after positioning.

[0170] Multi-sound source dynamic tracking technology is used to simultaneously track the positions of multiple sound sources after positioning to ensure the continuity and consistency of the sound.

[0171] In the present invention, specifically, a Kalman filter algorithm is used to dynamically track each sound source, and the process is as follows:

[0172] a. System modeling.

[0173] State vector: Define the state vector x of the sound source in the kth iteration k , including position and velocity.

[0174] State transfer model: define the state transfer equation of the sound source,

[0175] x k+1 =Fx k +wk ,

[0176] Among them, F is the sound source state transfer matrix, w k is the process noise in the kth iteration, x k and x k+1 are the sound source state vectors of the kth and k+1th iterations respectively.

[0177] Observation model: Define the sound source tracking observation equation,

[0178] z k =Hx k +v k ,

[0179] Among them, z k is the sound source tracking observation result of the kth iteration (including position and velocity), H is the sound source tracking observation matrix, v k is the sound source tracking observation noise of the kth iteration.

[0180] b. Initialization.

[0181] Initial state estimation: Set the initial state estimation of the sound source and the initial covariance matrix P0.

[0182] Noise Covariance: Set the process noise covariance matrix Q and the observation noise covariance matrix R.

[0183] c. Prediction.

[0184] Sound source state prediction:

[0185]

[0186] in, is the sound source state prediction from the k-1th to the kth iteration, is the sound source state of the k-1th iteration, and F is the sound source state transfer matrix.

[0187] Sound source covariance prediction:

[0188] P k|k-1 =FP k-1|k-1 F T +Q,

[0189] Among them, P k|k-1 is the sound source covariance prediction from the k-1th to the kth iteration, P k-1|k-1 is the sound source covariance of the k-1th iteration, and the remaining parameters are as in the above steps.

[0190] d. Update.

[0191] Kalman gain calculation:

[0192] K k =P k|k-1 H T (HP k|k-1 H T +R) -1 ,

[0193] Among them, K k is the Kalman gain of the kth iteration, P k|k-1 is the sound source covariance prediction from the k-1th to the kth iteration, and the remaining parameters are shown in the above steps.

[0194] Status Update:

[0195]

[0196] in, is the sound source state of the kth iteration, and the remaining parameters are as in the above steps.

[0197] Covariance update:

[0198] P k|k =(IK k H)P k|k-1 ,

[0199] in, is the sound source covariance of the kth iteration, I is the unit matrix, and the remaining parameters are as in the above steps.

[0200] e. Iteration.

[0201] Repeat steps c and d, and continuously update the state estimation of the sound source as new observation data arrives, thereby achieving dynamic tracking of multiple sound sources.

[0202] 4. Use multi-sound source real-time separation technology to separate multiple sound sources tracked in azimuth to obtain a clear single sound source signal.

[0203] In the present invention, specifically, beamforming technology is used to separate multiple sound sources, and the process is as follows:

[0204] a. System settings.

[0205] Microphone array: There are N microphone elements, and their positions in the three-dimensional environment space are known.

[0206] Sampling frequency: Set the sampling frequency f of the microphone array element s .

[0207] b. Signal model.

[0208] Input signal: Assume that the signal collected by the microphone array at time t is x(t) = [x1(t), x2(t), ..., xN (t)] T , where x N (t) is the sound signal collected by the Nth microphone array element.

[0209] Desired signal direction: Set the direction vector d(θ,φ) of the desired sound source, where θ and φ are the azimuth and elevation angles, respectively.

[0210] c. Beamforming weight calculation.

[0211] Calculate the spatial covariance matrix R of the sound source signal:

[0212] R = E[x(t)x H (t)],

[0213] Among them, x H (t) is the conjugate transpose of x(t), and E represents the mathematical expectation operator.

[0214] Beamforming weights: Calculate the MVDR beamforming weights w MVDR :

[0215]

[0216] Among them, R -1 is the inverse of the spatial covariance matrix R of the sound source signal, d H (θ, φ) is the conjugate transpose of the desired sound source direction vector d(θ, φ).

[0217] d. Signal processing.

[0218] Beamforming output: Apply beamforming weights to the microphone array multi-source input signal x(t) to obtain the separated sound source output signal y(t) in the desired direction.

[0219]

[0220] in, is the MVDR beamforming weight w MVDR The conjugate transpose of .

[0221] e. Dynamic adjustment.

[0222] Real-time update: The covariance matrix and beamforming weights are updated in real time according to the dynamic changes of the sound source.

[0223] f. Multi-sound source processing.

[0224] Repeat the above steps for each sound source direction and calculate the beamforming weights respectively, so as to obtain the separated output signals of each sound source.

[0225] 5. Use sound enhancement technology to enhance each separated single sound source signal in real time to obtain a sound signal with improved quality.

[0226] In the present invention, specifically, each single sound source signal is enhanced using a minimum mean square error estimation algorithm, and the process is as follows:

[0227] a. Signal modeling.

[0228] Assume that the sound source observation signal Y(t) is the superposition of the clean sound source signal X(t) and the noise N(t):

[0229] Y(t)=X(t)+N(t)

[0230] In it, X(t) and N(t) are assumed to be independent random processes, and t is the tth moment.

[0231] b. Short-time Fourier transform (STFT).

[0232] Convert the sound source time domain signal to the frequency domain so that processing can be performed on each frequency component.

[0233] The sound source signal is framed and windowed, and then each frame is Fourier transformed.

[0234] c. Noise estimation.

[0235] Estimate the noise power spectral density (PSD) for each frequency component using the minimum statistic noise estimate.

[0236] d. MMSE gain function calculation.

[0237] The gain function G(k) is calculated for each frequency component, where k represents the frequency index. The MMSE gain function is based on the signal-to-noise ratio (SNR) estimate,

[0238]

[0239] Among them, PriorSNR(k) is the prior signal-to-noise ratio, which is estimated in a decision-oriented manner. Specifically, the prior signal-to-noise ratio can be expressed as:

[0240]

[0241] where E[·] represents the expected value, |X(k)| 2 and |N(k)| 2 are the power spectral densities of the sound source signal and the noise respectively.

[0242] e. Signal estimation.

[0243] Use the gain function to weight each frequency component of the sound source to obtain the enhanced sound source signal.

[0244]

[0245] in, It is the frequency component of the enhanced sound source signal.

[0246] f. Inverse short-time Fourier transform (ISTFT).

[0247] Convert the enhanced sound source frequency domain signal back to the time domain.

[0248] Each frame is inverse Fourier transformed and then overlap-summed to reconstruct the sound source time domain signal.

[0249] g. Further smoothing and filtering the above sound source signal and compressing the dynamic range to reduce artifacts and residual noise, thereby obtaining an enhanced sound source signal.

[0250] Therefore, the present invention uses microphone array technology to efficiently collect sounds in a three-dimensional space environment, and ensures the clarity of sound signals through real-time noise reduction technology. Subsequently, multi-sound source real-time positioning technology and multi-sound source dynamic tracking technology are used to accurately locate and track the positions and dynamic changes of multiple sound sources. In order to further improve the recognition accuracy, multi-sound source real-time separation technology is used to separate sound signals from different sound sources, and sound enhancement technology is used to improve signal quality.

[0251] 3. Based on the sound signal with improved quality, an edge-arranged convolutional attention neural network sound recognition model is used to identify animal species.

[0252] The present invention uses an edge-arranged convolutional attention neural network sound recognition model to recognize the sound signal with improved quality. The convolutional attention neural network sound recognition model is obtained by correspondingly training the convolutional attention neural network, thereby effectively detecting and identifying the animal species category. The specific process is as follows:

[0253] 1. Preprocess the sound signal with improved quality to obtain a preprocessed sound spectrum.

[0254] In the present invention, the sound signal with improved quality is preprocessed as follows: the sound signal with improved quality is sampled and divided into fixed time slices, Fourier transformed to convert it into a spectrum signal, and the spectrum signal is subjected to Mel spectrum transformation with preset coefficients to obtain a preprocessed sound spectrum.

[0255] The Mel spectrum transform is:

[0256]

[0257] Among them, E mis the signal obtained after Mel spectrum transformation, H m (k), (m = 1, 2, ..., M) is the Mel filter bank with preset coefficients, m is the Mel spectrum index, M is the number of Mel transform preset coefficients, z k (k=0,1,...,N-1) is the spectrum signal obtained by the general Fourier transform. k is the frequency index in the frame, and N is the frequency length of each frame.

[0258] Take the logarithm of each spectrum amplitude spectrum to increase the dynamic range of the signal, E′ m =log(E m ), E′ m The result after processing.

[0259] Specifically, the preset learning weights are applied to each spectral component to further enhance the weights within the common spectral intervals of the sounds of the calling animals and reduce the weights of the components outside the common spectral intervals of the sounds of the calling animals, thereby reducing the impact of other sounds. m =E′ m ×(1+α×cos(ω m m+φ)), (m=1,2,...,M),

[0260] Among them, A m is the processed result, α is the enhancement coefficient, which is used to control the enhanced strength, ω m determines the frequency range of the boost, and φ is the phase shift.

[0261] Preset coefficient α, φ is obtained through deep learning on a preset dataset. In the preset dataset, the target sound of interest is compared with other selected sound signals for positive and negative sample learning to determine the frequency ranges for enhancement and reduction of weights.

[0262] 2. Use the convolutional layer, feedforward layer and residual network layer of the convolutional attention neural network sound recognition model to perform noise reduction on the preprocessed sound spectrum to obtain enhanced sound signals of the calling animals of interest.

[0263] In the present invention, the convolutional neural network is further used for intelligent noise reduction, identification and removal of complex nonlinear noise patterns, and enhancement of the sound signals of the singing animals of interest. That is, the pre-processed sound spectrum is input into the first layer of the convolutional neural network, the noise reduction enhancement layer, and the neural network noise reduction enhancement is performed.

[0264] Specifically, the preprocessed sound spectrum of each frame independently passes through the convolution layer and feedforward layer of the neural network, and then uses the residual network layer to connect the residual network with the sound spectrum before the input noise reduction enhancement layer to separate the two signals of the enhanced sound of the animal of interest and the background nonlinear noise.

[0265] Among them, the spectrum diagram of each frame after preprocessing is A W×H , W and H are the width and height of the spectrum graph; the output after the convolution layer and the feedforward layer is:

[0266]

[0267] Among them, conv K (A) is the output of the convolution layer, K is the convolution kernel weight, is the output of the feedforward layer, W f , b is the weight parameter of the feedforward layer.

[0268] The output of the residual network layer is:

[0269]

[0270] Among them, H(Y) is the enhanced sound signal of the animal of interest, N(Y) is the background signal, is the residual connection network, W r is the residual connection network weight.

[0271] Parameter W f ,b and W r It is obtained through comparative learning. Specifically, in a preset data set, the target sound of interest is randomly mixed with background noise collected in the wild, and then compared with the normal sample sound for learning and training.

[0272] 3. Use the multi-layer convolutional layers of the convolutional attention neural network sound recognition model to perform local feature clustering and separation on each frame of the enhanced sound signal of the calling animal of interest to obtain clustered local features.

[0273] After the noise reduction process in step 2, there are still overlapping simultaneous sounding scenes that are difficult to separate simply in complex outdoor scenes, and the existing methods cannot handle them well. In the present invention, for the enhanced sound signal H(Y) of the animal of interest obtained by step 2, each frame is independently input into a multi-layer convolutional neural network to extract the local features within each frame. Different from the existing methods, the local features within each frame support overlapping sounds.

[0274] Specifically, in the neural network training, single and multiple groups of mixed sounds are selected through preset strategies, and compared with the combined labels for clustering learning, the loss function distance between sounds of the same species is minimized, and the loss function distance between sounds of different species is maximized, and single or multiple features after intra-frame clustering are output.

[0275] Let a set of contrastive learning clustering samples be X i , Where X iFor the current sample, For samples of the same species, are samples of different species, and the features they extract are x i , The specific loss function is:

[0276]

[0277] in, and is a function that measures the similarity of sample features. This loss function design makes the similarity between samples of the same species increased and the similarity between samples of different species decreased during the learning process.

[0278] At the same time, in the neural network training, through the preset hybrid strategy, it not only supports the learning and recognition of a single sound of the same species, but also supports the multi-label recognition of multiple superimposed sounds after clustering of the same species.

[0279] 4. Use the attention encoding layer of the convolutional attention neural network sound recognition model to extract the inter-frame global features of the enhanced sound signal of the calling animal of interest to obtain the inter-frame contextual correlation features.

[0280] Traditional and existing methods identify animal species information based on single-frame sound, but have low recognition accuracy and are unable to meet the requirements of identifying a large number of species.

[0281] In the present invention, the features processed and outputted in step 3, that is, the local features after clustering, are further input into the subsequent attention neural network to obtain the features after self-attention interaction, so as to perform subsequent species information recognition, capture the global association of the same type of sounds between frames, and capture the global distinction of different types of sounds between frames.

[0282] That is, between the consecutive frames, the local features after clustering in step 3 are superimposed and positionally encoded, and then mutual attention association is performed between multiple frames to further cluster and optimize the features between multiple frames.

[0283] Specifically, the attention relationship between multiple frames is:

[0284] Attention(x′ (t-i) ,...,x′ (t) ),

[0285] Among them, x′ (t) Current frame superposition position code, x′ (t-i) Overlay position codes for historical frames.

[0286]

[0287] Among them, PE(i) is the position code, To encode the superposition operation, superposition is achieved by dimensional concatenation in the feature vector.

[0288] Specifically, the position code PE(i) is obtained by logarithmic transformation of the time between consecutive frames. The corresponding transformation formula and variables are as follows:

[0289]

[0290] in, i is the sequence index of the historical frame, j is the feature vector dimension index, and d is the feature vector dimension.

[0291] Different from the convolutional neural network and the fully connected layer in the traditional or existing methods, the present invention can capture the feature correlations in a longer time domain.

[0292] 5. Use the attention encoding layer of the convolutional attention neural network sound recognition model to perform feature aggregation on the clustered local features and the previous and next correlation features between frames to obtain aggregated features.

[0293] The attention is calculated for the clustered local features after the preset weights of the attention neural network, and feature aggregation is performed on the clustered local features and the previous and next correlation features between frames based on attention. Sounds of the same type are fused into features with close distances between classes, and sounds of different types are fused into features with far distances between classes.

[0294] 6. Use the decoding layer of the convolutional attention neural network sound recognition model to decode the aggregated features into animal species, so as to obtain animal species, family, genus and species information.

[0295] Traditional or existing methods either obtain species biological information by comparing the extracted vector features with the voiceprint feature library; or obtain species information through the fully connected layer of the convolutional network. However, the former does not support complex field scenes, and the latter only supports species identification of fewer categories. In the present invention, the decoding layer based on the attention neural network directly obtains species family, genus and species information with high accuracy, without relying on matching with the voiceprint model.

[0296] Specifically, during the neural network training process, a preset loss function is designed to obtain attention weights for features with short intra-class distances and long inter-class distances in a preset data set; the aggregated features are directly output as species category labels through a decoding layer.

[0297] During the neural network training process, loss calculation and error propagation are performed through the triplet loss function. The triplet refers to the current label sample, the same type of label sample, and the different type of label sample, in order to learn the feature differences between different types.

[0298] Same as cluster contrast loss, the triplet sample is X i , Where X i For the current sample, For samples of the same species, are samples of different species, and the features they extract are x i , The specific loss function is:

[0299]

[0300] in, is a function that measures the similarity of sample features. This loss function design makes the similarity between samples of the same species increased and the similarity between samples of different species decreased during the learning process.

[0301] Furthermore, through the attention decoding layer, when there are many species, it is easier and better to expand to high-accuracy recognition in a large number of categories than the traditional fully connected layer.

[0302] Specifically, in the design of the preset loss function, not only the high-dimensional cosine distance between the optimized features is calculated, but also the direct angular distance is calculated to make the feature distinction between categories more obvious.

[0303] Let the input feature vector be x and the class label be The weight matrix is ​​W l , the direct error is Instead, optimize θ = arccos(err) and add an angle interval θ m , so that The final loss function is

[0304]

[0305] Where s is the scaling factor, which is used to control the scale during training to make the training process more stable.

[0306] Moreover, in the preset loss function design, the original signal strength index of the feature is added to the loss function weight at the same time, which can distinguish the different learning weights of high-definition samples and low-definition difficult samples, and improve the recognition accuracy of low-definition difficult samples.

[0307] The present invention enhances the target interest sound spectrum interval in signal preprocessing through a neural network, further reduces background noise and separates interest sounds through a convolutional attention neural network, and simultaneously supports the recognition of a single sound and the recognition of superimposed sounds. In the process of species information recognition, a convolutional neural network is used to better capture local features, and an attention neural network captures and associates global features. End-to-end species information decoding is directly performed through an attention decoding layer, thereby improving the availability and accuracy of species information recognition of calling animals in complex wild environments in many aspects.

[0308] Figure 2 FIG. 1 shows a schematic diagram of the animal species identification system based on a microphone array and a sound recognition model of the present invention. Figure 2 As shown, the animal species identification system based on microphone array and sound recognition model of the present invention includes:

[0309] 1. Microphone array.

[0310] The microphone array is used to collect three-dimensional space environmental sounds in real time.

[0311] 2. Sound processing module.

[0312] The sound processing module is used to process the collected three-dimensional space environment sound to obtain a sound signal with improved quality.

[0313] 3. Edge-arranged convolutional attention neural network sound recognition model.

[0314] The edge-arranged convolutional attention neural network sound recognition model is used to identify animal species based on the quality-improved sound signal.

[0315] In addition, the present invention also provides an animal species identification device based on a microphone array and a sound recognition model, which includes: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the animal species identification method based on the microphone array and the sound recognition model as described above.

[0316] Finally, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the animal species identification method based on a microphone array and a sound recognition model as described above are implemented.

[0317] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art can modify or replace the technical solution of the present invention according to the idea of ​​the present invention without departing from the essence and scope of the technical solution of the present invention.

Claims

1. An animal species identification method based on a microphone array and a sound recognition model, characterized in that: The following steps are involved: 1) Use microphone array to collect three-dimensional environmental sound in real time; 2) Processing the collected three-dimensional space environment sound to obtain a sound signal with improved quality; 3) Based on the quality-improved sound signal, an edge-arranged convolutional attention neural network sound recognition model is used to identify animal species.

2. The method for animal species identification based on microphone array and sound recognition model according to claim 1, characterized in that: The step 3) specifically includes: 31) preprocessing the sound signal with improved quality to obtain a preprocessed sound spectrum; 32) using the convolutional layer, feedforward layer and residual network layer of the convolutional attention neural network sound recognition model to perform noise reduction processing on the preprocessed sound spectrum to obtain an enhanced sound signal of the calling animal of interest; 33) using the multi-layer convolutional layers of the convolutional attention neural network sound recognition model to perform local feature clustering and separation on each frame of the enhanced sound signal of the animal of interest to obtain clustered local features; 34) using the attention coding layer of the convolutional attention neural network sound recognition model to extract the inter-frame global features of the enhanced sound signal of the animal of interest, so as to obtain the inter-frame contextual correlation features; 35) using the attention coding layer of the convolutional attention neural network sound recognition model to perform feature aggregation on the clustered local features and the previous and next correlation features between frames to obtain aggregated features; 36) Using the decoding layer of the convolutional attention neural network sound recognition model, the aggregated features are decoded into the family, genus and species of the animal to obtain the family, genus and species information of the animal.

3. The animal species identification method based on microphone array and sound recognition model according to claim 2, characterized in that: The step 31) is specifically as follows: after sampling the sound signal with improved quality and dividing it into fixed time slices, Fourier transform is performed to convert it into a spectrum signal, and Mel spectrum transform is performed on the spectrum signal with preset coefficients to obtain a preprocessed sound spectrum.

4. The method for animal species identification based on microphone array and sound recognition model according to claim 1, characterized in that: The microphone array in step 1) is a spatial spherical microphone array, and the radius R and the number of microphone array elements N of the spatial spherical microphone array satisfy: Among them, d max is the maximum spacing between microphone array elements and d min is the minimum spacing between microphone array elements and C is the speed of sound, f max is the highest frequency picked up by the microphone array element, f min It is the lowest frequency picked up by the microphone array element.

5. The method for animal species identification based on microphone array and sound recognition model according to claim 4, characterized in that: The Cartesian coordinates of the i-th microphone array element in the spatial spherical microphone array are: x i =R×sin(φ i )×cos(θ i ) y i =R×sin(φ i )×sin(θ i ) z i =R×cos(φ i ) Among them, φ i is the polar angle of the i-th microphone array element and θ i is the azimuth angle of the i-th microphone array element and δ is the golden angle and 6. The method for animal species identification based on microphone array and sound recognition model according to claim 1, characterized in that: The step 2) specifically includes: 21) Using adaptive audio noise reduction technology to perform real-time noise reduction on the collected three-dimensional space environment sound; 22) Using multi-sound source real-time positioning technology to locate multiple sound sources in the three-dimensional space environment sound after noise reduction processing; 23) Use multi-sound source dynamic tracking technology to simultaneously track the positions of multiple located sound sources; 24) Use multi-sound source real-time separation technology to separate multiple sound sources tracked in position to obtain a clear single sound source signal; 25) Using sound enhancement technology, each separated single sound source signal is enhanced in real time to obtain a sound signal with improved quality.

7. The method for animal species identification based on microphone array and sound recognition model according to claim 6, characterized in that: In the step 2.1), the collected three-dimensional space environment sound is subjected to real-time noise reduction processing using the least mean square LMS iterative algorithm or the recursive least squares RLS iterative algorithm; In the step 22), for the three-dimensional environment sound after the noise reduction process, the independent component analysis ICA algorithm is first used to separate multiple sound sources, and then the TDOA algorithm is used to locate the sound source for each separated sound source; In the step 23), the Kalman filter algorithm is used to simultaneously track the positions of the multiple located sound sources; In the step 24), beamforming technology is used to separate multiple sound sources tracked in azimuth to obtain a clear single sound source signal; In the step 25), each separated single sound source signal is enhanced in real time using a minimum mean square error estimation algorithm to obtain a sound signal with improved quality.

8. An animal species identification system based on a microphone array and a sound recognition model, characterized in that: include: A microphone array is used to collect three-dimensional ambient sound in real time; A sound processing module, which is used to process the collected three-dimensional space environment sound to obtain a sound signal with improved quality; An edge-arranged convolutional attention neural network sound recognition model for identifying animal species based on the sound signals with improved quality.

9. An animal species identification device based on a microphone array and a sound recognition model, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the animal species identification method based on a microphone array and a sound recognition model as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the animal species identification method based on a microphone array and a sound recognition model as described in any one of claims 1 to 7 are implemented.

Images