A method, medium and system for classifying field sound recordings

Abstract

The present invention provides a method, medium and system for classifying outdoor sound recordings, which belongs to the technical field of sound recognition, and includes: using a sound collection device to obtain basic audio, and obtaining time-frequency data through preprocessing and fast Fourier transformation. Using a sliding window to cluster the time-frequency data to form approximate time-frequency data, and extracting its time-frequency features. Inputting the features into a pre-trained outdoor sound classification model to obtain the probability distribution of animal sound categories in each time window. Based on the probability distribution, continuous time windows are merged to form preliminary sound segments. Confidence comparison and segmentation processing are performed on segments with time overlap, and finally the sound segments are classified and stored according to animal categories, which can effectively solve the complex situation of multiple animals calling at the same time, ensure that each sound segment can accurately represent a complete animal sound, and solve the technical problem of large resource consumption of the outdoor sound classification model in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of call recognition, and in particular, relates to a method, medium and system for classifying field call records. Background Art

[0002] In the application scenarios of modern ecological monitoring and wildlife protection, the automatic identification and classification of wild animal calls has become an important technical requirement. By classifying and analyzing animal calls, researchers can understand the distribution of animals and population dynamics in different regions, and provide a scientific basis for formulating effective protection strategies. At the same time, this technology can also be applied to smart agriculture and forestry management to help monitor and warn animal activities in agricultural and forestry areas to prevent the destruction of crops and forest resources. Existing animal call classification technology is mainly based on artificial intelligence and machine learning methods. First, the original sound data in the wild environment is collected by recording equipment, and then the characteristics of the sound signal, such as spectrum, frequency change, amplitude, etc., are extracted by time-frequency analysis and other means. Next, these features are input into a pre-trained classification model, such as a hidden Markov model (HMM), a support vector machine (SVM) or a convolutional neural network (CNN), and the animal call category corresponding to each audio clip is output.

[0003] Although these methods perform well in laboratory environments, they still face some challenges in actual field monitoring. First, there is often strong background noise in the wild environment, such as wind, rain and other animal calls. These noises will interfere with the extraction of sound features, thereby affecting classification accuracy. Secondly, it is also common to have multiple animal calls at the same time, which requires the classification system to have strong robustness and be able to accurately distinguish and identify the calls of different animals. In addition, the existing technology often only uses a pre-trained model for analysis after simple pre-processing of field audio. The model consumes a lot of resources and is difficult to run on field embedded devices. Summary of the invention

[0004] In view of this, the present invention provides a method, medium and system for classifying field sound records, which can solve the technical problem of large resource consumption of field sound classification models in the prior art.

[0005] The present invention is achieved in that:

[0006] A first aspect of the present invention provides a method for classifying field sound recordings, which comprises the following steps:

[0007] S10, obtaining the audio collected by the sound collection device and recording it as basic audio;

[0008] S20, performing fast Fourier transform on the basic audio after preprocessing to obtain time-frequency data;

[0009] S30, using a time sliding window of a first length, clustering the time-frequency data in each time sliding window to obtain a plurality of cluster clusters in each time sliding window;

[0010] S40, replacing the corresponding time-frequency point with the cluster center of each cluster to form approximate time-frequency data;

[0011] S50, using a time sliding window of a second length to extract time-frequency features of the approximate time-frequency data;

[0012] S60, inputting the time-frequency features of the approximate time-frequency data into a pre-trained field sound classification model to obtain a probability distribution of animal sound categories within each time sliding window;

[0013] S70, based on the probability distribution obtained in S60, merging the continuous time windows to form preliminary sound segments, each segment including a start time, an end time, an animal category and a confidence level;

[0014] S80, for the chirping segments that overlap in time, compare the confidences of the overlapping segments; if the confidence difference between the overlapping segments is greater than or equal to a preset confidence difference threshold, retain the segment with high confidence and delete the segment with low confidence; if the confidence difference between the overlapping segments is less than the preset confidence difference threshold, segment the overlapping area according to the midpoint in time, and attribute the segmented parts to two adjacent segments respectively;

[0015] S90, classify and store the processed sound segments according to animal categories.

[0016] This method simplifies time-frequency data through clustering and approximation, reduces computational complexity, and can be applied to the built-in chip of infrared cameras, consuming little computing resources. Using two time windows of different lengths can capture features over a longer time range while maintaining temporal resolution. The final overlapping processing and classification storage steps ensure accurate classification and effective management of complex scenes.

[0017] Specifically, the step S10 specifically includes: obtaining the original audio data collected by the sound collection device as the basic audio, and performing a preprocessing operation on the basic audio. The preprocessing operation at least includes: resampling, removing DC components and noise reduction. The purpose of resampling is to unify the sampling frequency of the audio signal to facilitate subsequent signal processing. Removing DC components is to eliminate DC offsets in the audio signal. Noise reduction is to remove environmental noise and improve signal quality. After these preprocessing operations, the basic audio data is ready for further analysis.

[0018] Wherein, the step S20 specifically performs a fast Fourier transform (FFT) on the pre-processed basic audio signal. Fast Fourier transform can convert the audio signal in the time domain into the frequency domain to obtain time-frequency data. The time-frequency data contains information about the audio signal in two dimensions, time and frequency, and lays the foundation for subsequent feature extraction and animal sound classification.

[0019] In step S30, a time sliding window of a first length is used to perform cluster analysis on the time-frequency data. The clustering algorithm used here is K-means clustering, and the value of K is usually set to 3-5, indicating the maximum number of animals expected to make sounds at the same time. The distance metric uses the Euclidean distance. Through this step, multiple clusters can be obtained in each time sliding window.

[0020] In step S40, the corresponding time-frequency point is replaced by the cluster center of each cluster, thereby forming approximate time-frequency data. The purpose of this step is to simplify and compress the time-frequency data and reduce the complexity of subsequent calculations. The cluster center can well represent the characteristics of the internal data of each cluster, laying the foundation for the next step of feature extraction.

[0021] Wherein, in step S50, a time sliding window of the second length is used to extract the time-frequency features of the approximate time-frequency data. The time-frequency features mentioned here include but are not limited to: spectrum envelope, frequency change rate, amplitude change rate, etc. Compared with the first length time window used in step S30, the second length time window used here is usually 2-4 times the first length. For example, if the first length is 50 milliseconds, the second length can be set to 100-200 milliseconds. The purpose of this design is to capture the sound features within a longer time range while maintaining a high time resolution.

[0022] Wherein, in step S60, the time-frequency features extracted in step S50 are input into a pre-trained wild sound classification model to obtain the probability distribution of animal sound categories in each time sliding window. The classification model used here is a lightweight convolutional neural network structure, and the construction process of its training data set includes: collecting recordings of calls of various wild animals, segmenting the recordings to extract single animal sound segments, performing time-frequency transformation on each sound segment to generate time-frequency data, performing data enhancement on the time-frequency data, annotating the processed data with animal categories, and dividing the data set into a training set, a validation set, and a test set. The data set constructed in this way can train a robust wild sound classification model.

[0023] In step S70, based on the probability distribution obtained in step S60, continuous time windows are merged to form preliminary sound segments. Each sound segment contains a start time, an end time, an animal category, and a confidence level. The purpose of this step is to aggregate the scattered time-frequency features into a complete sound segment, so as to prepare for subsequent segment processing and classification storage.

[0024] Among them, in step S80, for the overlapping sound segments in time, the confidence of the overlapping segments is compared, and processing is performed according to the confidence difference. Specifically, if the confidence difference between the overlapping segments is greater than or equal to the preset confidence difference threshold, the segment with high confidence is retained and the segment with low confidence is deleted; if the confidence difference between the overlapping segments is less than the preset confidence difference threshold, the overlapping area is segmented according to the midpoint of time, and the segmented parts are respectively attributed to the two adjacent segments. The purpose of this step is to solve the overlap problem between the sound segments and ensure that each sound segment can accurately represent a complete animal sound.

[0025] The relationship between the first length and the second length is: the second length is greater than the first length, usually 2-4 times of the first length.

[0026] For example, if the first length is 50 milliseconds, the second length can be set to 100-200 milliseconds. This design can capture sound features in a longer time range while maintaining a high time resolution.

[0027] Furthermore, the clustering step uses the K-means clustering algorithm, with k set to the expected maximum number of animals making sounds at the same time.

[0028] k is usually 3-5, and the distance metric is Euclidean distance;

[0029] Furthermore, the field sound classification model adopts a lightweight convolutional neural network structure.

[0030] Furthermore, the steps of constructing the training data set of the field sound classification model specifically include:

[0031] Step 1601, collecting recordings of calls of various wild animals;

[0032] Step 1602: segment the recording to extract a single animal sound segment;

[0033] Step 1603: Perform time-frequency transformation on each sound segment to generate time-frequency data;

[0034] Step 1604: perform data enhancement on the time-frequency data;

[0035] Step 1605: label the processed data with animal categories;

[0036] Step 1606: Divide the data set into a training set, a validation set, and a test set.

[0037] Data enhancement for time-frequency data includes adding noise, time stretching, etc. The field call classification model is obtained by training using the training data set.

[0038] Furthermore, the step of obtaining the confidence difference threshold is specifically:

[0039] Step 1801: Run the trained model on the validation set to obtain all prediction results;

[0040] Step 1802, calculating the average confidence of correctly classified samples;

[0041] Step 1803, calculating the average confidence of the misclassified samples;

[0042] Step 1804: take half of the difference between the two average values ​​as the initial threshold;

[0043] Step 1805: Experiment with different thresholds on the validation set and select a threshold that can maximize the classification accuracy;

[0044] Step 1806: Verify the validity of the threshold on the test set.

[0045] Furthermore, the basic audio preprocessing at least includes: resampling, DC component removal, and noise reduction.

[0046] Furthermore, the first length is 0.5 seconds, and the second length is 2 seconds.

[0047] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run in a computer, they are used to execute the above-mentioned method for classifying field sound records.

[0048] A third aspect of the present invention provides a field sound recording classification system, which includes the above-mentioned computer-readable storage medium.

[0049] Compared with the prior art, the method, medium and system for classifying field sound recordings provided by the present invention have the following beneficial effects: 1. The computational complexity is reduced, and efficient operation can be achieved on embedded systems. By adopting time-frequency clustering and approximation, the present invention greatly reduces the dimension of the original time-frequency data, thereby reducing the amount of computation for subsequent feature extraction and classification. This enables the solution to be applied to resource-constrained devices such as infrared cameras, meeting the real-time requirements of field monitoring applications.

[0050] 2. Improved classification accuracy, especially in complex multi-sound environments. The present invention adopts a dual time window mechanism, which can capture sound features in a longer time range while maintaining high time resolution. This method can better describe the time-frequency characteristics of animal sounds, thereby improving classification accuracy. At the same time, by processing overlapping sound segments, the present invention further enhances robustness in complex scenarios.

[0051] 3. It can realize automated and unattended field monitoring. Compared with the traditional machine learning method that relies on manually labeled data, the technical solution of the present invention can fully automatically classify and manage the collected chirping data, greatly improving the working efficiency of the monitoring system. This lays the foundation for its wide application in the field of field ecological monitoring and protection.

[0052] In general, the field sound recording classification method proposed in the present invention solves the shortcomings of existing animal sound classification methods in terms of computational complexity, classification accuracy and application scenarios by innovatively combining time-frequency analysis, cluster approximation and dual time window technology. It provides a practical technical solution for field ecological monitoring and protection, and solves the technical problem of large resource consumption of field sound classification models in existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flow chart of the method provided by the present invention;

[0054] Figure 2 This is a resource consumption comparison chart;

[0055] Figure 3 This is a comparison chart of processing time;

[0056] Figure 4 This is a comparison chart of the relationship between the amount of training data and the model inference accuracy;

[0057] Figure 5 This is a comparison chart of the recognition accuracy of different animals. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0059] like Figure 1 As shown, it is a flow chart of a method for classifying field sound records provided by the present invention. The method comprises the following steps:

[0060] S10, obtaining the audio collected by the sound collection device and recording it as basic audio;

[0061] S20, performing fast Fourier transform on the basic audio after preprocessing to obtain time-frequency data;

[0062] S30, using a time sliding window of a first length, clustering the time-frequency data in each time sliding window to obtain a plurality of cluster clusters in each time sliding window;

[0063] S40, replacing the corresponding time-frequency point with the cluster center of each cluster to form approximate time-frequency data;

[0064] S50, using a time sliding window of a second length to extract time-frequency features of the approximate time-frequency data;

[0065] S60, inputting the time-frequency features of the approximate time-frequency data into a pre-trained field sound classification model to obtain a probability distribution of animal sound categories within each time sliding window;

[0066] S70, based on the probability distribution obtained in S60, merging the continuous time windows to form preliminary sound segments, each segment including a start time, an end time, an animal category and a confidence level;

[0067] S80, for the chirping segments that overlap in time, compare the confidences of the overlapping segments; if the confidence difference between the overlapping segments is greater than or equal to a preset confidence difference threshold, retain the segment with high confidence and delete the segment with low confidence; if the confidence difference between the overlapping segments is less than the preset confidence difference threshold, segment the overlapping area according to the midpoint in time, and attribute the segmented parts to two adjacent segments respectively;

[0068] S90, classify and store the processed sound segments according to animal categories.

[0069] The specific implementation methods of the above steps are described in detail below:

[0070] The specific implementation method of step S10 is to first obtain the original audio data collected by the sound collection device. The original audio data is the basic audio. After obtaining the basic audio, some preprocessing operations need to be performed on it. The preprocessing operations at least include: resampling, DC component removal and noise reduction. The purpose of resampling is to unify the sampling frequency of the audio signal to facilitate subsequent signal processing. The purpose of removing the DC component is to eliminate the DC offset in the audio signal. Noise reduction is to remove environmental noise and improve signal quality. After these preprocessing operations, the basic audio data is ready for further analysis.

[0071] The specific implementation of step S20 is to perform a fast Fourier transform (FFT) on the preprocessed basic audio signal. Fast Fourier transform can convert the audio signal in the time domain to the frequency domain to obtain time-frequency data. The time-frequency data contains information about the audio signal in two dimensions, time and frequency, which lays the foundation for subsequent feature extraction and animal sound classification.

[0072] The specific implementation of step S30 is to perform cluster analysis on the time-frequency data using a time sliding window of a first length. The clustering algorithm used here is K-means clustering. K-means clustering is an unsupervised learning algorithm that can divide the time-frequency data into multiple clusters according to their similarity. The value of K is usually set to 3-5, indicating the maximum number of animals expected to make sounds at the same time. The distance metric uses the Euclidean distance. Through this step, multiple clusters within each time sliding window can be obtained.

[0073] The specific implementation of step S40 is to replace the corresponding time-frequency point with the cluster center of each cluster, thereby forming approximate time-frequency data. The purpose of this step is to simplify and compress the time-frequency data and reduce the complexity of subsequent calculations. The cluster center can well represent the characteristics of the internal data of each cluster, thus laying the foundation for the next step of feature extraction.

[0074] The specific implementation of step S50 is to use a time sliding window of the second length to extract the time-frequency features of the approximate time-frequency data. The time-frequency features mentioned here include but are not limited to: spectrum envelope, frequency change rate, amplitude change rate, etc. Compared with the first length time window used in step S30, the second length time window used here is usually 2-4 times the first length. For example, if the first length is 50 milliseconds, the second length can be set to 100-200 milliseconds. The purpose of this design is to capture the sound features within a longer time range while maintaining a high time resolution.

[0075] The specific implementation of step S60 is to input the time-frequency features extracted in step S50 into a pre-trained field sound classification model to obtain the probability distribution of animal sound categories in each time sliding window. The classification model used here is a lightweight convolutional neural network structure. The training data set construction process of the model is as follows:

[0076] 1. Collect recordings of various wild animal calls;

[0077] 2. Segment the recording and extract the single animal sound segment;

[0078] 3. Perform time-frequency transformation on each sound segment to generate time-frequency data;

[0079] 4. Perform data enhancement on time-frequency data, such as adding noise, time stretching, etc.;

[0080] 5. Label the processed data with animal categories;

[0081] 6. Divide the dataset into training set, validation set and test set.

[0082] Through such a dataset construction process, a robust field call classification model can be trained. The model can output the category probability distribution of animal calls in each time sliding window, providing a basis for the subsequent merging and classification of call segments.

[0083] The specific implementation of step S70 is to merge continuous time windows based on the probability distribution obtained in step S60 to form preliminary sound segments. Each sound segment contains the following information: start time, end time, animal category and confidence. The purpose of this step is to aggregate the scattered time-frequency features into a complete sound segment, so as to prepare for the subsequent segment processing and classification storage.

[0084] The specific implementation of step S80 is to compare the confidences of the overlapping sound segments in time, and perform processing according to the confidence differences. Specifically:

[0085] 1. If the confidence difference between overlapping segments is greater than or equal to the preset confidence difference threshold, the segments with high confidence are retained and the segments with low confidence are deleted.

[0086] 2. If the confidence difference of the overlapping segments is less than the preset confidence difference threshold, the overlapping area is segmented according to the midpoint of time, and the segmented parts are respectively attributed to the two adjacent segments.

[0087] The purpose of this step is to solve the problem of overlap between the sound segments and ensure that each sound segment can accurately represent a complete animal sound. The confidence difference threshold is obtained as follows:

[0088] 1. Run the trained model on the validation set to obtain all prediction results;

[0089] 2. Calculate the average confidence of correctly classified samples;

[0090] 3. Calculate the average confidence of misclassified samples;

[0091] 4. Take half of the difference between the two averages as the initial threshold;

[0092] 5. Experiment with different thresholds on the validation set and select the threshold that maximizes the classification accuracy;

[0093] 6. Verify the effectiveness of the threshold on the test set.

[0094] The setting of this threshold can ensure that different animal calls are distinguished as accurately as possible while retaining segments with higher confidence.

[0095] The specific implementation of step S90 is to classify and store the processed sound segments according to animal categories. The purpose of this step is to facilitate subsequent sound analysis and management. By classifying and storing the sound segments according to animal categories, users can quickly find and analyze relevant sound information for the animal categories they are interested in.

[0096] In general, this method of classifying field call records simplifies time-frequency data through clustering and approximation, reduces computational complexity, and can be applied to the built-in chip of infrared cameras, consuming little computing resources. Using two time windows of different lengths can capture features over a longer time range while maintaining temporal resolution. The final overlapping processing and classification storage steps ensure accurate classification and effective management of complex scenes.

[0097] In order to better understand and implement the present invention, a specific embodiment 1 of the method provided by the first aspect of the present invention is provided below. The specific implementation of the steps in this embodiment 1 is described in detail as follows: First, in step S10, it is necessary to obtain the original audio data collected by the sound collection device, that is, the basic audio x(t). In order to further process these audio signals, some preprocessing operations need to be performed on them. The preprocessing operations at least include: resampling, removing DC components and noise reduction.

[0098] The purpose of resampling is to change the sampling frequency f of the audio signal s Unification facilitates subsequent signal processing. Assume that the original audio sampling frequency is f so , then the sampling frequency after resampling is f s , can be calculated by the following formula:

[0099]

[0100] Where r is the resampling factor, which is an integer greater than 0. After resampling, the original audio signal x(t) will be converted into a discrete time series x[n], where n is the discrete time index.

[0101] The purpose of removing the DC component is to eliminate the DC offset in the audio signal. The DC component can be eliminated by the following formula:

[0102]

[0103] in, is the mean value of x[n], representing the DC component.

[0104] Noise reduction is to remove environmental noise and improve signal quality. Here we can use a noise reduction method based on spectral subtraction, and its formula is as follows:

[0105]

[0106] Where X[k] is the discrete Fourier transform of x[n], N x [k] is the noise power spectrum, and α is the empirical coefficient used to adjust the degree of suppression. This method can effectively remove environmental noise and improve signal quality.

[0107] After these preprocessing operations, the basic audio data x'n is ready for further analysis.

[0108] In step S20, it is necessary to perform a fast Fourier transform (FFT) on the preprocessed basic audio signal x'n to obtain time-frequency data X'k,m. Fast Fourier transform can convert the audio signal in the time domain to the frequency domain, and its formula is as follows:

[0109] Among them, k is the frequency index, m is the time frame index, and N is the FFT length. The time-frequency data X'k,m contains the information of the audio signal in two dimensions, time and frequency, laying the foundation for subsequent feature extraction and animal sound classification.

[0110] In step S30, a time sliding window of the first length is used to perform cluster analysis on the time-frequency data X'k,m. The clustering algorithm used here is K-means clustering, and its objective function is as follows:

[0111]

[0112] Among them, K is the number of clusters, C i is the i-th cluster, μ i is the center of the ith cluster, ||x-μ i || is the Euclidean distance. The value of K is usually set to 3-5, indicating the maximum number of animals expected to make sounds at the same time. Through this step, multiple clusters can be obtained within each time sliding window.

[0113] In step S40, the cluster center μ of each cluster is i Replace the corresponding time-frequency point X'k,m, thus forming an approximate time-frequency data The purpose of this step is to simplify and compress the time-frequency data to reduce the complexity of subsequent calculations. i It can well represent the characteristics of the data within each cluster, thus laying the foundation for the next step of feature extraction. Approximate time-frequency data The calculation formula is as follows:

[0114]

[0115] Among them, F(x∈C i ) is an indicator function, indicating that when x belongs to the i-th cluster, it takes the value 1, otherwise it takes the value 0. In step S50, a time sliding window of the second length is used to extract the approximate time-frequency data The time-frequency characteristics mentioned here include but are not limited to: spectrum envelope A[m], frequency change rate F'[m] and amplitude change rate A'[m]. Their calculation formulas are as follows:

[0116]

[0117] Wherein, F0 is the center frequency of the signal. Compared with the first length time window used in step S30, the second length time window used here is usually 2-4 times the first length, for example, the first length is 50 milliseconds, and the second length can be set to 100-200 milliseconds. The purpose of this design is to capture the sound features in a longer time range while maintaining a high time resolution.

[0118] In step S60, the time-frequency features {A[m], F′[m], A′[m]} extracted in step S50 are input into a pre-trained field sound classification model to obtain the animal sound category probability distribution P(y|X′[k,m]) in each time sliding window. The classification model used here is a lightweight convolutional neural network structure, and its network structure is as follows:

[0119] Convolutional layer 1:

[0120] Pooling layer 1: y = max{y1,y2,...,y n};

[0121] Convolutional layer 2:

[0122] Pooling layer 2: z = max{z1,z2,...,z m};

[0123] Fully connected layer:

[0124] Output layer: P(y|X′[k,m])=softmax(W4z+b4);

[0125] Among them, σ is the activation function, such as ReLU function, W i and b iis a learnable parameter. This network structure can effectively extract the nonlinear pattern of time-frequency features and obtain a more accurate probability distribution of animal sound categories.

[0126] The construction process of the training data set for this classification model is as follows:

[0127] 1. Collect recordings of various wild animal calls (i) (t), where i is the sample index;

[0128] 2. Segment the recording and extract the single animal sound segment x (i) [n];

[0129] 3. Perform time-frequency transformation on each sound segment to generate time-frequency data X (i) [k,m];

[0130] 4. Perform data enhancement on time-frequency data, such as adding noise, time stretching, etc., to obtain enhanced time-frequency data

[0131] 5. Label the processed data with animal category y (i) ;

[0132] 6. Divide the dataset into training set, validation set and test set.

[0133] The dataset constructed in this way can be used to train a robust model for field call classification.

[0134] In step S70, based on the probability distribution P(y|X′[k,m]) obtained in step S60, the continuous time windows are merged to form preliminary chirping segments. j Contains the following information:

[0135] 1. Start time

[0136] 2. End time

[0137] 3. Animal category j ;

[0138] 4. Confidence

[0139] The purpose of this step is to aggregate the scattered time-frequency features into complete sound segments, preparing for subsequent segment processing and classification storage.

[0140] In step S80, for the chirping segments S that overlap in time, i and S j , compare their confidence c i and cj , and processed according to the confidence difference. Specifically:

[0141] 1. If |c i -c j |≥θ, where θ is the preset confidence difference threshold, the segments with high confidence are retained and the segments with low confidence are deleted.

[0142] 2. If |c i -c j |<θ, the overlapping area is divided according to the midpoint of time, and the divided parts are respectively attributed to the two adjacent segments.

[0143] The purpose of this step is to solve the problem of overlap between the sound segments and ensure that each sound segment can accurately represent a complete animal sound. The confidence difference threshold θ is obtained as follows:

[0144] 1. Run the trained model on the validation set to obtain all prediction results;

[0145] 2. Calculate the average confidence of correctly classified samples

[0146] 3. Calculate the average confidence of misclassified samples

[0147] 4. Take half of the difference between the two averages as the initial threshold

[0148] 5. Experiment with different thresholds θ on the validation set and select the threshold θ that maximizes the classification accuracy;

[0149] 6. Verify the effectiveness of the threshold on the test set.

[0150] The setting of this threshold can ensure that different animal calls are distinguished as accurately as possible while retaining segments with higher confidence.

[0151] In step S90, the processed sound segment S j By animal category j The purpose of this step is to facilitate the subsequent analysis and management of the calls. By classifying and storing the call segments according to animal categories, users can quickly find and analyze relevant call information for the animal category of interest.

[0152] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run in a computer, they are used to execute the above-mentioned method for classifying field sound records.

[0153] A third aspect of the present invention provides a field sound recording classification system, which includes the above-mentioned computer-readable storage medium.

[0154] Specifically, the principle of the present invention is:

[0155] First, this method greatly improves the quality of the sound signal by preprocessing the original audio data, including resampling, DC component removal and noise reduction, laying the foundation for subsequent time-frequency analysis and feature extraction. These preprocessing operations effectively eliminate the interference of environmental noise, making the subsequent classification process more stable and reliable.

[0156] Secondly, this method uses fast Fourier transform (FFT) to convert the time domain audio signal into the time-frequency domain, and obtains time-frequency data that describes the characteristics of the sound signal in two dimensions: time and frequency. The time-frequency data contains rich audio feature information, which provides a good basis for the subsequent classification of animal sounds.

[0157] Next, the method cleverly uses time-frequency clustering and approximation techniques to significantly reduce the dimension and complexity of time-frequency data. Specifically, the K-means clustering algorithm is first used to divide the time-frequency points in each time window into multiple clusters, and then the center point of each cluster is used to replace the original time-frequency data. This clustering approximation method not only greatly reduces the amount of calculation, but also retains the key features of the sound signal, laying a good foundation for subsequent feature extraction and classification.

[0158] Another innovation is that the method uses a dual time window mechanism. Compared with the traditional single time window, the present invention uses two time windows of different lengths. The first shorter time window is used for time-frequency clustering, maintaining a high time resolution; while the second longer time window is used to extract time-frequency features, which can capture sound features over a longer time range. This design balances time resolution and feature description capabilities, further improving classification accuracy.

[0159] Finally, the method also greatly enhances its applicability in complex scenarios through intelligent processing of overlapping call segments. Specifically, for call segments that overlap in time, the method will classify them according to their confidence, retaining the segments with high confidence and deleting the segments with low confidence. For overlapping segments with similar confidence, the overlapping area will be split at the midpoint and assigned to two adjacent segments respectively. This method can effectively solve the complex situation of multiple animals calling at the same time, ensuring that each call segment can accurately represent a complete animal sound.

[0160] In order to further better understand and implement the present invention, the following is an example 2 of a specific application scenario of the present invention: In a wildlife reserve, researchers urgently need to understand the distribution and activities of different animals in the area to provide a basis for formulating effective protection strategies. After investigation, the following animals are mainly found in the reserve: black bears, gorals, gibbons and white cranes. In order to comprehensively monitor the activities of these animals, researchers decided to deploy multiple sets of infrared cameras in the reserve, each of which integrates a sound collection device and a single-chip microcomputer, and the single-chip microcomputer is provided with a computer-readable storage medium or system provided by the present invention.

[0161] After the deployment was completed, the researchers began to run the field sound recording classification method proposed in the present invention to automatically identify and classify the collected sound data. The whole process is as follows:

[0162] 1. Data preprocessing

[0163] First, the system preprocesses the raw audio data x(t) collected by each sound collection device. The preprocessing operations include: resampling, DC component removal and noise reduction.

[0164] The purpose of resampling is to convert the original audio signal (sampling frequency is ) unified to f s =16kHz to facilitate subsequent time-frequency analysis. The resampling factor is r=3, so the resampled discrete time series x[n] can be expressed as:

[0165] x[n]=x(nt / f s );

[0166] Where n is the discrete time index.

[0167] Next, the system will eliminate the DC component of x[n], the formula is as follows:

[0168]

[0169] in, is the mean of x[n].

[0170] Finally, the system uses a spectral subtraction method to reduce the noise of x'[n], the formula is as follows:

[0171]

[0172] Where X[k] is the discrete Fourier transform of x'[n], N x [k] is the noise power spectrum, and α=2 is the empirical coefficient.

[0173] After these preprocessing operations, the base audio data x'[n] is already prepared for subsequent time-frequency analysis.

[0174] 2. Time-Frequency Analysis

[0175] Next, the system will perform a Fast Fourier Transform (FFT) on the preprocessed base audio signal x'[n] to obtain time-frequency data X'[k,m]. The calculation formula of the FFT is as follows:

[0176]

[0177] where k is the frequency index (0 ≤ k < N / 2), m is the time frame index, and N = 512 is the FFT length.

[0178] 3. Time-Frequency Clustering

[0179] To reduce the computational complexity of subsequent feature extraction and classification, the system adopts time-frequency clustering and approximation methods. First, use a time sliding window with a first length of T1 = 50ms to perform K-means clustering on the time-frequency data X'[k,m], and its objective function is as follows:

[0180]

[0181] where K = 4 is the expected maximum number of simultaneously vocalizing animals, C i is the i-th cluster, and μ i is the center of the i-th cluster. Through this step, the system can obtain 4 cluster clusters within each time sliding window.

[0182] 4. Time-Frequency Approximation

[0183] Based on the clustering results obtained in the previous step, the system uses the center point μ i of each cluster to replace the original time-frequency data X'[k,m], thereby forming approximate time-frequency data

[0184]

[0185] where F(X′∈C i ) is the indicator function, which takes the value 1 when X' belongs to the i-th cluster and 0 otherwise. This clustering approximation method significantly reduces the dimension and complexity of the time-frequency data, laying the foundation for subsequent feature extraction and classification.

[0186] 5. Feature Extraction

[0187] The system uses a second time sliding window with a length of T2 = 150ms to extract the approximate time-frequency data These features include: spectrum envelope A[m], frequency change rate F'[m] and amplitude change rate A'[m]. The specific calculation formula is as follows:

[0188]

[0189] Among them, F0=2kHz is the center frequency of the signal.

[0190] 6. Classification of animal sounds

[0191] The system inputs the time-frequency features {A[m], F′[m], A′[m]} extracted in step 5 into the pre-trained lightweight convolutional neural network model to obtain the probability distribution P(y|X′[k,m]) of the animal sound category in each time sliding window. The network structure of the classification model is shown in Table 1 below:

[0192] Table 1 Network structure diagram of classification model

[0193] Layer Type Input size Output size Number of parameters Convolutional layer 1 1×64×3 32×32×32 896 Pooling layer 1 32×32×32 32×16×16 0 Convolutional Layer 2 32×16×16 64×8×8 18,496 Pooling layer 2 64×8×8 64×4×4 0 Fully connected layer 1024 128 131,200 Output Layer 128 4 516

[0194] The input size is 1×64×3, which means a single channel, 64 time frames, and 3 feature channels (spectral envelope, frequency change rate, and amplitude change rate). The output size is 4, corresponding to the probability distribution of 4 animal sound categories (black bear, goral, gibbon, and white crane). The model has a total of about 150,000 trainable parameters and is a lightweight structure, which is very suitable for deployment on embedded devices for real-time reasoning.

[0195] The training data set construction process of the model is as follows:

[0196] 1. Collect 6,000 samples of calls from four species of animals (black bear, goral, gibbon and white crane);

[0197] 2. Segment the recording, extract the single animal sound segment, and obtain 6000 time-frequency data samples X (i) [k,m];

[0198] 3. Perform data enhancement on time-frequency data, such as adding Gaussian noise, time stretching, etc., to obtain enhanced data

[0199] 4. Label the processed data with animal category y (i) ;

[0200] 5. Divide the dataset into training set (4000 samples), validation set (1000 samples) and test set (1000 samples).

[0201] 7. Merging and processing of call segments

[0202] Based on the animal sound category probability distribution P(y|X′[k,m]) obtained in step 6, the system merges continuous time windows to form a preliminary sound segment S j Each sound segment S j Contains the following information:

[0203] Start time

[0204] End Time

[0205] Animal Category j ;

[0206] Confidence

[0207] On this basis, the system further processes the chirping segments with time overlap. Specifically, if two overlapping segments S i and S j The confidence difference of i -c j |≥θ=0.2, the segments with higher confidence are retained and the segments with lower confidence are deleted; if |c i -c j |<0.2, the overlapping area is divided according to the time midpoint, and the divided parts are respectively assigned to two adjacent segments. This can effectively solve the situation where multiple animals call at the same time, ensuring that each call segment can accurately represent a complete animal sound.

[0208] 8. Categorized storage

[0209] Finally, the system will process the sound segment S j By animal category j Classified storage is carried out to facilitate subsequent animal activity analysis and protection strategy formulation.

[0210] After the above 8 steps, the system successfully performed automatic sound recognition and classification on the sound data collected in the protected area. Let's take a look at the performance of the entire system in actual application:

[0211] 1. Computational complexity

[0212] By using time-frequency clustering and approximation methods, the system significantly reduces the dimension and complexity of time-frequency data. Taking a 10-minute audio clip as an example, the original time-frequency data size is 4096×30000, and after clustering and approximation, the size of the approximate time-frequency data is only 64×4000, a reduction of about 93%. This enables the method to run in real time on embedded devices such as infrared cameras, meeting the needs of field monitoring applications.

[0213] 2. Classification accuracy

[0214] In the test set in the reserve, the system's animal call classification accuracy reached 82.3%, close to the traditional machine learning-based method (about 86%). This is mainly due to the use of a dual time window mechanism in the method, which not only maintains a high time resolution, but also can capture sound features over a longer time range. At the same time, the intelligent processing of overlapping call segments also greatly improves the robustness of the system in complex scenarios.

[0215] The following is a comparison of the method of the present invention and the conventional neural network (such as CNN, etc.) method in the prior art to achieve field sound classification in Example 2: Figure 2 This is a resource consumption comparison chart, which shows the comparison between the CPU usage, memory usage and storage usage of the method of the present invention and the conventional neural network method. It can be seen that the resource consumption of the method of the present invention is significantly lower than that of the conventional method, reflecting its superior computing efficiency. Figure 3 This is a processing time comparison chart, which compares the time required for the two methods to process one minute of audio. The method of the present invention only takes 0.5 seconds, while the conventional method takes 2.3 seconds, which fully demonstrates the efficiency of the method of the present invention, which is crucial for real-time processing of large amounts of field audio data. Figure 4 This is a comparison chart of the relationship between the amount of training data and the model inference accuracy, which shows the classification accuracy change trend of the two methods as the amount of training data increases. It can be seen that the method of the present invention not only maintains a high accuracy rate under various data amounts, but also improves its performance more significantly as the amount of data increases, reflecting the method's ability to efficiently utilize data. Figure 5 This is a comparison chart of the recognition accuracy of different animals. This chart compares the accuracy of the two methods in recognizing the calls of different animals, and the processing effect is not much different from that of the conventional neural network method.

[0216] In summary, based on the field call recording classification method proposed in this invention, researchers have successfully built an intelligent and efficient animal activity monitoring system, which effectively supports the management and protection of the wildlife reserve. The system has made significant progress in reducing computational complexity, improving classification accuracy, and expanding application scenarios, providing a practical technical solution for field ecological monitoring and protection.

[0217] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for classifying field sound recordings, characterized in that: The following steps are involved: S10, obtaining the audio collected by the sound collection device and recording it as basic audio; S20, performing fast Fourier transform on the basic audio after preprocessing to obtain time-frequency data; S30, using a time sliding window of a first length, clustering the time-frequency data in each time sliding window to obtain a plurality of cluster clusters in each time sliding window; S40, replacing the corresponding time-frequency point with the cluster center of each cluster to form approximate time-frequency data; S50, using a time sliding window of a second length to extract time-frequency features of the approximate time-frequency data; S60, inputting the time-frequency features of the approximate time-frequency data into a pre-trained field sound classification model to obtain a probability distribution of animal sound categories within each time sliding window; S70, based on the probability distribution obtained in S60, merging the continuous time windows to form preliminary sound segments, each segment including a start time, an end time, an animal category and a confidence level; S80, for the chirping segments that overlap in time, compare the confidences of the overlapping segments; if the confidence difference between the overlapping segments is greater than or equal to a preset confidence difference threshold, retain the segment with high confidence and delete the segment with low confidence; if the confidence difference between the overlapping segments is less than the preset confidence difference threshold, segment the overlapping area according to the midpoint in time, and attribute the segmented parts to two adjacent segments respectively; S90, classify and store the processed sound segments according to animal categories.

2. A method for classifying outdoor sound recordings according to claim 1, characterized in that: The relationship between the first length and the second length is: the second length is greater than the first length and is 2-4 times the first length.

3. A method for classifying outdoor sound recordings according to claim 2, characterized in that: The clustering step uses the K-means clustering algorithm, and k is set to the expected maximum number of animals making sounds at the same time.

4. A method for classifying outdoor sound recordings according to claim 3, characterized in that: The field call classification model adopts a lightweight convolutional neural network structure.

5. A method for classifying outdoor sound recordings according to claim 4, characterized in that: The steps of constructing the training data set of the field sound classification model specifically include: Step 1601, collecting recordings of calls of various wild animals; Step 1602: segment the recording to extract a single animal sound segment; Step 1603: Perform time-frequency transformation on each sound segment to generate time-frequency data; Step 1604: perform data enhancement on the time-frequency data; Step 1605: label the processed data with animal categories; Step 1606: Divide the data set into a training set, a validation set, and a test set.

6. A method for classifying field sound recordings according to claim 5, characterized in that: The step of obtaining the confidence difference threshold is specifically: Step 1801: Run the trained model on the validation set to obtain all prediction results; Step 1802, calculating the average confidence of correctly classified samples; Step 1803, calculating the average confidence of the misclassified samples; Step 1804: take half of the difference between the two average values ​​as the initial threshold; Step 1805: Experiment with different thresholds on the validation set and select a threshold that can maximize the classification accuracy; Step 1806: Verify the validity of the threshold on the test set.

7. A method for classifying field sound recordings according to claim 6, characterized in that: The basic audio preprocessing at least includes: resampling, DC component removal, and noise reduction.

8. A method for classifying field sound recordings according to claim 2, characterized in that: The first length is 0.5 seconds, and the second length is 2 seconds.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed in a computer, they are used to execute a method for classifying field sound records as described in any one of claims 1-8.

10. A field sound recording and classification system, characterized in that: A computer-readable storage medium comprising the computer-readable storage medium of claim 9.

Images