Animal sound event detection model training method, animal sound event detection method and animal sound event detection device

Abstract

The invention provides a training method, a detection method and a device of an animal sound event detection model, and relates to the technical field of intelligent audio signal processing. The training method comprises the steps that a sample support set and a multi-class training set are obtained, the sample support set comprises first positive and negative samples and a query sample, and the multi-class training set comprises the first positive sample; extracting sample frame-level acoustic features, wherein the sample frame-level acoustic features comprise a sample Log-Mel feature and a sample PCEN feature; performing multi-task joint fine tuning on the pre-training detection model through the sample frame-level acoustic features to obtain an intermediate detection model; performing animal sound event detection on the query sample through the intermediate detection model to obtain a first detection result so as to determine a second positive sample, and updating the frame-level acoustic features of the sample; and iteratively executing multi-task joint fine tuning and query sample detection to finally obtain an animal sound event detection model. According to the invention, high-precision and high-robustness animal sound event detection can be realized under the condition of small samples.

Description

Technical Field

[0001] This invention relates to the field of intelligent audio signal processing technology, and in particular to a training method, detection method, and apparatus for an animal sound event detection model. Background Technology

[0002] Animal sound event detection has crucial applications in ecological research, environmental monitoring, biodiversity assessment, and species conservation. By automatically identifying and locating the vocalizations of specific animals in the natural environment, researchers can non-invasively monitor species distribution, population dynamics, behavioral patterns, and habitat changes, providing critical data support for environmental protection decisions and biodiversity conservation.

[0003] In animal sound event detection research, the evolution of technology has primarily shifted from traditional methods to deep learning methods. Early traditional methods relied mainly on signal processing techniques and hand-designed acoustic feature engineering, followed by classification using classic machine learning algorithms. These methods are poorly adapted to complex scenes and struggle to effectively capture fine-grained acoustic features. In recent years, deep learning-based models, especially convolutional neural networks and recurrent neural networks, have significantly improved detection performance by extracting effective features from audio signals through end-to-end learning. However, the success of deep learning heavily depends on large-scale, high-quality labeled data, making it difficult to apply in scenarios involving endangered animals or where sound is difficult to obtain. Currently, although some research has introduced few-shot learning techniques, existing few-shot detection methods are prone to overfitting and insufficient robustness when dealing with short-lived, sudden animal vocalization events.

[0004] Therefore, how to achieve high-precision and robust animal sound event detection under small sample conditions is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This invention provides a training method, detection method, and apparatus for an animal sound event detection model, which can achieve high-precision and high-robustness animal sound event detection under small sample conditions.

[0006] This invention provides a training method for an animal sound event detection model, comprising: Obtain a sample support set and a multi-class training set; wherein, the sample support set includes a first positive sample, a first negative sample and a query sample, and the multi-class training set includes the first positive sample; Extract sample frame-level acoustic features from the sample support set and the sample audio in the multi-class training set. The sample frame-level acoustic features include sample log-Mel spectrogram features and sample per-channel energy normalized PCEN features. By using the sample frame-level acoustic features, the pre-trained detection model is jointly fine-tuned using multiple tasks to obtain an intermediate detection model; The intermediate detection model is used to detect animal sound events in the query sample to obtain a first detection result. Based on the first detection result, a second positive sample is determined, and the frame-level acoustic features of the sample are updated. By using the updated sample frame-level acoustic features, the multi-task joint fine-tuning and query sample detection are iteratively performed until the preset iteration stopping condition is met, thus obtaining the animal sound event detection model.

[0007] According to the training method of an animal sound event detection model provided by the present invention, the step of performing multi-task joint fine-tuning on the pre-trained detection model using the sample frame-level acoustic features to obtain an intermediate detection model includes: The sample frame-level acoustic features are input into the feature embedding layer of the pre-trained detection model to obtain the feature embedding vector output by the feature embedding layer; wherein, the feature embedding vector includes a first positive sample embedding vector, a first negative sample embedding vector, a query sample embedding vector, and a training set sample embedding vector; The first positive sample embedding vector, the first negative sample embedding vector, and the query sample embedding vector are input into the first classifier of the pre-trained detection model to perform a binary classification task and obtain the first classification prediction result of the query sample output by the first classifier. The training set sample embedding vector is input into the second classifier of the pre-trained detection model to perform a multi-class classification task, and the second classification prediction result of the training set sample output by the second classifier is obtained. Calculate the joint loss based on the first classification prediction result and the second classification prediction result; The parameters of the feature embedding layer, the first classifier, and the second classifier are updated using the joint loss to obtain the intermediate detection model.

[0008] According to a training method for an animal sound event detection model provided by the present invention, the step of inputting the first positive sample embedding vector, the first negative sample embedding vector, and the query sample embedding vector into the first classifier of the pre-trained detection model to perform a binary classification task and obtain the first classification prediction result of the query sample output by the first classifier includes: Calculate the mean of the first positive sample embedding vector and use it as the positive sample center embedding vector; and calculate the mean of the first negative sample embedding vector and use it as the negative sample center embedding vector. Calculate the first similarity between the query sample embedding vector and the positive sample center embedding vector, and calculate the second similarity between the query sample embedding vector and the negative sample center embedding vector; The first similarity and the second similarity are normalized by the softmax function to obtain the first classification prediction result of the query sample.

[0009] According to the training method of an animal sound event detection model provided by the present invention, the feature embedding layer includes a first basic residual block, a second basic residual block, a first convolutional layer and a second convolutional layer connected in sequence.

[0010] According to a training method for an animal sound event detection model provided by the present invention, the step of calculating a joint loss based on the first classification prediction result and the second classification prediction result includes: Calculate the first cross-entropy loss based on the first classification prediction result and the first true classification label corresponding to the query sample; The second cross-entropy loss is calculated based on the second classification prediction result and the second true classification label corresponding to the training set sample. The joint loss is obtained by weighted summation of the first cross-entropy loss and the second cross-entropy loss.

[0011] According to the training method of an animal sound event detection model provided by the present invention, the step of extracting sample frame-level acoustic features of the sample audio in the sample support set and the multi-class training set includes: The sample audio from the sample support set and the multi-class training set is converted into a Mel spectrogram at a preset Mel frequency. Log-Mel features of the sample are obtained by performing logarithmic scaling on the Mel spectrogram. The energy of each channel in the Mel spectrogram is adaptively normalized and dynamically compressed to obtain the sample PCEN features.

[0012] According to the training method of an animal sound event detection model provided by the present invention, before the step of performing multi-task joint fine-tuning of the pre-trained detection model using the sample frame-level acoustic features to obtain an intermediate detection model, the method further includes: The initial detection model is pre-trained using the multi-class training set to obtain the pre-trained detection model.

[0013] The present invention also provides a method for detecting animal sound events, comprising: The target animal sound event sample in the audio to be detected is obtained as the third positive sample, and the non-target animal sound sample is obtained as the third negative sample, and the query segment in the audio to be detected is obtained. Extract the target frame-level acoustic features of the third positive sample, the third negative sample, and the query segment. The target frame-level acoustic features include target Log-Mel features and target PCEN features. The target frame-level acoustic features are input into the animal sound event detection model to obtain the second detection result of the query segment output by the animal sound event detection model; The animal sound event detection model is trained based on the training method for the animal sound event detection model described in any of the above-mentioned embodiments.

[0014] According to the present invention, an animal sound event detection method is provided, wherein acquiring target animal sound event samples as third positive samples and non-target animal sound samples as third negative samples in the audio to be detected includes: N target animal sound event samples from the audio to be detected are obtained as the third positive sample; where N is a positive integer less than 10; Sampling is performed within the audio interval between the N third positive samples to obtain M non-target animal sound samples, which are used as the third negative samples.

[0015] The present invention also provides a training device for an animal sound event detection model, comprising: The first acquisition module is used to acquire a sample support set and a multi-class training set; wherein, the sample support set includes a first positive sample, a first negative sample and a query sample, and the multi-class training set includes the first positive sample; The first extraction module is used to extract sample frame-level acoustic features of the sample audio in the sample support set and the multi-class training set. The sample frame-level acoustic features include sample Log-Mel features and sample PCEN features. The joint fine-tuning module is used to perform multi-task joint fine-tuning on the pre-trained detection model using the sample frame-level acoustic features to obtain an intermediate detection model; The sample update module is used to perform animal sound event detection on the query sample through the intermediate detection model to obtain a first detection result, determine a second positive sample based on the first detection result, and update the frame-level acoustic features of the sample. The iterative training module is used to iteratively perform multi-task joint fine-tuning and query sample detection using the updated sample frame-level acoustic features until the preset iteration stopping condition is met, thus obtaining the animal sound event detection model.

[0016] The present invention also provides an animal sound event detection device, comprising: The second acquisition module is used to acquire target animal sound event samples in the audio to be detected as third positive samples, and non-target animal sound samples as third negative samples, and to acquire the query segment in the audio to be detected. The second extraction module is used to extract the target frame-level acoustic features of the third positive sample, the third negative sample, and the query segment. The target frame-level acoustic features include target Log-Mel features and target PCEN features. The event detection module is used to input the target frame-level acoustic features into the animal sound event detection model to obtain the second detection result of the query segment output by the animal sound event detection model; The animal sound event detection model is trained based on the training method for the animal sound event detection model described in any of the above-mentioned embodiments.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a training method for an animal sound event detection model as described in any of the preceding claims, or an animal sound event detection method as described in any of the preceding claims.

[0018] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a training method for an animal sound event detection model as described in any of the preceding claims, or an animal sound event detection method as described in any of the preceding claims.

[0019] The present invention provides a training method, detection method, and apparatus for an animal sound event detection model. First, it acquires a support set consisting of a small number of labeled samples and a large-scale multi-class training set. The support set includes a first positive sample, a first negative sample, and a query sample; the multi-class training set includes the first positive sample. Then, it extracts frame-level acoustic features from the sample audio samples in the support set and the multi-class training set, specifically including sample Log-Mel features and sample PCEN features, providing a robust foundation for the model. Next, using the sample frame-level acoustic features, the pre-trained detection model is jointly fine-tuned across multiple tasks to obtain an intermediate detection model. By applying the pre-trained detection model to a few-sample task, the sample requirement during training is reduced. Then, through joint fine-tuning across multiple tasks, the model can learn rapid adaptation capabilities through few-sample learning while being subject to strong regularization constraints from multi-class learning, learning general acoustic features. This fundamentally avoids overfitting to a small number of samples, enhances the model's generalization ability, and ensures that the knowledge learned by the model can be transferred to new sound categories in the future. Next, an intermediate detection model is used to detect animal sound events on the query samples, obtaining the first detection result. Based on the first detection result, a second positive sample is determined, and the frame-level acoustic features of the sample are updated. Using the updated frame-level acoustic features, multi-task joint fine-tuning and query sample detection are iteratively performed until a preset iteration stopping condition is reached, resulting in an animal sound event detection model. By further introducing a self-training iteration step, high-confidence samples are selected to update the frame-level acoustic features of the sample and the training is repeated. This allows the model to adaptively optimize for the current task data distribution, ultimately obtaining an animal sound event detection model with both high generalization and high environmental adaptability. This effectively solves the problems of overfitting and insufficient robustness of the model under few sample conditions, improving the detection accuracy and robustness of the model. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is one of the flowcharts illustrating the training method for the animal sound event detection model provided by this invention.

[0022] Figure 2 This is the second flowchart illustrating the training method for the animal sound event detection model provided by this invention.

[0023] Figure 3 This is the third flowchart illustrating the training method for the animal sound event detection model provided by this invention.

[0024] Figure 4 This is a flowchart illustrating the animal sound event detection method provided by the present invention.

[0025] Figure 5 This is a schematic diagram of the training device for the animal sound event detection model provided by the present invention.

[0026] Figure 6 This is a schematic diagram of the animal sound event detection device provided by the present invention.

[0027] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] Animal sound event detection has crucial applications in ecological research, environmental monitoring, biodiversity assessment, and species conservation. By automatically identifying and locating the vocalizations of specific animals in the natural environment, researchers can non-invasively monitor species distribution, population dynamics, behavioral patterns, and habitat changes, providing critical data support for environmental protection decisions and biodiversity conservation.

[0030] In research on animal sound event detection, the evolution of technology has primarily involved a shift from traditional methods to deep learning methods. Existing methods mainly include: (1) Traditional methods: Traditional animal sound event detection methods mainly rely on signal processing and feature engineering. These methods are usually based on hand-designed features, and then classic machine learning algorithms are used for model training and event classification. The core of traditional methods is to extract effective features from audio signals. These features usually include time-domain features, frequency-domain features, and time-frequency features. Common features include Mel-Frequency Cepstral Coefficients (MFCC), Short-Time Fourier Transform (STFT), and Wavelet Transform. Commonly used machine learning algorithms include Support Vector Machine (SVM) and K-Nearest Neighbors (KNN). These methods have poor adaptability to complex scenes and are difficult to effectively capture fine-grained acoustic features.

[0031] (2) Deep Learning Methods: With the rapid development of deep learning, especially the successful application of networks such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Transformers, deep learning technology has gradually become the mainstream method for animal sound event detection. Deep learning methods can automatically learn effective features from raw audio signals, avoiding the manual feature extraction process in traditional methods. Currently, although deep learning-based sound event detection methods have made some progress, most methods rely on a large amount of training data, making them difficult to apply in scenarios involving endangered animals or where sound is difficult to obtain.

[0032] (3) Few-shot learning: Through the N-way-K-shot (N classes, K samples) paradigm, classification prediction is achieved under the condition of very few samples, which is suitable for data-scarce situations. However, existing few-shot detection methods are prone to overfitting and insufficient robustness when dealing with short-term, sudden animal vocalization events.

[0033] Therefore, how to achieve high-precision and robust animal sound event detection under small sample conditions is a technical problem that urgently needs to be solved.

[0034] Based on the above, this invention proposes a training method, detection method, and apparatus for an animal sound event detection model. The following is a detailed explanation. Figures 1-7 Describe it.

[0035] Figure 1This is one of the flowcharts illustrating the training method for the animal sound event detection model provided by this invention, such as... Figure 1 As shown, the training method for the animal sound event detection model includes steps S110, S120, S130, S140, and S150.

[0036] Step S110: Obtain a sample support set and a multi-class training set; wherein the sample support set includes a first positive sample, a first negative sample and a query sample, and the multi-class training set includes the first positive sample.

[0037] The sample support set is a dataset simulating a small number of animal sound events.

[0038] In one implementation, the sample support set is constructed as follows: A category is randomly selected from L categories in a multi-category training set as the target animal sound event for training. An audio file containing the target animal sound event and other animal sound events is obtained. Then, the audio file is manually labeled. The first N (e.g., N=5) audio segments labeled as the target animal sound events are designated as positive samples, denoted as the first positive samples. From the audio intervals between the N positive samples, M audio segments are randomly sampled as negative samples, denoted as the first negative samples. Specifically, a non-target animal sound audio segment can be selected between every two positive samples as a negative sample. This sampling strategy ensures that the negative samples are temporally close to the positive samples and may contain easily confused interference segments, helping the model learn more accurate decision boundaries. Using the above sampling method, four audio segments without animal sound events can be obtained from the five first positive samples labeled with the start and end positions of the animal sound events as the first negative samples. All audio segments after the first five first positive samples are used as query samples. The true labels of these query samples (denoted as the first true classification label) are known and are used to indicate whether the query sample belongs to the target animal sound event. For example, if it belongs, it is labeled as positive class 1, and if it does not belong, it is labeled as negative class 0. These labels are used to calculate the loss during training and supervise the model to learn how to classify using the sample support set.

[0039] In another implementation, the sample support set is constructed as follows: A category is randomly selected from L categories in the multi-category training set as the target animal sound event for training. Then, N different audio segments are randomly sampled from the selected target category (e.g., N=5) as positive samples, denoted as the first positive samples; M audio segments (e.g., M=4) are randomly sampled from the remaining L-1 non-target categories as negative samples, denoted as the first negative samples; simultaneously, a new batch of audio segments that do not overlap with the sample support set are sampled from the same target category and some non-target categories as query samples.

[0040] The multi-class training set is a large-scale, high-quality dataset of sound events containing L (e.g., L=20) known categories of animal sounds, such as dog barks, cat meows, and bird calls. Each category has ample labeled samples (e.g., hundreds per category) with precise start and end timestamps. This multi-class training set is used to provide the model with a broad acoustic knowledge base.

[0041] Step S120: Extract sample frame-level acoustic features from the sample support set and the sample audio in the multi-class training set. The sample frame-level acoustic features include sample log-Mel spectrogram features and sample per-channel energy normalized PCEN features.

[0042] Extract frame-level acoustic features (denoted as sample frame-level acoustic features) of each sample audio in the sample support set and multi-class training set. The sample frame-level acoustic features include Log-Mel (Logarithmic Mel-scaled Spectrogram) features (denoted as sample Log-Mel features) and PCEN (Per-Channel Energy Normalization) features (denoted as sample PCEN features).

[0043] Log-Mel features are used to provide computationally efficient and stable features for animal sound event detection, while PCEN features are used to provide highly robust features for animal sound event detection, making them particularly suitable for applications in complex or noisy environments. By extracting these two types of features, high-quality feature inputs can be provided for subsequent training, thereby improving the model's detection and classification performance.

[0044] The specific feature extraction process is as follows: The sample audio from the support set and the multi-class training set is converted into Mel spectrograms with preset Mel frequencies; the Mel spectrograms are logarithmically scaled to obtain the sample Log-Mel features; the energy of each channel in the Mel spectrogram is adaptively normalized and dynamically compressed to obtain the sample PCEN features. The specific execution process can be found in the following embodiments, which will not be elaborated here.

[0045] In one implementation, the sample Log-Mel features and sample PCEN features are directly used as sample frame-level acoustic features.

[0046] In another embodiment, after extracting the sample Log-Mel features and sample PCEN features, the sample Log-Mel features and sample PCEN features are concatenated to obtain sample frame-level acoustic features.

[0047] In another embodiment, after extracting the sample Log-Mel features and sample PCEN features, the sample Log-Mel features and sample PCEN features are weighted and fused to obtain sample frame-level acoustic features.

[0048] Step S130: Using the sample frame-level acoustic features, perform multi-task joint fine-tuning on the pre-trained detection model to obtain an intermediate detection model.

[0049] By using the acoustic features at the sample frame level, the pre-trained detection model is jointly fine-tuned using multiple tasks to obtain the jointly fine-tuned detection model, which is denoted as the intermediate detection model.

[0050] Multi-task joint fine-tuning includes binary classification task training on a sample support set (few samples) and multi-class classification task training on a multi-class training set.

[0051] In one embodiment, the initial detection model is pre-trained for multi-class animal sound classification directly using the multi-class training set obtained in step S110 above, resulting in a pre-trained detection model. The initial detection model includes a shared feature embedding layer and a second classifier for multi-class classification; during the pre-training process, only the parameters of the feature embedding layer and the second classifier are updated, while the first classifier for binary classification detection is not used or has not yet been initialized at this stage.

[0052] In another embodiment, another multi-class training set (different from or partially the same as the multi-class training set obtained in step S110 above, for example, including samples of animal sounds of other categories) can be obtained, and the initial detection model can be pre-trained for multi-class classification based on this dataset. Similarly, only the feature embedding layer and the second classifier are trained to obtain the pre-trained model.

[0053] The pre-trained detection model includes the pre-trained feature embedding layer and the second classifier. In the subsequent multi-task joint fine-tuning stage, the first classifier is further introduced, and the feature embedding layer, the first classifier, and the second classifier are jointly optimized based on the sample support set and the multi-class pre-training set, thereby completing the training of the animal sound event detection model.

[0054] Furthermore, the pre-trained detection model adopts a network structure based on ResNet (residual network), and the feature embedding layer includes a first basic residual block, a second basic residual block, a first convolutional layer and a second convolutional layer connected in sequence.

[0055] The specific multi-task joint fine-tuning process is as follows: The sample frame-level acoustic features (including the frame-level acoustic features of the first positive sample, the first negative sample, and the query sample in the sample support set, as well as the frame-level acoustic features of each training set sample in the multi-class training set) are input into the feature embedding layer of the pre-trained detection model to obtain the feature embedding vector output by the feature embedding layer. The feature embedding vector includes the first positive sample embedding vector, the first negative sample embedding vector, the query sample embedding vector, and the training set sample embedding vector. The first positive sample embedding vector, the first negative sample embedding vector, and the query sample embedding vector are input into the first classifier of the pre-trained detection model to perform a binary classification task, obtaining the first classification prediction result of the query sample output by the first classifier. The training set sample embedding vector is input into the second classifier of the pre-trained detection model to perform a multi-class classification task, obtaining the second classification prediction result of the training set sample output by the second classifier. Based on the first and second classification prediction results, the joint loss is calculated. The parameters of the feature embedding layer, the first classifier, and the second classifier are updated using the joint loss to obtain the intermediate detection model. The specific execution process can be referred to in the following embodiment, which will not be elaborated here.

[0056] By pre-training on a large-scale dataset and fine-tuning it jointly with a supervised multi-class training set and a few-shot support set, the model's ability to detect animal sound events was improved. Transfer learning can apply the pre-trained detection model to few-shot tasks, using existing knowledge for initialization, thereby reducing the sample requirements during training. Especially in acoustic tasks, pre-trained detection models can accelerate convergence and improve generalization ability through transfer learning.

[0057] Furthermore, multi-task joint fine-tuning enables simultaneous training of multiple tasks within the same network. This approach allows the model to share representation layers across multiple tasks, improving training efficiency and reducing overfitting. Specifically, multi-class classification and binary classification tasks share the underlying feature embedding layers within the network, while at higher levels, task-specific classifiers are optimized for the specific task. Multi-class classification tasks, trained on multi-class training sets, may involve identifying different types of animal sounds or other category labels. This multi-class learning allows the model to share representations across learning tasks, improving accuracy for each. Simultaneously, binary classification tasks, using sample support sets, are typically used to determine the presence of a specific animal sound event within a given time period. This binary classification task not only helps the model better understand the temporal distribution of signals globally but also facilitates the model's learning of temporal and spatial features.

[0058] Step S140: Using the intermediate detection model, animal sound event detection is performed on the query sample to obtain a first detection result. Based on the first detection result, a second positive sample is determined, and the frame-level acoustic features of the sample are updated.

[0059] The query sample embedding vector corresponding to the query sample is input into the intermediate detection model to detect animal sound events on the query sample. The detection result output by the intermediate detection model is denoted as the first detection result. The first detection result includes the predicted classification result and its probability (confidence) corresponding to the query sample.

[0060] It should be understood that when performing animal sound event detection on query samples, the second classifier should be removed and predictions should be made using the first classifier.

[0061] Then, a second positive sample is determined based on the first detection result. Specifically, a fixed high threshold is set, such as 0.8, and samples predicted as positive with a probability exceeding this threshold are selected as the second positive samples. These samples are considered to be target animal sound event samples that the model is highly confident in.

[0062] The selected second positive samples are added to the sample support set and multi-class training set for the next round of training, and merged with the original first positive samples to form a richer and more diverse set of positive samples. The frame-level acoustic features of the second positive samples are extracted to update the sample frame-level acoustic features.

[0063] Furthermore, new negative samples can be selected from those predicted as negative and with high probability (confidence) and added to the first negative sample, while the acoustic features of the sample frames can be updated further.

[0064] Step S150: Using the updated sample frame-level acoustic features, iteratively perform multi-task joint fine-tuning and query sample detection until the preset iteration stop condition is met, and obtain the animal sound event detection model.

[0065] Preset iteration stopping conditions may include, but are not limited to: (1) reaching the maximum number of iterations, such as 100 epochs; (2) the number of newly added high-confidence samples in two consecutive rounds is less than the preset value. Specific iteration stopping conditions can be set according to the actual situation, and are not specifically limited here.

[0066] Using the updated sample frame-level acoustic features, the multi-task joint fine-tuning in step S130 and the query sample detection in step S140 are repeated until the preset iteration stopping condition is met. The model obtained from the last training is the final animal sound event detection model. At this point, the second classifier and its corresponding parameters can be removed or discarded, as multi-classification is no longer needed in the application stage. The final saved animal sound event detection model includes a feature embedding layer with fixed weights and a first classifier.

[0067] By iterating and gradually optimizing the model, it can be made better suited to classification tasks with few samples.

[0068] The training method for the animal sound event detection model provided in this invention first acquires a support set of a small number of labeled samples and a large-scale multi-class training set. The support set includes a first positive sample, a first negative sample, and a query sample, while the multi-class training set includes the first positive sample. Then, sample frame-level acoustic features are extracted from the sample audio samples in the support set and the multi-class training set, specifically including sample Log-Mel features and sample PCEN features, providing a robust foundation for the model. Next, the pre-trained detection model is jointly fine-tuned using these sample frame-level acoustic features to obtain an intermediate detection model. By applying the pre-trained detection model to a few-sample task, the sample requirement during training is reduced. Then, through multi-task joint fine-tuning, the model can learn rapid adaptation capabilities through few-sample learning while being subject to strong regularization constraints from multi-class learning, learning general acoustic features. This fundamentally avoids overfitting to a small number of samples, enhances the model's generalization ability, and ensures that the knowledge learned by the model can be transferred to new sound categories in the future. Next, an intermediate detection model is used to detect animal sound events on the query samples, obtaining the first detection result. Based on the first detection result, a second positive sample is determined, and the frame-level acoustic features of the sample are updated. Using the updated frame-level acoustic features, multi-task joint fine-tuning and query sample detection are iteratively performed until a preset iteration stopping condition is reached, resulting in an animal sound event detection model. By further introducing a self-training iteration step, high-confidence samples are selected to update the frame-level acoustic features of the sample and the training is repeated. This allows the model to adaptively optimize for the current task data distribution, ultimately obtaining an animal sound event detection model with both high generalization and high environmental adaptability. This effectively solves the problems of overfitting and insufficient robustness of the model under few sample conditions, improving the detection accuracy and robustness of the model.

[0069] Based on any of the above embodiments Figure 2 This is the second flowchart illustrating the training method for the animal sound event detection model provided by this invention, as shown below. Figure 2 As shown, step S130 includes: step S131, step S132, step S133, step S134 and step S135.

[0070] Step S131: Input the sample frame-level acoustic features into the feature embedding layer of the pre-trained detection model to obtain the feature embedding vector output by the feature embedding layer; wherein, the feature embedding vector includes a first positive sample embedding vector, a first negative sample embedding vector, a query sample embedding vector, and a training set sample embedding vector.

[0071] The pre-trained detection model includes a feature embedding layer and a second classifier. It should be understood that the feature embedding layer and the second classifier are obtained by pre-training the initial detection model on a multi-class training set. Furthermore, in the multi-task joint fine-tuning stage, a first classifier is introduced into the pre-trained detection model.

[0072] The feature embedding layer is a deep convolutional neural network, preferably comprising a first basic residual block, a second basic residual block, a first convolutional layer, and a second convolutional layer connected in sequence, used to convert frame-level acoustic features into feature embedding vectors.

[0073] The first classifier, also known as the few-shot classifier head, is used to perform classification prediction for binary classification tasks with few samples. Specifically, it dynamically generates first positive and negative sample center embedding vectors based on the input first positive and negative sample feature embedding vectors, calculates the similarity between the query sample embedding vector and these two center embedding vectors, and then obtains the binary classification prediction result through the softmax (normalized exponent) activation function.

[0074] The second classifier, also known as the multi-class head, can be a fully connected layer used to perform classification predictions for multi-class tasks.

[0075] The sample frame-level acoustic features of a batch of data (including support set samples and multi-class training set samples) are input into the feature embedding layer of the pre-trained detection model to obtain the first positive sample embedding vector, the first negative sample embedding vector, the query sample embedding vector, and the training set sample embedding vector.

[0076] Step S132: Input the first sample embedding vector, the first negative sample embedding vector, and the query sample embedding vector into the first classifier of the pre-trained detection model to perform a binary classification task and obtain the first classification prediction result of the query sample output by the first classifier.

[0077] The first positive sample embedding vector, the first negative sample embedding vector, and the query sample embedding vector are input into the first classifier of the pre-trained detection model to perform a binary classification task. Specifically, the first classifier first calculates the mean of the first positive sample embedding vectors as the positive sample center embedding vector, and simultaneously calculates the mean of the first negative sample embedding vectors as the negative sample center embedding vector. Then, it calculates the first similarity between the query sample embedding vector and the positive sample center embedding vector, and the second similarity between the query sample embedding vector and the negative sample center embedding vector. The first and second similarities obtained above are used as classification scores and input into the softmax (normalization exponent) function for probability normalization. Finally, the classification prediction result of the query sample output by the first classifier is obtained, which is denoted as the first classification prediction result.

[0078] The first category prediction result includes the predicted category label and its probability for each query sample.

[0079] Step S133: Input the training set sample embedding vector into the second classifier of the pre-trained detection model to perform a multi-class classification task and obtain the second classification prediction result of the training set sample output by the second classifier.

[0080] The training set sample embedding vectors are input into the second classifier of the pre-trained detection model to perform a multi-class classification task. The classification prediction results of the training set samples output by the second classifier are obtained and are denoted as the second classification prediction results. The second classification prediction results include the predicted class label and its probability for each training set sample.

[0081] Step S134: Calculate the joint loss based on the first classification prediction result and the second classification prediction result.

[0082] In one embodiment, a first cross-entropy loss is calculated based on the first classification prediction result and the first true classification label corresponding to the query sample; a second cross-entropy loss is calculated based on the second classification prediction result and the second true classification label corresponding to the training set sample; and the first cross-entropy loss and the second cross-entropy loss are summed to obtain the joint loss.

[0083] In another embodiment, a first cross-entropy loss is calculated based on the first classification prediction result and the first true classification label corresponding to the query sample; a second cross-entropy loss is calculated based on the second classification prediction result and the second true classification label corresponding to the training set sample; and the first cross-entropy loss and the second cross-entropy loss are weighted and summed to obtain the joint loss.

[0084] Step S135: Update the parameters of the feature embedding layer, the first classifier, and the second classifier using the joint loss to obtain the intermediate detection model.

[0085] The backpropagation algorithm is executed to calculate the gradient of the joint loss with respect to the parameters of the feature embedding layer, the first classifier, and the fully connected layer of the second classifier. The optimizer is then used to update these parameters, ultimately resulting in an intermediate detection model.

[0086] The training method for the animal sound event detection model provided in this invention deeply couples few-shot classification tasks with multi-shot classification tasks through a shared feature embedding layer and a collaborative parameter update mechanism. This allows the model's feature representation learning during optimization to be directly guided by two objectives: to cluster similar samples in the embedding space (facilitating binary classification with few shots) and to fully separate samples of different classes (facilitating multi-shot classification). This approach ensures that the feature embedding layer ultimately learned by the model possesses strong generalization and discriminative power, which is crucial for solving the problem of overfitting with few shots.

[0087] Based on any of the above embodiments, step S132 includes: step S1321, step S1322 and step S1323.

[0088] Step S1321: Calculate the mean of the first positive sample embedding vector as the positive sample center embedding vector, and calculate the mean of the first negative sample embedding vector as the negative sample center embedding vector.

[0089] Calculate the mean of the first positive sample embedding vector and use it as the positive sample center embedding vector. At the same time, calculate the mean of the first negative sample embedding vector and use it as the negative sample center embedding vector.

[0090] Step S1322: Calculate the first similarity between the query sample embedding vector and the positive sample center embedding vector, and calculate the second similarity between the query sample embedding vector and the negative sample center embedding vector.

[0091] Calculate the cosine similarity between the query sample embedding vector and the first positive sample center embedding vector and the first negative sample center embedding vector respectively to obtain the first similarity and the second similarity.

[0092] The formula for calculating the first similarity is as follows: Similarity1=cos(Query Emb,POS Center Emb), The formula for calculating the second similarity is as follows: Similarity2=cos(Query Emb,NEG Center Emb); Where Similarity1 represents the first similarity, Similarity2 represents the second similarity, cos() represents the cosine similarity, Query Emb represents the query sample embedding vector, POS Center Emb represents the positive sample center embedding vector, and NEG Center Emb represents the negative sample center embedding vector.

[0093] Step S1323: Normalize the first similarity and the second similarity using the normalization exponent softmax function to obtain the first classification prediction result of the query sample.

[0094] The first and second similarity scores calculated above are used as classification scores and input into the softmax (normalization index) function for probability normalization, yielding the first classification prediction result: whether the query sample belongs to the positive or negative class. Specifically, the softmax output is as follows: ; ; in, This indicates the probability that the query sample is predicted as the target animal sound event (positive class). This indicates the probability that the query sample is predicted as a non-target animal sound event (negative class).

[0095] The training method for the animal sound event detection model provided in this invention constructs stable and representative positive and negative sample center embedding vectors by calculating the mean of the first positive and negative sample embedding vectors. This effectively integrates the commonalities of similar samples and suppresses abnormal interference. Then, by calculating and normalizing the relative similarity between the query sample and the two center embedding vectors, the classification decision is based on robust relative comparison, significantly improving the tolerance and discriminative power to feature variations and noise. Furthermore, this process is computationally efficient and structurally simple, effectively reducing the risk of overfitting in small sample scenarios.

[0096] Based on any of the above embodiments, the feature embedding layer includes a first basic residual block, a second basic residual block, a first convolutional layer, and a second convolutional layer connected in sequence.

[0097] In this embodiment of the invention, the initial detection model and the pre-trained detection model adopt a ResNet-based network structure, as shown in Table 1 below. As shown in Table 1, it specifically includes the following modules: two BasicBlocks (basic residual blocks), denoted as the first basic residual block and the second basic residual block, respectively; two CNN convolutional layers, denoted as the first convolutional layer and the second convolutional layer, respectively; one fully connected layer (FC); and one Decoder. Detailed configurations of each module, such as the number of channels per layer, kernel size, stride, and activation function, are shown in Table 1.

[0098] Table 1 ResNet Feature Extraction Network Structure

[0099] Note: Conv stands for convolution, BN (Batch Normalization) stands for batch normalization, ReLU (Rectified Linear Unit) stands for corrected linear unit, and softmax stands for normalized exponential function.

[0100] The feature embedding layer includes a first basic residual block, a second basic residual block, a first convolutional layer, and a second convolutional layer connected in sequence, corresponding to BasicBlock1, BasicBlock2, CNN Block3, and CNNBlock4 in Table 1, respectively.

[0101] BasicBlock is the core building block of ResNet, offering significant advantages over traditional convolutional layers. Its skip connections directly add the input to the output, allowing gradients to bypass multiple network layers during backpropagation, effectively mitigating the vanishing gradient problem. Furthermore, skip connections help the network converge faster, reducing training loss, while providing regularization effects to reduce overfitting and enhance the model's generalization ability. CNN convolutional layers are used to capture local time-frequency features. Basic features are extracted through the skip connections of BasicBlock, ensuring gradient flow; then, CNN convolutional layers perform depth transformations on stable features to capture more complex patterns, achieving progressive feature extraction.

[0102] It should be noted that FC and Decoder 2 represent two parallel fully connected heads used for multi-task learning. FC is used for multi-class classification tasks (the parameters in Table 1 are used for 20-class classification tasks), and Decoder 2 is used for binary classification tasks.

[0103] The training method for the animal sound event detection model provided in this embodiment of the invention, in the process of variable length audio processing, on the basis of using frame-level methods to mine similar voiceprint features between adjacent frames, can capture local features and extract stable feature embedding vectors through the feature embedding layer of the above structure, thereby showing better adaptability during model training and avoiding the degradation of model performance caused by changes in audio length.

[0104] Based on any of the above embodiments, step S134 includes: step S1341, step S1342 and step S1343.

[0105] Step S1341: Calculate the first cross-entropy loss based on the first classification prediction result and the first true classification label corresponding to the query sample.

[0106] Step S1342: Calculate the second cross-entropy loss based on the second classification prediction result and the second true classification label corresponding to the training set sample.

[0107] The first classification prediction result includes the predicted classification label and its probability for each query sample, and the second classification prediction result includes the predicted classification label and its probability for each training set sample. The first true classification label indicates whether the query sample belongs to the target animal sound event corresponding to the first positive sample, and the second true classification label is used to indicate the true animal sound label corresponding to the multi-class training set samples.

[0108] Based on the first classification prediction result and the first true classification label corresponding to the query sample, calculate the first cross-entropy loss. At the same time, based on the second classification prediction result and the second true classification label corresponding to the training set sample, calculate the second cross-entropy loss.

[0109] The first cross-entropy loss and the second cross-entropy loss can be calculated using the following formulas: ; in, This represents the cross-entropy loss, where C represents the total number of classes. Represents the true category label, and log() represents the logarithmic function. This represents the classification probability predicted by the model.

[0110] It's important to note that during the training phase, a simple cross-entropy (CE) loss function is chosen instead of loss functions specifically designed for few-shot learning (such as prototype network loss or contrastive loss). This is because while common loss functions in few-shot learning (such as prototype loss, triplet loss, or contrastive loss) are suitable for handling small-shot scenarios, they also have limitations. For example, they are not suitable for multi-class classification; few-shot loss functions are more used to learn the similarity of the embedding space rather than directly optimizing the probability distribution of class labels. For multi-class classification tasks, directly using cross-entropy loss is more intuitive and effective. Since this embodiment of the invention is based on transfer learning, the model already possesses good generalization ability, and fine-tuning with cross-entropy loss can significantly improve classification performance without further optimizing the embedding space using few-shot loss.

[0111] Step S1343: The first cross-entropy loss and the second cross-entropy loss are weighted and summed to obtain the joint loss.

[0112] The joint loss is obtained by weighted summation of the first cross-entropy loss and the second cross-entropy loss.

[0113] The training method for the animal sound event detection model provided in this invention calculates the cross-entropy loss of binary and multi-class classification tasks, and then weights and sums them to obtain a joint loss for updating model parameters. This loss calculation method is simpler, more intuitive, and more effective, contributing to improved model performance.

[0114] Based on any of the above embodiments Figure 3 This is the third flowchart illustrating the training method for the animal sound event detection model provided by this invention, as shown below. Figure 3 As shown, step S120 includes: step S121, step S122 and step S123.

[0115] Step S121: Convert the sample audio from the sample support set and the multi-class training set into a Mel spectrogram with a preset Mel frequency.

[0116] In existing audio processing, segment-level feature extraction is generally employed. Specifically, the audio signal is divided into multiple segments, and each segment undergoes independent feature extraction. The aim of this method is to capture the local features of each audio segment for more detailed analysis. However, segment-level feature extraction also has some significant drawbacks, particularly for processing complex audio events. For example, in detecting animal sounds, since audio signal segments are typically of fixed length, this may not match the actual boundaries of the sound event, leading to loss of detail, especially in the recognition of short animal calls. For instance, animal calls or other natural sounds may not perfectly conform to the set segment length. This mismatch can cause important features to appear at segment boundaries, resulting in them being cut off or segmented into multiple segments, affecting the effectiveness of feature extraction. Furthermore, in some cases, animal calls or sound events may occur and end abruptly. Fixed-length segments may not accurately capture such instantaneous or short-lived sound changes, also impacting the event recognition results.

[0117] To address this, this invention employs two frame-level acoustic features—Log-Mel and PCEN—to provide high temporal resolution audio features for model training. This method divides the audio signal into multiple small time frames, each typically containing a few milliseconds of audio data, and then extracts the log-Mel spectral features of these frames, thereby meticulously capturing short-term changes in the audio signal. For many animal vocal events (such as calls, vocalizations, and breathing), which exhibit rapid dynamic changes, frame-level features can accurately capture these instantaneous changes, avoiding signal loss or errors caused by segment boundaries in segment-level feature extraction.

[0118] First, using short-time Fourier transform and a Mel filter bank, all sample audio data in the sample support set and the multi-class training set are converted into Mel spectrograms at preset Mel frequencies. The preset Mel frequencies are 64-256 bins, preferably 128 bins, meaning a 128-bin Mel filter bank is preferred.

[0119] Mel spectrograms are a crucial foundation for audio feature extraction. By converting audio signals into a perceptually relevant frequency scale, they capture the signal's spectral information. The key parameters in the Mel spectrogram generation process are as follows: FFT points (Fast Fourier Transform samples): 1024. A 1024-point Fast Fourier Transform is selected to convert the time-domain signal to the frequency-domain signal. A larger FFT point count can provide higher frequency resolution.

[0120] Hopsize (frame shift step size): 256. The frame shift step size is set to 256 sampling points, representing the time interval between adjacent frames. A smaller step size helps improve temporal resolution but increases computational overhead.

[0121] Mel filter bank: 128 filters. The spectral distribution is mapped onto 128 Mel filters, each corresponding to a Mel frequency range, thus compressing the frequency dimension while preserving key spectral information.

[0122] Step S122: Perform logarithmic scaling on the Mel spectrogram to obtain the sample Log-Mel features.

[0123] After generating the Mel spectrogram, logarithmic scaling is used to extract Log-Mel features, denoted as sample Log-Mel features. The logarithmic transformation formula is: LogMel(t,f)=log(M(t,f)+ϵ); Where LogMel(t,f) represents the LogMel feature value on time frame t and Mel frequency channel f, M(t,f) represents the Mel spectrogram value, and ϵ is a small positive number used to prevent zero value problems in logarithmic operations.

[0124] Log-Mel features are computationally efficient and stable, and can provide a good basic spectral profile.

[0125] Step S123: Adaptive normalization and dynamic compression processing are performed on the energy of each channel in the Mel spectrogram to obtain the sample PCEN features.

[0126] The energy of each channel in the Mel spectrogram is adaptively normalized and dynamically compressed to obtain the PCEN feature, denoted as the sample PCEN feature. The specific calculation formula is as follows: ; M(t,f)=(1-s)M(t-1,f)+sE(t,f); in, M(t,f) represents the PCEN eigenvalue at time frame t and Mel frequency channel f; M(t,f) is the smoothing of filter bank energy E(t,f), which represents the filter bank energy at time frame t and Mel frequency channel f; M(t-1,f) is the smoothing of filter bank energy E(t-1,f), which represents the filter bank energy at time frame t-1 and Mel frequency channel f. This represents the compression gain factor. denoted by , r represents the dynamic range compression factor, and s is a smoothing coefficient.

[0127] PCEN features are better suited for tasks requiring high robustness, such as environmental sound detection and classification, especially performing well in noisy environments.

[0128] The training method for the animal sound event detection model provided in this invention extracts Log-Mel features and PCEN features. Log-Mel features provide computationally efficient and stable features for animal sound event detection, while PCEN features provide highly robust features, making them particularly suitable for applications in complex or noisy environments. Extracting these two types of features provides high-quality feature input for subsequent training, thereby improving the model's detection and classification performance.

[0129] Based on any of the above embodiments, step S160 is included before step S130.

[0130] Step S160: The initial detection model is pre-trained using the multi-class training set to obtain the pre-trained detection model.

[0131] In this embodiment, the initial detection model is first pre-trained using a multi-class training set to obtain a pre-trained detection model. The initial detection model includes a shared feature embedding layer and a second classifier for multi-class classification. During pre-training, only the parameters of the feature embedding layer and the second classifier are updated, while the first classifier for binary classification detection is not used or has not yet been initialized at this stage. The pre-trained detection model includes the aforementioned pre-trained feature embedding layer and second classifier. In the subsequent multi-task joint fine-tuning stage, the first classifier for binary classification detection is further introduced.

[0132] The training method for the animal sound event detection model provided in this embodiment of the invention pre-trains the initial detection model using a large-scale multi-class training set, and then applies the pre-trained model to subsequent few-sample binary classification tasks, using existing knowledge for initialization, thereby reducing the sample requirements during model training.

[0133] Figure 4 This is a flowchart illustrating the animal sound event detection method provided by the present invention, as shown below. Figure 4 As shown, the animal sound event detection method includes steps S210, S220 and S230.

[0134] Step S210: Obtain the target animal sound event sample in the audio to be detected as the third positive sample, and the non-target animal sound sample as the third negative sample, and obtain the query segment in the audio to be detected.

[0135] Obtain a new audio segment for animal sound event detection, denoted as the audio to be detected. For example, this could be a one-hour recording of an animal in a wild forest. Suppose the user's goal is to detect a specific animal call that the model has never learned before, such as the call of a rare bird, "X".

[0136] By listening to the beginning of the audio to be detected, for example, the first 5 minutes, segments containing the "X" call of the target rare bird are manually identified and labeled. A certain number of positive samples are selected from the labeled data, designated as the third positive samples. The length of each audio segment in the third positive sample can be set to 300ms-500ms. Simultaneously, non-target animal sound samples are acquired as negative samples, designated as the third negative samples. These non-target animal sound samples can be background noise or sounds from other animals.

[0137] Then, all the remaining audio in the audio to be detected, except for those already labeled as the third positive sample and the third negative sample, is divided into audio segments according to a preset length (e.g., 500ms) to serve as query segments, denoted as query segments.

[0138] Furthermore, during segmentation, the remaining audio can be segmented into continuous, potentially overlapping short segments using a sliding window (e.g., 500ms window length, 250ms window shift) to obtain the segment to be queried.

[0139] Step S220: Extract the target frame-level acoustic features of the third positive sample, the third negative sample, and the segment to be queried. The target frame-level acoustic features include target Log-Mel features and target PCEN features.

[0140] Extract the frame-level acoustic features of the third positive sample, the third negative sample, and the query segment, and denote them as target frame-level acoustic features. Target frame-level acoustic features include Log-Mel features (denoted as target Log-Mel features) and PCEN features (denoted as target PCEN features).

[0141] For specific extraction methods, please refer to the above-mentioned methods for extracting acoustic features at the sample frame level, which will not be repeated here.

[0142] In one implementation, the target Log-Mel features and the target PCEN features are directly used as the target frame-level acoustic features.

[0143] In another embodiment, after extracting the target Log-Mel features and the target PCEN features, the target Log-Mel features and the target PCEN features are concatenated to obtain the target frame-level acoustic features.

[0144] In another embodiment, after extracting the target Log-Mel features and the target PCEN features, the target Log-Mel features and the target PCEN features are weighted and fused to obtain the target frame-level acoustic features.

[0145] It should be understood that the acquisition methods for target frame-level acoustic features and sample frame-level acoustic features are similar.

[0146] Step S230: Input the target frame-level acoustic features into the animal sound event detection model to obtain the second detection result of the query segment output by the animal sound event detection model.

[0147] The animal sound event detection model is trained based on the training method of the animal sound event detection model provided in any of the above embodiments.

[0148] The target frame-level acoustic features are input into the animal sound event detection model to perform animal sound event detection. The detection result of the query segment output by the model is recorded as the second detection result.

[0149] Specifically, the target frame-level acoustic features are input into the feature embedding layer of the animal sound event detection model to obtain the feature embedding vector output by the feature embedding layer (denoted as the target feature embedding vector). The target feature embedding vector includes positive sample embedding vectors (denoted as the target positive sample embedding vector), negative sample embedding vectors (denoted as the target negative sample embedding vector), and the query sample embedding vector. The target positive sample embedding vector, target negative sample embedding vector, and query sample embedding vector are input into the first classifier of the animal sound event detection model. The first classifier calculates the mean of the target positive sample embedding vectors, which is used as the positive sample center embedding vector for the current detection task. Simultaneously, the mean of the target negative sample embedding vectors is calculated, which is used as the negative sample center embedding vector for the current detection task. Then, the similarity between the query sample embedding vector and the positive and negative sample center embedding vectors of the current detection task is calculated. Finally, the similarity is normalized using the softmax function to obtain the second detection result for the query sample. This second detection result includes the classification of the query sample and its probability (confidence).

[0150] The animal sound event detection method provided in this invention involves acquiring target animal sound event samples as third positive samples and non-target animal sound samples as third negative samples from the audio to be detected, and then acquiring the query segment from the audio. Next, target frame-level acoustic features are extracted from the third positive samples, third negative samples, and the query segment. These target frame-level acoustic features include target Log-Mel features and target PCEN features. Finally, the target frame-level acoustic features are input into an animal sound event detection model to obtain the detection result of the query segment output by the model. The animal sound event detection model is trained using the aforementioned training method. In this invention, only a small number of target and non-target animal sound event samples are needed to construct a task-specific support set, eliminating the need to collect and label massive amounts of data for the new target animal sound, thus significantly reducing the application threshold and cost. Then, by extracting robust dual-path features consistent with those from the training phase, and leveraging the powerful general feature extraction capabilities acquired by the animal sound event detection model through multi-task learning during the training phase, the essential features of the target animal sounds can be extracted from a small number of samples. Furthermore, animal sound event detection is performed by calculating task-related positive and negative sample center embedding vectors, yielding a second detection result. Through this method, high-precision and highly robust animal sound event detection is achieved under conditions of small sample input.

[0151] Based on any of the above embodiments, the step "obtaining target animal sound event samples in the audio to be detected as third positive samples, and non-target animal sound samples as third negative samples" includes: step S211 and step S212.

[0152] Step S211: Obtain N target animal sound event samples from the audio to be detected as the third positive sample; where N is a positive integer less than 10.

[0153] Suppose the user has already labeled N (e.g., N=5) third positive samples in the audio to be detected, with timestamps [t1] and [t2] respectively. start ,t1 end ], [t2 start t2 end ], ..., [t5 start t5 end ].

[0154] Step S212: Sample within the audio interval between the N third positive samples to obtain M non-target animal sound samples, which are used as the third negative samples.

[0155] On the time axis, the region between two adjacent third positive samples is used as a candidate negative sample pool. For example, the interval (t1) end t2 start), (t2 end t3 start And so on. These areas theoretically do not contain the target event (because only N are labeled), and usually contain representative background sounds.

[0156] Within each such interval, multiple audio segments are extracted by sliding at fixed durations (e.g., 500ms) and fixed intervals (e.g., 1 second). It is ensured that the segments do not overlap and do not contain the time range of positive samples. From all the segments extracted from the intervals, either random selections or all are retained until M non-target animal sound sample segments are obtained, which serve as the final third negative samples. For example, selecting one audio segment between every two positive samples as a negative sample will ultimately yield four third negative samples from five third positive samples.

[0157] The animal sound event detection method provided in this invention reduces the workload of users manually selecting negative samples and enables rapid construction of the target support set. Simultaneously, sampling from regions between positive samples effectively acquires background sounds that are temporally close and acoustically similar to the target animal sound event. Such negative samples help the model effectively distinguish boundaries, thus improving the accuracy of the detection results.

[0158] The training apparatus for the animal sound event detection model provided in the embodiments of this application will be described below. The training apparatus for the animal sound event detection model described below can be referred to in correspondence with the training method for the animal sound event detection model described above.

[0159] Figure 5 This is a schematic diagram of the training device for the animal sound event detection model provided by the present invention, as shown below. Figure 5 As shown, the device includes a first acquisition module 510, a first extraction module 520, a joint fine-tuning module 530, a sample update module 540, and an iterative training module 550; wherein: The first acquisition module 510 is used to acquire a sample support set and a multi-class training set; wherein, the sample support set includes a first positive sample, a first negative sample and a query sample, and the multi-class training set includes the first positive sample; The first extraction module 520 is used to extract sample frame-level acoustic features of the sample audio in the sample support set and the multi-class training set. The sample frame-level acoustic features include sample Log-Mel features and sample PCEN features. The joint fine-tuning module 530 is used to perform multi-task joint fine-tuning on the pre-trained detection model using the sample frame-level acoustic features to obtain an intermediate detection model; The sample update module 540 is used to perform animal sound event detection on the query sample through the intermediate detection model to obtain a first detection result, determine a second positive sample based on the first detection result, and update the frame-level acoustic features of the sample. The iterative training module 550 is used to iteratively perform multi-task joint fine-tuning and query sample detection using the updated sample frame-level acoustic features until the preset iteration stopping condition is met, thus obtaining the animal sound event detection model.

[0160] The training device for the animal sound event detection model provided in this embodiment of the invention first acquires a support set of samples including a small number of labeled samples and a large-scale multi-class training set. The support set includes a first positive sample, a first negative sample, and a query sample, while the multi-class training set includes the first positive sample. Then, it extracts the sample frame-level acoustic features of the sample audio from the support set and the multi-class training set, specifically including sample Log-Mel features and sample PCEN features, providing a robust foundation for the model. Next, the pre-trained detection model is jointly fine-tuned using the sample frame-level acoustic features to obtain an intermediate detection model. By applying the pre-trained detection model to a few-sample task, the sample requirement during training can be reduced. Then, through multi-task joint fine-tuning, the model can learn rapid adaptation capabilities through few-sample learning while being subject to strong regularization constraints from multi-class learning, learning general acoustic features. This fundamentally avoids overfitting to a small number of samples, enhances the model's generalization ability, and ensures that the knowledge learned by the model can be transferred to new sound categories in the future. Next, an intermediate detection model is used to detect animal sound events on the query samples, obtaining the first detection result. Based on the first detection result, a second positive sample is determined, and the frame-level acoustic features of the sample are updated. Using the updated frame-level acoustic features, multi-task joint fine-tuning and query sample detection are iteratively performed until a preset iteration stopping condition is reached, resulting in an animal sound event detection model. By further introducing a self-training iteration step, high-confidence samples are selected to update the frame-level acoustic features of the sample and the training is repeated. This allows the model to adaptively optimize for the current task data distribution, ultimately obtaining an animal sound event detection model with both high generalization and high environmental adaptability. This effectively solves the problems of overfitting and insufficient robustness of the model under few sample conditions, improving the detection accuracy and robustness of the model.

[0161] According to the training method of an animal sound event detection model provided by the present invention, the joint fine-tuning module 530 includes: The first input unit is used to input the sample frame-level acoustic features into the feature embedding layer of the pre-trained detection model to obtain the feature embedding vector output by the feature embedding layer; wherein, the feature embedding vector includes a first positive sample embedding vector, a first negative sample embedding vector, a query sample embedding vector, and a training set sample embedding vector; The second input unit is used to input the first positive sample embedding vector, the first negative sample embedding vector, and the query sample embedding vector into the first classifier of the pre-trained detection model to perform a binary classification task and obtain the first classification prediction result of the query sample output by the first classifier. The third input unit is used to input the training set sample embedding vector into the second classifier of the pre-trained detection model to perform a multi-class classification task and obtain the second classification prediction result of the training set sample output by the second classifier. The loss calculation unit is used to calculate the joint loss based on the first classification prediction result and the second classification prediction result; The parameter update unit is used to update the parameters of the feature embedding layer, the first classifier, and the second classifier using the joint loss to obtain the intermediate detection model.

[0162] According to the training method of an animal sound event detection model provided by the present invention, the second input unit is specifically used for: Calculate the mean of the first positive sample embedding vector and use it as the positive sample center embedding vector; Calculate the mean of the first negative sample embedding vector and use it as the negative sample center embedding vector; Calculate the first similarity between the query sample embedding vector and the positive sample center embedding vector, and calculate the second similarity between the query sample embedding vector and the negative sample center embedding vector; The first similarity and the second similarity are normalized by the softmax function to obtain the first classification prediction result of the query sample.

[0163] According to the training method of an animal sound event detection model provided by the present invention, the feature embedding layer includes a first basic residual block, a second basic residual block, a first convolutional layer and a second convolutional layer connected in sequence.

[0164] According to the training method of an animal sound event detection model provided by the present invention, the loss calculation unit is specifically used for: Calculate the first cross-entropy loss based on the first classification prediction result and the first true classification label corresponding to the query sample; The second cross-entropy loss is calculated based on the second classification prediction result and the second true classification label corresponding to the training set sample. The joint loss is obtained by weighted summation of the first cross-entropy loss and the second cross-entropy loss.

[0165] According to the training method of an animal sound event detection model provided by the present invention, the first extraction module 520 is specifically used for: The sample audio from the sample support set and the multi-class training set is converted into a Mel spectrogram at a preset Mel frequency. Log-Mel features of the sample are obtained by performing logarithmic scaling on the Mel spectrogram. The energy of each channel in the Mel spectrogram is adaptively normalized and dynamically compressed to obtain the sample PCEN features.

[0166] It should be noted that the training device for the animal sound event detection model provided in this embodiment of the invention can implement all the method steps implemented in the above-mentioned training method embodiment of the animal sound event detection model, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0167] The animal sound event detection device provided in the embodiments of this application is described below. The animal sound event detection device described below can be referred to in correspondence with the animal sound event detection method described above.

[0168] Figure 6 This is a schematic diagram of the animal sound event detection device provided by the present invention, as shown below. Figure 6 As shown, the device includes a second acquisition module 610, a second extraction module 620, and an event detection module 630; wherein: The second acquisition module 610 is used to acquire target animal sound event samples in the audio to be detected as third positive samples, and non-target animal sound samples as third negative samples, and to acquire the query segment in the audio to be detected. The second extraction module 620 is used to extract the target frame-level acoustic features of the third positive sample, the third negative sample, and the query segment, wherein the target frame-level acoustic features include target Log-Mel features and target PCEN features; Event detection module 630 is used to input the target frame-level acoustic features into the animal sound event detection model to obtain the second detection result of the query segment output by the animal sound event detection model; The animal sound event detection model is trained based on the training method for the animal sound event detection model described in any of the above-mentioned embodiments.

[0169] The animal sound event detection device provided in this invention acquires target animal sound event samples as third positive samples and non-target animal sound samples as third negative samples in the audio to be detected, and acquires the query segment in the audio to be detected; then, it extracts target frame-level acoustic features from the third positive samples, third negative samples, and the query segment, including target Log-Mel features and target PCEN features; finally, it inputs the target frame-level acoustic features into an animal sound event detection model to obtain the detection result of the query segment output by the animal sound event detection model; wherein, the animal sound event detection model is trained based on the above-mentioned training method for animal sound event detection models. In this invention, only a small number of target and non-target animal sound event samples are needed to construct a task-specific support set, without the need to collect and label massive amounts of data for the new target animal sound, greatly reducing the application threshold and cost. Then, by extracting robust dual-path features consistent with those from the training phase, and leveraging the powerful general feature extraction capabilities acquired by the animal sound event detection model through multi-task learning during the training phase, the essential features of the target animal sounds can be extracted from a small number of samples. Furthermore, animal sound event detection is performed by calculating task-related positive and negative sample center embedding vectors, yielding a second detection result. Through this method, high-precision and highly robust animal sound event detection is achieved under conditions of small sample input.

[0170] According to the animal sound event detection method provided by the present invention, the second acquisition module 610 is specifically used for: N target animal sound event samples from the audio to be detected are obtained as the third positive sample; where N is a positive integer less than 10; Sampling is performed within the audio interval between the N third positive samples to obtain M non-target animal sound samples, which are used as the third negative samples.

[0171] It should be noted that the animal sound event detection device provided in this embodiment of the invention can implement all the method steps implemented in the above-described animal sound event detection method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0172] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call logical instructions in the memory 730 to execute the training method of the animal sound event detection model provided in the above embodiments or the animal sound event detection method provided in the above embodiments.

[0173] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0174] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for the animal sound event detection model provided in the above embodiments or the animal sound event detection method provided in the above embodiments.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary high-resource hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for an animal sound event detection model, characterized in that, The method comprises: obtaining a sample support set and a multi-class training set; wherein the sample support set comprises a first positive sample, a first negative sample and a query sample, and the multi-class training set comprises the first positive sample; extracting sample frame-level acoustic features of the sample audio in the sample support set and the multi-class training set, wherein the sample frame-level acoustic features comprise sample log-mel spectrum (Log-Mel) features and sample per-channel energy normalization (PCEN) features; performing multi-task joint fine-tuning on a pre-trained detection model based on the sample frame-level acoustic features to obtain an intermediate detection model; performing animal sound event detection on the query sample based on the intermediate detection model to obtain a first detection result, determining a second positive sample based on the first detection result, and updating the sample frame-level acoustic features; iteratively performing multi-task joint fine-tuning and query sample detection based on the updated sample frame-level acoustic features until a preset iteration stopping condition is reached to obtain an animal sound event detection model. 2.The method of claim 1, wherein, The method comprises: inputting the sample frame-level acoustic features into a feature embedding layer of the pre-trained detection model to obtain a feature embedding vector output by the feature embedding layer; wherein the feature embedding vector comprises a first positive sample embedding vector, a first negative sample embedding vector, a query sample embedding vector and a training set sample embedding vector; inputting the first positive sample embedding vector, the first negative sample embedding vector and the query sample embedding vector into a first classifier of the pre-trained detection model to perform a binary classification task and obtain a first classification prediction result of the query sample output by the first classifier; inputting the training set sample embedding vector into a second classifier of the pre-trained detection model to perform a multi-class classification task and obtain a second classification prediction result of the training set sample output by the second classifier; calculating a joint loss based on the first classification prediction result and the second classification prediction result; updating parameters of the feature embedding layer, the first classifier and the second classifier based on the joint loss to obtain the intermediate detection model. 3.The method of claim 2, wherein, The method comprises: calculating a mean value of the first positive sample embedding vector as a positive sample center embedding vector and a mean value of the first negative sample embedding vector as a negative sample center embedding vector; calculating a first similarity between the query sample embedding vector and the positive sample center embedding vector and a second similarity between the query sample embedding vector and the negative sample center embedding vector; normalizing the first similarity and the second similarity based on a normalized exponential softmax function to obtain the first classification prediction result of the query sample. 4.The method of claim 2, wherein, The feature embedding layer comprises a first basic residual block, a second basic residual block, a first convolutional layer and a second convolutional layer connected in sequence. 5.The method of claim 2, wherein, The calculating the joint loss based on the first classification prediction result and the second classification prediction result comprises: calculating a first cross-entropy loss according to the first classification prediction result and a first real classification label corresponding to the query sample; calculating a second cross-entropy loss according to the second classification prediction result and a second real classification label corresponding to the training set sample; performing weighted summation on the first cross-entropy loss and the second cross-entropy loss to obtain the joint loss. 6.The method of claim 1 to 5, wherein, The extracting the sample frame-level acoustic features of the sample audio in the sample support set and the multi-class training set comprises: converting the sample audio in the sample support set and the multi-class training set into a mel spectrogram of a preset mel frequency; performing logarithmic scaling processing on the mel spectrogram to obtain sample Log-Mel features; performing adaptive normalization and dynamic compression processing on the energy of each channel in the mel spectrogram to obtain sample PCEN features. 7.The method of claim 1 to 5, wherein, Before the multi-task joint fine-tuning of the pre-training detection model based on the sample frame-level acoustic features to obtain the intermediate detection model, the method further comprises: pre-training an initial detection model based on the multi-class training set to obtain the pre-training detection model.

8. An animal vocalization event detection method, characterized by, comprises: obtaining a target animal sound event sample in the to-be-detected audio as a third positive sample, and a non-target animal sound sample as a third negative sample, and obtaining a to-be-queried segment in the to-be-detected audio; extracting target frame-level acoustic features of the third positive sample, the third negative sample and the to-be-queried segment, the target frame-level acoustic features comprising target Log-Mel features and target PCEN features; inputting the target frame-level acoustic features into an animal sound event detection model to obtain a second detection result of the to-be-queried segment output by the animal sound event detection model; wherein the animal sound event detection model is trained based on the animal sound event detection model training method in any one of claims 1 to 7.

9. The animal vocalization event detection method of claim 8, wherein, The obtaining a target animal sound event sample in the to-be-detected audio as a third positive sample, and a non-target animal sound sample as a third negative sample comprises: obtaining N target animal sound event samples in the to-be-detected audio as third positive samples; wherein N is a positive integer less than 10; sampling in an audio interval between the N third positive samples to obtain M non-target animal sound samples as the third negative samples. 10.A device for training an animal sound event detection model, comprising: comprises: a first obtaining module configured to obtain a sample support set and a multi-class training set; wherein the sample support set comprises first positive samples, first negative samples and query samples, and the multi-class training set comprises the first positive samples; a first extracting module configured to extract sample frame-level acoustic features of sample audio in the sample support set and the multi-class training set, the sample frame-level acoustic features comprising sample Log-Mel features and sample PCEN features; The joint fine-tuning module is configured to perform multi-task joint fine-tuning on the pre-trained detection model by using the sample frame-level acoustic features, and obtain an intermediate detection model. The sample updating module is configured to perform animal sound event detection on the query sample by using the intermediate detection model, obtain a first detection result, determine a second positive sample according to the first detection result, and update the sample frame-level acoustic features. The iterative training module is configured to perform the multi-task joint fine-tuning and the query sample detection iteratively by using the updated sample frame-level acoustic features until a preset iteration stopping condition is reached, and obtain an animal sound event detection model.

11. An animal vocalization event detection apparatus, characterized by, The method comprises the following steps: The second acquisition module is configured to acquire a target animal sound event sample in the audio to be detected as a third positive sample, acquire a non-target animal sound sample as a third negative sample, and acquire a query segment in the audio to be detected. The second extraction module is configured to extract target frame-level acoustic features of the third positive sample, the third negative sample and the query segment, wherein the target frame-level acoustic features comprise target Log-Mel features and target PCEN features. The event detection module is configured to input the target frame-level acoustic features into an animal sound event detection model to obtain a second detection result of the query segment output by the animal sound event detection model. The animal sound event detection model is trained based on the training method of the animal sound event detection model according to any one of claims 1 to 7.

12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the training method of the animal sound event detection model according to any one of claims 1 to 7, or the animal sound event detection method according to any one of claims 8 to 9.

13. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the training method of the animal sound event detection model according to any one of claims 1 to 7, or the animal sound event detection method according to any one of claims 8 to 9.

Images