A method for bird multi-label sound recognition

Abstract

This invention discloses a multi-labeled sound recognition method for birds, including a training phase and an inference phase. The training phase includes the following steps: inputting audio data labeled with multiple tags, and extracting three complementary acoustic representations in parallel through a multi-view feature extraction module. This application uses parallel configuration of Mel spectrogram extraction submodules, CQT spectrogram extraction submodules, and CWT spectrogram extraction submodules to capture complementary features in the audio signal related to human auditory perception, such as energy distribution, pitch and harmonic structure, transients, and sharp calls. Furthermore, the three acoustic representations are aligned in the time dimension to form a comprehensive multi-view feature representation. Compared to existing technologies using fixed view combinations or single feature extraction methods, this design effectively avoids the problem of single-view failure due to noise or occlusion, providing a rich and reliable feature foundation for subsequent recognition tasks and significantly improving feature adaptation capabilities in complex soundscape environments.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a method for multi-label sound recognition of birds. Background Technology

[0002] Bird sound recognition technology plays an increasingly important role in ecological monitoring, species diversity assessment, and urban park management. Real-world sound environments are complex, often containing multiple bird calls and ambient noise simultaneously, exhibiting natural multi-label correspondences. Furthermore, bird sound signals are short, variable, and easily obscured, placing high demands on the robustness of recognition technologies. In recent years, domestic research has evolved from early recognition schemes based on traditional features and support vector machines to a deep learning-led approach. The application of convolutional neural networks and Transformer models has significantly improved the accuracy and robustness of bird sound recognition. The current mainstream technical solutions can be divided into three categories: First, single-view Mel spectrum convolutional networks, which extract features using energy distributions consistent with human auditory perception, and have a relatively simple structure; second, multi-feature / multi-model fusion schemes, which extract Mel spectrum, harmonic sensitive features, time domain or short-time sub-band features in parallel, and then fuse them through fully connected or attention-weighted methods to improve the recognition recall and robustness in complex sound scenes; third, introducing Transformer or attention mechanisms to build self-attention mechanisms on time-frequency or feature sequences, which strengthens the modeling of local spikes, syllable structures and cross-temporal dependencies. Some solutions also combine transfer learning or few-shot optimization strategies to adapt to application scenarios with limited data resources.

[0003] However, existing technologies still have many shortcomings that urgently need to be addressed: First, in multi-species, multi-label acoustic environments, most recognition systems rely on fixed view combinations or manually selected feature views, lacking the ability to adaptively adjust to noisy scenes. When a view fails due to noise interference or signal occlusion, the weights of feature fusion cannot be dynamically adapted, leading to a significant decrease in recognition recall and robustness. Second, the multi-feature fusion process is mostly achieved through convolution operations or simple splicing. Although this improves the appearance recognition accuracy to some extent, it ignores the reliability differences between different views, cannot clearly distinguish between valid and invalid views, and lacks a dedicated suppression mechanism for false positives. First, false positives are prone to accumulate in long-tailed distributions or resonant backgrounds. Second, the model training process generally uses symmetrical binary cross-entropy, Focal loss, or uniformly weighted class balance loss, which fails to separate the focus of positive and negative samples, resulting in limited suppression of easily judged negative samples. At the same time, the positive sample signal is easily diluted, leading to a high false positive rate in complex mixed-sound environments. Third, the recognition thresholds in the inference stage are mostly fixed values ​​or determined through a one-time global search, without linkage with the false positive feedback during the training process. Furthermore, the thresholds for different bird species lack an adaptive adjustment mechanism, which can easily lead to an imbalance where some species have a high false positive rate and others have a high false negative rate.

[0004] Therefore, there is an urgent need for a multi-label bird sound recognition technology that can adapt to complex acoustic environments, accurately suppress false positives, and balance high recall and low false alarm rates. By optimizing feature extraction, fusion strategies, loss function design, and threshold adjustment mechanisms, this technology can address the shortcomings of existing technologies in terms of adaptability, false alarm control, and sample balance, and meet the needs of practical application scenarios such as ecological monitoring and species diversity assessment. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-label sound recognition method for birds, so as to solve the problems existing in the prior art mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multi-label sound recognition method for birds, comprising a training phase and an inference phase;

[0007] The training phase includes the following steps:

[0008] S1. Input audio data with multiple labels, and extract three complementary acoustic representations in parallel through the multi-view feature extraction module, namely Mel spectrogram, constant Q transform spectrum (CQT spectrum) and continuous wavelet transform spectrum (CWT spectrum).

[0009] S2. Each acoustic representation is input into the corresponding view feature extraction branch. Each branch extracts high-level abstract features through concatenated convolutional blocks and outputs a feature vector.

[0010] S3. The feature vectors output from each branch are fused using the feature fusion module to obtain a 1024-dimensional fused feature.

[0011] S4. The fusion feature input classification output module obtains the existence probability of each category after mapping and activation processing;

[0012] S5. The loss value is calculated using an asymmetric loss function that only weights the negative terms. The model parameters are updated by combining class-balanced weights based on the theory of effective sample number. At the same time, the false positive rate (FPR) of each category is calculated on the validation set. The negative term focusing parameter and pruning parameter of the asymmetric loss function are dynamically adjusted based on the false positive rate.

[0013] S6. Search for the optimal threshold for each category on the validation set, complete model training, and store the model parameters and the optimal threshold for each category;

[0014] The reasoning phase includes the following steps:

[0015] T1. Input the audio data to be recognized, call the model parameters, and repeat steps S1-S4 to obtain the existence probability of each category;

[0016] T2. Call the optimal threshold for each category, compare the existence probability with the corresponding optimal threshold, and output the multi-label recognition result.

[0017] Preferably, in step S1, the multi-view feature extraction module includes three parallel sub-modules: a Mel spectrum extraction sub-module, a constant Q transform spectrum extraction sub-module (CQT spectrum extraction sub-module), and a continuous wavelet transform spectrum extraction sub-module (CWT spectrum extraction sub-module); wherein,

[0018] The Mel spectrogram extraction submodule converts the audio time-domain signal into a frequency-domain representation through short-time Fourier transform (STFT), and then maps it to the Mel scale through the Mel filter bank, outputting a Mel spectrogram with dimensions B×1×T×84.

[0019] The CQT spectrum extraction submodule processes the audio through constant Q transformation and uses a logarithmic frequency scale to enhance the sensitivity to pitch and harmonic structure, outputting a CQT spectrum with dimensions B×1×T×64.

[0020] The CWT spectrum extraction submodule uses Moret wavelet as mother wavelet and captures transient and sharp cry features in the signal through multi-scale wavelet transform, outputting a CWT spectrum with dimensions of B×1×T×64.

[0021] Where B is the batch size and T is the time step, the three acoustic representations are aligned in the time dimension to form a multi-view feature representation.

[0022] Preferably, in step S2, each of the view feature extraction branches contains six concatenated convolutional blocks, each convolutional block consisting of two convolutional layers, two batch normalization layers and one average pooling layer;

[0023] The kernel size of the convolutional layer is 3×3, and the pooling window of the average pooling layer is 2×2.

[0024] The output channels of the six convolutional blocks are 64, 128, 256, 512, 1024, and 2048, respectively. Each branch is finally aggregated into a 2048-dimensional feature vector in the time dimension through global average pooling.

[0025] Preferably, in step S3, the feature fusion module completes the fusion using one of the following two methods:

[0026] Fully connected fusion: The 2048-dimensional feature vectors of the three branches are concatenated in the feature dimension to obtain a 6144-dimensional concatenated vector. This concatenated vector is then subjected to nonlinear transformation and dimensionality reduction through two fully connected layers. Each layer is followed by a ReLU activation function and a dropout layer with a dropout rate of 0.5, resulting in a 1024-dimensional fused feature output.

[0027] Self-attention fusion: The feature vectors of each view are mapped to a 512-dimensional common space through a linear projection layer to form a view sequence. The input Transformer encoder uses a self-attention mechanism to calculate the correlation weights between views. After the encoded sequence is aggregated by average pooling, it is reduced to 1024-dimensional fusion features through a fully connected layer.

[0028] Preferably, in step S4, the processing procedure of the classification output module is as follows:

[0029] It receives 1024-dimensional fused features, maps them to dimensions corresponding to the number of bird categories through a fully connected layer, outputs logits values, and converts them into the existence probability of each category through a Sigmoid activation function.

[0030] Preferably, in step S5, the asymmetric loss function weighted only by negative terms employs a differentiated focusing strategy for positive and negative samples:

[0031] The weighted formula for positive term loss is: ,in Predict the probability for positive samples. This is the positive term focusing parameter, used to focus on hard-to-classify positive samples;

[0032] The weighted formula for negative term losses is: ,in For clipping parameters, This is the negative term focusing parameter, used to suppress easily confused negative samples;

[0033] The class-balanced weights are applied only to negative losses to avoid diluting the learning signal of positive samples.

[0034] Preferably, in step S5, the false positive rate, after being smoothed by an exponential moving average, is used for parameter adjustment. The dynamic adjustment formulas for the negative term focusing parameter γ⁻ and the clipping parameter m are as follows:

[0035]

[0036]

[0037] in, Focusing on the baseline parameters for the negative terms, These are the cutting reference parameters. and To adjust the coefficient, The normalized false positive rate is used; the updated parameters avoid mutations through a pruning and smoothing update strategy.

[0038] Preferably, in step S6, the search for the optimal threshold employs one of the following two strategies:

[0039] Smooth F1 Maximization Strategy: Iterate through all possible threshold split points and select the threshold with the largest corresponding smooth F1 score;

[0040] The negative sample maximum strategy sets the threshold to the maximum predicted probability of negative samples in that class plus a small increment, ensuring extremely low false positives.

[0041] Preferably, in step T2, when the probability of the category exists is greater than the corresponding optimal threshold, the existence of the bird species is determined;

[0042] When the probability of a category is less than or equal to the corresponding optimal threshold, it is determined that the bird species does not exist, thus completing the conversion from probability to binary label.

[0043] Preferably, the two fusion methods of the feature fusion module can be dynamically selected according to the noise intensity and data resource conditions of the actual application scenario. In scenarios with complex noise, the self-attention fusion method is preferred, while in scenarios with limited resources, the fully connected fusion method is preferred.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] 1) This application sets up a Mel spectrogram extraction submodule, a CQT spectrogram extraction submodule, and a CWT spectrogram extraction submodule in parallel to capture complementary features in audio signals related to human hearing perception, such as energy distribution, pitch and harmonic structure, transients and sharp cries. The three acoustic representations are aligned in the time dimension to form a comprehensive multi-view feature representation. Compared with the fixed view combination or single feature extraction method in the prior art, this design can effectively avoid the problem of single view failure due to noise and occlusion, provide a rich and reliable feature foundation for subsequent recognition tasks, and significantly improve the feature adaptation capability in complex sound scene environments.

[0046] 2) The loss function of this application adopts different focusing strategies for positive and negative samples. It focuses on difficult-to-classify positive samples by weighting the positive loss, while applying the class balance weight only to the negative loss to avoid diluting the learning signal of positive samples. Through the pruning and focusing design of the negative loss, the suppression effect on easily confused negative samples is strengthened. This design solves the defects of existing symmetric loss functions, such as insufficient focus on positive and negative samples and easy dilution of positive sample signals. While ensuring the recall rate of rare bird species in long-tail distribution, it effectively reduces the overall false positive rate and improves the accuracy of the identification results.

[0047] 3) This application calculates the false positive rate of each category during the verification stage, smooths it with an exponential moving average, and dynamically adjusts the negative term focusing and pruning parameters of the asymmetric loss function. It increases the negative term focusing and pruning intensity for categories with high false positive rates and maintains a mild optimization strategy for categories with low false positive rates. This mechanism establishes a linkage between the training process and false positive feedback. Compared with the fixed loss parameters or uniform weighting in the prior art, it can perform customized optimization for the false positive characteristics of different categories, achieve accurate and fine-grained suppression of false positives, and further reduce the false positive accumulation problem in complex mixed-sound environments.

[0048] 4) This application independently searches for the optimal threshold for each bird species on the validation set, providing two strategies: smooth F1 maximization and negative sample maximum value. Compared with the design of fixed threshold or global single threshold in the prior art, it can adapt to the feature distribution differences and identification risk requirements of different species, effectively solve the problems of high false positives for some species and high false negatives for some species, and achieve balanced optimization of identification accuracy and recall for each category.

[0049] 5) This application achieves a balance between false positive suppression and recognition accuracy through a collaborative design of multi-view complementary feature extraction, negative-only weighted asymmetric loss, false positive rate feedback parameter tuning, and class-by-class threshold optimization. It achieves this balance from feature input and model training to inference output. In practical application scenarios such as complex mixing, noise interference, and signal occlusion, it can significantly reduce the false positive rate while maintaining a high recognition recall rate. Compared with existing technologies, it can better meet the practical needs of ecological monitoring and species diversity assessment, and has stronger practicality and promotion value. Attached Figure Description

[0050] Figure 1 This is a flowchart of the application;

[0051] Figure 2 This is a structural diagram of the multi-view feature extraction module of this application;

[0052] Figure 3 This is a flowchart of the method in this application. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In the description of the invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front end," "rear end," "both ends," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0055] In the description of the invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0056] In the description of the invention, it should be noted that the execution order of the steps is not limited by the sequence number. The possible changes in the order of some steps, the synchronous execution of steps, and the split execution of steps are all within the scope of protection of this application.

[0057] Please see Figure 1-3 The present invention provides a technical solution: a method for multi-label bird sound recognition, including a training phase and an inference phase;

[0058] The training phase includes the following steps:

[0059] S1. Input audio data with multiple labels, and extract three complementary acoustic representations in parallel through the multi-view feature extraction module, namely Mel spectrogram, constant Q transform spectrum (hereinafter referred to as CQT spectrum) and continuous wavelet transform spectrum (hereinafter referred to as CWT spectrum).

[0060] S2. Each acoustic representation is input into the corresponding view feature extraction branch. Each branch extracts high-level abstract features through concatenated convolutional blocks and outputs a feature vector.

[0061] S3. The feature vectors output from each branch are fused using the feature fusion module to obtain a 1024-dimensional fused feature.

[0062] S4. The fusion feature input classification output module obtains the existence probability of each category after mapping and activation processing;

[0063] S5. The loss value is calculated using an asymmetric loss function that only weights the negative terms. The model parameters are updated by combining class balance weights based on the theory of effective sample number. At the same time, the false positive rate (FPR) of each category is calculated on the validation set. The negative term focusing parameters and pruning parameters of the asymmetric loss function are dynamically adjusted based on the false positive rate.

[0064] S6. Search for the optimal threshold for each category on the validation set, complete model training, and store the model parameters and the optimal threshold for each category;

[0065] The reasoning stage includes the following steps:

[0066] T1. Input the audio data to be recognized, call the model parameters, and repeat steps S1-S4 to obtain the existence probability of each category;

[0067] T2. Call the optimal threshold for each category, compare the existence probability with the corresponding optimal threshold, and output the multi-label recognition result.

[0068] Specifically, this application constructs a complete multi-label bird sound recognition process from the training phase to the inference phase. It clarifies the core steps of multi-view feature extraction, feature fusion, asymmetric loss calculation, false positive rate feedback parameter tuning, class-by-class threshold search, and model parameter and threshold storage during the training phase. It also defines the execution logic for calling stored parameters and thresholds to complete recognition during the inference phase, thus comprehensively solving the problems of lack of systematic optimization and disconnect between training and inference in existing technologies. Compared to the shortcomings of existing technologies, such as fixed view combinations, unreasonable loss function design, and lack of linkage between thresholds and training, this application provides comprehensive feature support for recognition through multi-view feature extraction, achieves positive and negative sample differentiation optimization through an asymmetric loss function that only weights negative terms, accurately suppresses false alarms through dynamic parameter tuning based on false positive rate feedback, adapts to different species distributions through class-by-class threshold search, and ensures consistency and synergy between training and inference by providing a unified and reliable execution basis for the inference phase using model parameters and optimal thresholds stored during the training phase. Ultimately, this complete process effectively overcomes the bottlenecks of existing technologies, such as weak adaptability, high false positive rate, and imbalance between recall and false alarm in complex mixed-sound environments. It achieves a balance between high recall and low false alarm rate, fully meeting the needs of accurate identification in practical scenarios such as ecological monitoring and species diversity assessment.

[0069] In step S1, the multi-view feature extraction module includes three parallel sub-modules: a Mel spectrum extraction sub-module, a constant Q-transform spectrum extraction sub-module (hereinafter referred to as the CQT spectrum extraction sub-module), and a continuous wavelet transform spectrum extraction sub-module (hereinafter referred to as the CWT spectrum extraction sub-module); among them,

[0070] The Mel spectrogram extraction submodule converts the audio time-domain signal into a frequency-domain representation through short-time Fourier transform (STFT), and then maps it to the Mel scale through the Mel filter bank, outputting a Mel spectrogram with dimensions B×1×T×84.

[0071] The CQT spectrum extraction submodule processes the audio through constant Q transformation and uses a logarithmic frequency scale to enhance sensitivity to pitch and harmonic structure, outputting a CQT spectrum with dimensions B×1×T×64.

[0072] The CWT spectrum extraction submodule uses the Moret wavelet as the mother wavelet and captures transient and sharp cry features in the signal through multi-scale wavelet transform, outputting a CWT spectrum with dimensions of B×1×T×64.

[0073] Where B is the batch size and T is the time step, the three acoustic representations are aligned in the time dimension to form a multi-view feature representation.

[0074] Specifically, the Mel spectrogram extraction submodule captures energy distribution features consistent with human auditory perception through short-time Fourier transform and Mel filter banks; the CQT spectrogram extraction submodule enhances sensitivity to pitch and harmonic structure through constant Q transform and logarithmic frequency scaling; and the CWT spectrogram extraction submodule accurately captures transient and sharp cry features through Morlet wavelet and multi-scale transform. These three submodules are aligned in the time dimension to form a complementary multi-view feature representation. Compared to existing single-view Mel spectrogram convolutional networks, which are insufficient in characterizing harmonics and transients and neglect view-specificity in multi-feature fusion, this design comprehensively covers key information of the audio signal, effectively avoids the risk of single-view failure due to noise and occlusion, provides a rich and reliable feature foundation for subsequent feature fusion and classification, significantly improves the completeness and adaptability of feature extraction in complex sound environments, and lays a solid foundation for improving overall recognition accuracy.

[0075] In step S2, each view feature extraction branch contains six concatenated convolutional blocks, each of which consists of two convolutional layers, two batch normalization layers and one average pooling layer.

[0076] The kernel size of the convolutional layer is 3×3, and the pooling window of the average pooling layer is 2×2.

[0077] The output channels of the six convolutional blocks are 64, 128, 256, 512, 1024, and 2048, respectively. Each branch is finally aggregated into a 2048-dimensional feature vector in the time dimension through global average pooling.

[0078] Specifically, this application defines the specific structure of the view feature extraction branch, including the composition of six concatenated convolutional blocks, the size of the convolutional kernels, the size of the pooling window, and the increasing rule of the number of output channels. This solves the problems of simple feature extraction branch structures and insufficient high-level abstract feature capture capabilities in existing technologies. Each convolutional block consists of two convolutional layers, two batch normalization layers, and one average pooling layer. The 3×3 convolutional kernel enhances the local feature capture capability, the 2×2 pooling window gradually reduces the spatial resolution of the feature map, and the increasing design of the number of output channels from 64 to 2048 achieves a gradual enhancement of feature representation capability. Finally, a high-dimensional, highly discriminative 2048-dimensional feature vector is obtained through global average pooling aggregation. Compared to the shortcomings of existing technologies, such as insufficient feature extraction depth and low level of abstraction leading to poor recognition robustness, this deep convolutional structure can fully extract high-level abstract features from the original features of multiple views. Furthermore, the independent training of the three branches ensures the exclusive optimization of features for various views, effectively improving the discriminative ability of features. This makes the subsequent feature fusion more targeted, thereby enhancing the ability to distinguish the sounds of different bird species in complex environments and reducing the risk of misjudgment due to insufficient feature recognition.

[0079] As an additional note, each view feature extraction branch employs an independent initialization and joint fine-tuning training mode to ensure that each branch can fully learn the specific features of its corresponding view. During branch initialization, the Mel spectrogram extraction branch initializes the first three convolutional blocks with ImageNet pre-trained weights, while the CQT and CWT spectrogram extraction branches initialize all convolutional layer parameters with a He normal distribution. Batch normalization layer parameters are initialized to a mean of 0 and a variance of 1. In the independent training phase, each branch iterates independently for 10-15 rounds using the cross-entropy loss function, with a learning rate set to 1e-4, and the AdamW optimizer is used. In the joint fine-tuning phase, each branch is combined with the feature fusion module and the classification output module to form a complete model. The asymmetric loss function of this invention is then used for another 20-30 rounds of training, with the learning rate decaying to 1e-5. Gradient backpropagation is used to synchronously update all module parameters. This training mode ensures the specificity of feature extraction for each branch while achieving collaborative adaptation between modules, resolving the feature conflicts or insufficient adaptation problems that easily occur in existing multi-branch model training.

[0080] In step S3, the feature fusion module completes the fusion using one of the following two methods:

[0081] Fully connected fusion: The 2048-dimensional feature vectors of the three branches are concatenated in the feature dimension to obtain a 6144-dimensional concatenated vector. This concatenated vector is then subjected to nonlinear transformation and dimensionality reduction through two fully connected layers. Each layer is followed by a ReLU activation function and a dropout layer with a dropout rate of 0.5, resulting in a 1024-dimensional fused feature output.

[0082] Self-attention fusion: The feature vectors of each view are mapped to a 512-dimensional common space through a linear projection layer to form a view sequence. The input Transformer encoder uses a self-attention mechanism to calculate the correlation weights between views. After the encoded sequence is aggregated by average pooling, it is reduced to 1024-dimensional fusion features through a fully connected layer.

[0083] Specifically, fully connected fusion achieves effective integration of multi-view features through feature concatenation, nonlinear transformation, and dimensionality reduction, while the dropout layer reduces the risk of overfitting. Self-attention fusion maps linear projection to a common space and uses a Transformer encoder to calculate the correlation weights between views, dynamically identifying reliable views and allocating higher attention to them, thus accurately capturing the interaction relationships between views. Compared to existing technologies that achieve fusion through convolution or simple concatenation and cannot adapt to dynamic changes in view credibility, the two fusion methods can be flexibly selected according to the actual scenario: fully connected fusion has a simple structure and high computational efficiency, adapting to resource-constrained scenarios; self-attention fusion can accurately mine the correlations between views, adapting to scenarios with complex noise and large differences in view credibility. This design fully leverages the complementary advantages of multi-view features, effectively integrates the high-level abstract features of each branch, improves the comprehensive discriminative ability of the fused features, avoids the limitations of a single fusion method, and further strengthens the recognition robustness in complex soundscape environments.

[0084] In step S4, the processing procedure of the classification output module is as follows:

[0085] It receives 1024-dimensional fused features, maps them to dimensions corresponding to the number of bird categories through a fully connected layer, outputs logits values, and converts them into the existence probability of each category through a Sigmoid activation function.

[0086] Specifically, this application addresses the problems of inaccurate category probability output and failure to accurately reflect the presence or absence of species in existing technologies. The fully connected layer maps the 1024-dimensional fused features to dimensions corresponding to the number of bird categories, ensuring a precise mapping between features and categories. The sigmoid activation function converts logits values ​​into presence probabilities in the 0-1 range, intuitively reflecting the likelihood of each bird category's presence. Compared to the shortcomings of existing technologies, such as unreasonable probability transformation methods and output results that cannot effectively support subsequent loss calculations and threshold determinations, this process accurately outputs the presence probability of each category, providing a reliable basis for loss calculations during the training phase. This ensures that the negative-only weighted asymmetric loss function can achieve differentiated optimization of positive and negative samples based on accurate probabilities. Simultaneously, the accurate probability output provides a clear basis for threshold comparisons during the inference phase, making the determination of the presence or absence of each category more scientific, effectively reducing the risk of false positives or false negatives caused by distorted probability outputs, and improving the accuracy and reliability of the recognition results.

[0087] In step S5, the asymmetric loss function weighted only by the negative term employs a differentiated focusing strategy for positive and negative samples:

[0088] The weighted formula for positive term loss is: ,in Predict the probability for positive samples. This is the positive term focusing parameter, used to focus on hard-to-classify positive samples;

[0089] The weighted formula for negative term losses is: ,in For clipping parameters, This is the negative term focusing parameter, used to suppress easily confused negative samples;

[0090] Class-balanced weights are applied only to negative losses to avoid diluting the learning signal of positive samples.

[0091] Specifically, positive losses are achieved through... Weighting allows for a focus on difficult-to-classify positive samples, ensuring effective recall of hard-to-identify samples such as rare bird species; negative losses are addressed through... Weighted loss can accurately suppress easily confused negative samples and reduce meaningless false positives. Class-balanced weights are applied only to negative loss terms, which alleviates the imbalance caused by differences in sample numbers and avoids dilution of the learning signal for positive samples. Compared with the shortcomings of existing technologies such as binary cross-entropy and symmetric Focal loss in suppressing easily classified negative samples and having a high false positive rate for long-tailed species, this asymmetric loss function achieves targeted optimization for positive and negative samples. While ensuring the recall rate of difficult-to-classify positive samples and rare species, it significantly enhances the suppression effect on easily confused negative samples, effectively reduces the overall false positive rate in complex mixed-sound environments, and solves the contradiction between recall rate and false positive suppression that existing loss functions cannot balance, thus improving the accuracy and practicality of the recognition results.

[0092] In step S5, the false positive rate, after being smoothed by an exponential moving average, is used for parameter adjustment. The dynamic adjustment formulas for the negative term focusing parameter γ⁻ and the clipping parameter m are as follows:

[0093]

[0094]

[0095] in, Focusing on the baseline parameters for the negative terms, These are the cutting reference parameters. and To adjust the coefficient, The normalized false positive rate is used; the updated parameters avoid mutations through a pruning and smoothing update strategy.

[0096] Specifically, by calculating the false positive rate of each category in the validation set and smoothing it using an exponential moving average, the negative term focusing parameter γ⁻ and the pruning parameter m are dynamically adjusted using a preset formula. This achieves customized adjustments, increasing the suppression intensity for categories with high false positive rates and maintaining a gentle optimization for categories with low false positive rates. Furthermore, the pruning and smoothing update strategies avoid model instability caused by parameter mutations. Compared to the shortcomings of existing technologies with fixed loss parameters and uniform weighting that cannot adapt to the false positive characteristics of different categories, this dynamic parameter tuning mechanism establishes a close linkage between the training process and false positive feedback. It can perform fine-grained optimization for the false positive situation of different bird species, accurately suppressing false positives of various categories, effectively solving the problem of false positive accumulation in long-tailed distributions or resonance backgrounds, further improving the model's adaptability to complex environments, and ensuring the stability and reliability of recognition performance.

[0097] In step S6, the search for the optimal threshold employs one of the following two strategies:

[0098] Smooth F1 Maximization Strategy: Iterate through all possible threshold split points and select the threshold with the largest corresponding smooth F1 score;

[0099] The negative sample maximum strategy sets the threshold to the maximum predicted probability of negative samples in that class plus a small increment, ensuring extremely low false positives.

[0100] Specifically, the smoothed F1 maximization strategy selects the optimal value by traversing threshold cutoff points, achieving a balance between precision and recall for each category. The negative sample maximum strategy, by setting a threshold higher than the maximum predicted probability of negative samples, minimizes the false positive rate, meeting the needs of scenarios with high false alarm control requirements. Compared to the rigid threshold settings and inability to adapt to different species feature distributions and identification risks in existing technologies, class-by-class threshold search can customize the optimal threshold for the specific distribution characteristics of each bird species, effectively avoiding the imbalance caused by global thresholds, where some species have excessively high false positives and others have excessively high false negatives. Both strategies can be flexibly selected according to actual application needs, ensuring balanced recognition performance while improving the flexibility of threshold settings, further enhancing the accuracy and rationality of the overall recognition results, and meeting the recognition needs in different scenarios.

[0101] In step T2, when the probability of a category being present is greater than the corresponding optimal threshold, the bird species is determined to exist.

[0102] When the probability of a category is less than or equal to the corresponding optimal threshold, it is determined that the bird species does not exist, thus completing the conversion from probability to binary label.

[0103] Specifically, this application clarifies the rules for determining the existence of a category during the inference stage, namely, the comparison logic between the probability of existence and the optimal threshold. This solves the problem of unclear probability-to-binary label conversion rules and inconsistent judgment standards in existing technologies, which leads to chaotic recognition results. When the probability of a category's existence is greater than the corresponding optimal threshold, the species is determined to exist; when it is less than or equal to the threshold, it is determined not to exist. This clear and explicit rule ensures the consistency and scientific nature of the probability-to-binary label conversion. Existing technologies suffer from unclear judgment rules and susceptibility to subjective factors. This rule, based on the optimal threshold determined during the training stage, can accurately match the recognition characteristics of each category of species, avoiding misjudgments caused by unclear judgment logic and ensuring the consistency and reliability of recognition results for different bird species. Simultaneously, this judgment rule synergizes with the category-by-category optimal threshold search strategy, further consolidating the balance between recognition accuracy and recall for each category, effectively reducing the risk of missed detections and false alarms, making the final multi-label recognition results more credible and directly meeting the accuracy requirements of practical application scenarios such as ecological monitoring and species diversity assessment.

[0104] The feature fusion module can dynamically select between two fusion methods based on the noise intensity and data resource conditions of the actual application scenario. In scenarios with complex noise, the self-attention fusion method is preferred, while in scenarios with limited resources, the fully connected fusion method is preferred.

[0105] Specifically, the dynamic selection criterion for feature fusion methods is clearly defined, namely, adapting fully connected fusion or self-attention fusion based on noise intensity and data resource conditions. In noisy and complex scenarios, self-attention fusion is preferred, as it can accurately calculate the correlation weights between views, dynamically focus on reliable views, and effectively resist noise interference. In resource-constrained scenarios, fully connected fusion is preferred, as its simple structure and high computational efficiency ensure recognition efficiency under limited resources. Existing technologies suffer from fixed fusion methods and cannot balance recognition accuracy in complex scenarios with computational efficiency in resource-constrained scenarios. This dynamic selection mechanism achieves precise adaptation between the fusion method and the application scenario, ensuring robustness in noisy and complex environments while meeting the high-efficiency recognition requirements in resource-constrained scenarios, thus improving the flexibility and practicality of the technical solution. Furthermore, this mechanism fully leverages the advantages of both fusion methods, avoiding the limitations of a single fusion method, further broadening the application scope of the technical solution and enhancing its promotional value.

[0106] As an additional note, this invention integrates general engineering optimization modules such as data augmentation, mixed-precision training, and gradient pruning during the implementation of its technical solution, to further improve the training efficiency and generalization ability of the model. Specifically, the data augmentation module employs operations such as time stretching (stretching coefficient range 0.8-1.2), random pruning (pruning length 0.5-1.5 seconds), Gaussian noise addition (signal-to-noise ratio 20-30dB), and spectral masking (time mask length 0-20%, frequency mask quantity 1-2) to effectively expand the diversity of training data and alleviate overfitting problems in small sample scenarios. Mixed-precision training combines FP16 half-precision and FP32 single-precision training, reducing memory usage and improving training speed while ensuring model training accuracy. The gradient pruning module sets the gradient norm threshold to 1.0 to avoid model instability caused by gradient explosion during training. These engineering optimization modules work in conjunction with the core technical solution to address the lack of automated adjustment in resource-constrained scenarios in existing technologies, enabling this invention to operate efficiently under different hardware resources and data scales.

[0107] As an additional note, this invention forms a specialized adaptation scheme for small-sample scenarios through the complementarity of multi-view features and the optimized design of the loss function. In small-sample datasets, due to the extremely limited sample size of some bird species, single-view features are easily affected by data bias. However, the multi-view feature extraction module of this invention can capture signal features from different dimensions, reducing the bias caused by a single data distribution. At the same time, the asymmetric loss function that only weights negative terms avoids model bias caused by the imbalance of positive and negative samples in small-sample data by protecting the learning signal of positive samples and strengthening the suppression of hard negative samples. Furthermore, in small-sample scenarios, the trained view feature extraction branch parameters can be used as pre-training weights, and only the feature fusion module and classification output module are fine-tuned. Combined with a class-by-class threshold optimization strategy, the recognition performance of small-sample species can be further improved, enabling reliable promotion of the technical solution in scenarios with limited data resources.

[0108] Example 1: Specific Implementation of Multi-Label Sound Recognition for Birds in an Ecological Reserve

[0109] 1. Implementation Scenarios and Dataset Preparation

[0110] This embodiment is applied to bird species diversity monitoring in a mountain ecological reserve, aiming to identify 30 common and rare bird species. The dataset covers various soundscapes including forests, thickets, and streams, and includes interference noise such as wind noise, insect chirping, and human activity. The dataset contains 10,000 audio samples, each 0.5-2.0 seconds long, with a sampling rate of 16kHz, mono, and 16-bit quantization. Samples are labeled in a multi-label format, with rare bird species (such as the red-bellied tragopan and silver pheasant) comprising 15% of the samples, simulating a long-tailed distribution. The training, validation, and test sets are divided in a 7:1:2 ratio, with the validation and test sets containing 50% samples from complex noisy scenes.

[0111] 2. Technical Solution Parameter Setting and Execution Steps

[0112] (1) Multi-view feature extraction

[0113] Mel spectrogram extraction: The frame length of the short-time Fourier transform (STFT) is 25ms, the frame shift is 10ms, the window function is Hanning window, the number of Mel filter banks is 84, and the output dimension after the magnitude logarithmic transformation is 32×1×150×84 (B=32 is the batch size, T=150 is the time step).

[0114] CQT spectrum extraction: frequency range 200-8000Hz, constant Q factor is 12, number of frequency bins is 64, output dimension is 32×1×150×64;

[0115] CWT spectrum extraction: The mother wavelet is Morlet wavelet, with 64 scales, scale range 1-64, and output dimension 32×1×150×64.

[0116] The three features are kept synchronized in the time dimension through a frame alignment mechanism, forming a multi-view feature set.

[0117] (2) View feature extraction branch

[0118] Each branch contains 6 concatenated convolutional blocks, with the following structure:

[0119] Convolutional blocks 1-6: Convolutional layer (3×3 convolutional kernel, padding=1)×2 → Batch normalization layer (momentum 0.9, epsilon=1e-5)×2 → Average pooling layer (2×2 pooling window, stride 2);

[0120] The number of output channels is set to 64, 128, 256, 512, 1024, and 2048 respectively;

[0121] Finally, the feature vectors are aggregated into a 2048-dimensional feature vector through global average pooling. The output feature vectors of the three branches are denoted as follows: , , .

[0122] (3) Feature fusion

[0123] This embodiment uses a self-attention fusion method, the specific process of which is as follows:

[0124] 1. Linear projection: Projected through three independent linear layers. , , Mapped to a 512-dimensional public space respectively, we obtain , , ;

[0125] 2. View sequence construction: Concatenating into a view sequence V=[ (Dimensions 32×3×512);

[0126] 3. Transformer encoding: 2-layer encoder, 8 multi-head attention heads, 2048 dimensions of FeedForward network, dropout rate of 0.1, output encoding sequence after calculating the correlation weights between views;

[0127] 4. Aggregation and Dimensionality Reduction: After average pooling, the dimensionality is reduced to 1024 dimensions through a fully connected layer and the features are fused. .

[0128] (4) Classification output and loss calculation

[0129] Classified output: The input is mapped to logits by a fully connected layer (1024-dimensional input, 30-dimensional output), and then converted into the presence probability of each class by a sigmoid activation function. (i=1,2,...,30);

[0130] Applications of asymmetric loss functions:

[0131] 1. Weighted average of positive term losses: = ×log( ),in =2.0, focusing on difficult-to-classify positive samples;

[0132] 2. Weighted average of negative losses: = ×log(1- ), where the initial =3.0, =0.1;

[0133] 3. Balanced weights: = N=7000 (total number of samples in the training set). Let C = 30 (number of classes) be the number of samples in class i. This weight only applies to the negative loss, i.e. = × ;

[0134] 4. Total Losses: =(1 / )×Σ( + ), The batch size is 32.

[0135] (5) False positive rate feedback dynamic parameter adjustment

[0136] 1. Calculate the false positive rate for each category after each round of validation. =Number of false alarms / Number of actual negative samples;

[0137] 2. FPR normalization: = Mapped to the interval [0,1];

[0138] 3. EMA smoothing: =0.9× +0.1× ,initial =0.5;

[0139] 4. Parameter adjustment:

[0140] = ×(1+k_× )

[0141] Where, k_=0.8;

[0142] = ×(1+ × ),

[0143] in, =0.5;

[0144] Example: Rare bird (Category 5, Red-bellied Tragopan) Round 10 validation =0.32, after adjustment =3.0×(1+0.8×0.32)=3.768, =0.1×(1+0.5×0.32)=0.116; Common birds (Category 12, Sparrow) =0.08, after adjustment =3.0×(1+0.8×0.08)=3.192, =0.1×(1+0.5×0.08)=0.104;

[0145] 5. Parameter clipping: Limited to between 2.0 and 5.0, Limit it to between 0.05 and 0.3 to avoid abnormal parameters.

[0146] (6) Class-by-class threshold optimization and reasoning

[0147] Threshold search: A smoothed F1 maximization strategy is adopted, traversing the threshold range [0.01, 0.99] with a step size of 0.01, calculating the smoothed F1 score (β=1.0), and selecting the optimal threshold. ;

[0148] Example: Optimal threshold for Category 5 (Red-bellied Tragopan) =0.38, optimal threshold for category 12 (sparrow) =0.26;

[0149] Reasoning process: Input the audio to be identified (1.8 seconds, including the call of the red-bellied tragopan and stream noise), after feature extraction and fusion, the probability of existence of category 5 is determined. =0.45, greater than =0.38, indicating its existence; the probabilities of other categories are all less than the corresponding thresholds, and the final multi-label result is output as {Red-bellied Tragopan}.

[0150] 3. Implementation effect verification

[0151] Compared with existing technologies (simple multi-feature concatenation, symmetric Focal loss, fixed threshold of 0.5), the key performance indicators of this embodiment are as follows:

[0152] Performance indicators Existing technology Embodiments of the present invention Increase Average false positive rate 18.7% 10.8% 42.2% Rare bird recall rate 62.3% 84.1% 35.0% Overall average F1 score 0.71 0.91 28.2% Accuracy in complex noisy scenes 65.8% 87.3% 32.7%

[0153] The verification results show that the present invention significantly reduces the false positive rate in complex soundscapes and improves the recall rate of rare birds through the synergistic effect of multi-view feature fusion, asymmetric loss function and dynamic parameter tuning, and class-by-class threshold optimization, thus fully meeting the actual needs of ecological monitoring scenarios.

[0154] Example 2: Simplified Implementation in Resource-Constrained Scenarios (Fully Connected Fusion)

[0155] This embodiment targets portable ecological monitoring devices (hardware resources: CPU is ARM Cortex-A76, graphics memory is 2GB), and adopts a fully connected fusion approach to simplify the computation process:

[0156] Feature fusion: splicing , , A 6144-dimensional vector is obtained, which is then passed through a fully connected layer 1 (6144-dimensional input, 2048-dimensional output, ReLU activation, dropout=0.5) and a fully connected layer 2 (2048-dimensional input, 1024-dimensional output, ReLU activation, dropout=0.5) to obtain fused features.

[0157] Loss function parameter simplification: =1.5, =2.5, =0.08, k_=0.6, =0.4;

[0158] Threshold search: Employs a negative sample maximum strategy. =max( )+0.05, where The maximum predicted probability for a negative sample of class i;

[0159] Implementation results: While ensuring recognition accuracy (average F1=0.86), the model inference speed is improved by 40%, the memory usage is reduced by 55%, and it is suitable for resource-constrained portable device application scenarios.

[0160] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method of bird multi-label sound recognition, characterized by, Includes a training phase and an inference phase; The training phase includes the following steps: S1. Input audio data with multiple labels, and extract three complementary acoustic representations in parallel through the multi-view feature extraction module, namely Mel spectrogram, constant Q transform spectrum (CQT spectrum) and continuous wavelet transform spectrum (CWT spectrum). S2. Each acoustic representation is input into the corresponding view feature extraction branch. Each branch extracts high-level abstract features through concatenated convolutional blocks and outputs a feature vector. S3. The feature vectors output from each branch are fused using the feature fusion module to obtain a 1024-dimensional fused feature. S4. The fusion feature input classification output module obtains the existence probability of each category after mapping and activation processing; S5. The loss value is calculated using an asymmetric loss function that only weights the negative terms. The model parameters are updated by combining class-balanced weights based on the theory of effective sample number. At the same time, the false positive rate (FPR) of each category is calculated on the validation set. The negative term focusing parameter and pruning parameter of the asymmetric loss function are dynamically adjusted based on the false positive rate. S6. Search for the optimal threshold for each category on the validation set, complete model training, and store the model parameters and the optimal threshold for each category; The reasoning phase includes the following steps: T1. Input the audio data to be recognized, call the model parameters, and repeat steps S1-S4 to obtain the existence probability of each category; T2. Call the optimal threshold for each category, compare the existence probability with the corresponding optimal threshold, and output the multi-label recognition result.

2. The bird multi-label sound recognition method of claim 1, wherein, In step S1, the multi-view feature extraction module includes three parallel sub-modules: a Mel spectrum extraction sub-module, a constant Q transform spectrum extraction sub-module (CQT spectrum extraction sub-module), and a continuous wavelet transform spectrum extraction sub-module (CWT spectrum extraction sub-module); wherein, The Mel spectrogram extraction submodule converts the audio time-domain signal into a frequency-domain representation through short-time Fourier transform (STFT), and then maps it to the Mel scale through the Mel filter bank, outputting a Mel spectrogram with dimensions B×1×T×84. The CQT spectrum extraction submodule processes the audio through constant Q transformation and uses a logarithmic frequency scale to enhance the sensitivity to pitch and harmonic structure, outputting a CQT spectrum with dimensions of B×1×T×64. The CWT spectrum extraction submodule uses Moret wavelet as mother wavelet and captures transient and sharp cry features in the signal through multi-scale wavelet transform, outputting a CWT spectrum with dimensions of B×1×T×64. Where B is the batch size and T is the time step, the three acoustic representations are aligned in the time dimension to form a multi-view feature representation.

3. The bird multi-label sound recognition method of claim 1, wherein, In step S2, each of the view feature extraction branches contains six concatenated convolutional blocks, each of which consists of two convolutional layers, two batch normalization layers and one average pooling layer. The kernel size of the convolutional layer is 3×3, and the pooling window of the average pooling layer is 2×2. The output channels of the six convolutional blocks are 64, 128, 256, 512, 1024, and 2048, respectively. Each branch is finally aggregated into a 2048-dimensional feature vector in the time dimension through global average pooling.

4. The bird multi-label sound recognition method of claim 1, wherein, In step S3, the feature fusion module completes the fusion using one of the following two methods: Fully connected fusion: The 2048-dimensional feature vectors of the three branches are concatenated in the feature dimension to obtain a 6144-dimensional concatenated vector. This concatenated vector is then subjected to nonlinear transformation and dimensionality reduction through two fully connected layers. Each layer is followed by a ReLU activation function and a dropout layer with a dropout rate of 0.5, resulting in a 1024-dimensional fused feature output. Self-attention fusion: The feature vectors of each view are mapped to a 512-dimensional common space through a linear projection layer to form a view sequence. The input Transformer encoder uses a self-attention mechanism to calculate the correlation weights between views. After the encoded sequence is aggregated by average pooling, it is reduced to 1024-dimensional fusion features through a fully connected layer.

5. The bird multi-label sound recognition method of claim 1, wherein, In step S4, the processing procedure of the classification output module is as follows: It receives 1024-dimensional fused features, maps them to dimensions corresponding to the number of bird categories through a fully connected layer, outputs logits values, and converts them into the existence probability of each category through a Sigmoid activation function.

6. The bird multi-label sound recognition method of claim 1, wherein, In step S5, the negative-only weighted asymmetric loss function employs a differentiated focusing strategy for positive and negative samples: The weighted formula for positive term loss is: ,in Predict the probability for positive samples. This is the positive term focusing parameter, used to focus on hard-to-classify positive samples; The weighted formula for negative term losses is: ,in For clipping parameters, This is the negative term focusing parameter, used to suppress easily confused negative samples; The class-balanced weights are applied only to negative losses to avoid diluting the learning signal of positive samples.

7. The bird multi-tag sound recognition method according to claim 1, characterized in that, In step S5, the false positive rate, after being smoothed by an exponential moving average, is used for parameter adjustment. The dynamic adjustment formulas for the negative term focusing parameter γ⁻ and the clipping parameter m are as follows: in, Focusing on the baseline parameters for the negative terms, These are the cutting reference parameters. and To adjust the coefficient, The normalized false positive rate is used; the updated parameters avoid mutations through a pruning and smoothing update strategy.

8. The bird multi-tag sound recognition method according to claim 1, characterized in that, In step S6, the search for the optimal threshold employs one of the following two strategies: Smooth F1 Maximization Strategy: Iterate through all possible threshold split points and select the threshold with the largest corresponding smooth F1 score; The negative sample maximum strategy sets the threshold to the maximum predicted probability of negative samples in that class plus a small increment, ensuring extremely low false positives.

9. The bird multi-tag sound recognition method according to claim 1, characterized in that, In step T2, when the probability of a category being present is greater than the corresponding optimal threshold, the bird species is determined to exist. When the probability of a category is less than or equal to the corresponding optimal threshold, it is determined that the bird species does not exist, thus completing the conversion from probability to binary label.

10. The bird multi-tag sound recognition method according to claim 1, characterized in that, The two fusion methods of the feature fusion module can be dynamically selected according to the noise intensity and data resource conditions of the actual application scenario. In scenarios with complex noise, the self-attention fusion method is preferred, while in scenarios with limited resources, the fully connected fusion method is preferred.

Images