A pet sound emotion recognition method and system based on self-supervised learning

Abstract

The application discloses a pet sound emotion recognition method and system based on self-supervised learning, comprising the following steps: obtaining original pet audio to be recognized; converting the pet call time domain signal into a time-frequency domain audio feature sequence; inputting the audio feature sequence into a time sequence emotion recognition layer, using a self-supervised pre-training backbone network to extract frame-by-frame acoustic features, obtaining recalibrated time sequence features through feature projection, local time sequence perception and channel attention recalibration; inserting a learnable global aggregation token into the recalibrated time sequence features and superimposing position coding to obtain sequence features; using a global token guided-local enhancement adaptive fusion strategy to fuse the global aggregation token output features and the pooling statistical features of all effective frames to obtain audio level feature representation; inputting the audio level feature representation into a classification projection layer to output at least ten-dimensional pet sound emotion state probability distribution. Multi-dimensional refined emotion output: outputting at least ten-dimensional pet sound emotion probability distribution.

Description

Technical Field

[0001] This invention relates to the fields of audio signal processing, speech recognition and deep learning, specifically to a method and system for pet voice emotion recognition based on self-supervised learning. Background Technology

[0002] With the rapid development of the pet economy and the deepening trend of pets becoming "family members," accurately and continuously understanding the emotional state of pets has become a common underlying need in many fields such as pet medical care, behavioral research, smart hardware, and content platforms. However, in stark contrast to the abundant achievements already accumulated in the field of human speech emotion computing, pet voice emotion recognition faces three fundamental challenges: First, complete semantic lack—pet vocalizations do not have a semantic system similar to human language; emotions are entirely embedded in acoustic features such as pitch, loudness, rhythm, and spectrum, which cannot be directly transferred from human speech emotion recognition methods. Second, emotional expression is highly embedded in temporal acoustic sequences, such as the gradual change in pitch of a growl before an attack, or the abrupt change in vocalization from relaxation to fear; single-frame spectral snapshots cannot capture these temporal evolution processes. Third, severe interference from real, uncontrolled environments; the collected audio is mixed with environmental noise, household appliance noise, human voices, and other animal sounds, placing stringent demands on the algorithm's anti-interference capabilities.

[0003] Specifically, existing audio-based pet emotion recognition technologies have the following main shortcomings: Many existing methods rely heavily on short audio slices, neglecting long-term temporal contextual information. Most existing methods classify emotions based on fixed-length audio slices. However, the evolution of a pet's emotions is reflected in the temporal changes of its vocalizations, and slice sampling easily fragments the complete emotional evolution process.

[0004] Severe environmental noise interference results in a low percentage of effective pet vocalizations. The actual recorded audio contains a large amount of irrelevant background noise, introducing characteristic noise and significantly reducing recognition accuracy.

[0005] It can only identify a few basic emotions. Existing solutions typically only cover two to four basic emotions (such as happiness and fear), which cannot meet the needs of refined pet behavioral analysis.

[0006] It lacks robustness to changes in audio length. Actual pet recording durations vary greatly, ranging from a few seconds to several minutes. Most existing solutions force truncation and resampling alignment, resulting in the loss of useful information or the introduction of distortion.

[0007] Annotated data is scarce, and models generalize poorly. Annotating audio recordings of pet emotions is costly, the number of professional behavioral annotation samples is limited, traditional full-scale fine-tuning models are prone to overfitting, and self-supervised pre-training is not fully utilized.

[0008] Insufficient temporal modeling capabilities. Simple temporal pooling and RNN / LSTM are inadequate for modeling the complex long-distance dependencies in pet vocalizations.

[0009] Therefore, there is an urgent need in this field for a technical solution that can achieve automated, fine-grained pet voice emotion recognition based on self-supervised learning from variable-length real-world audio. Summary of the Invention

[0010] The purpose of this invention is to provide a pet voice emotion recognition method and system based on self-supervised learning, in order to address the challenges raised in the background art regarding how to suppress environmental interference from real recordings with mixed noise and extract pure acoustic features of pet vocalizations; how to achieve robust temporal modeling for variable-length pet audio sequences of arbitrary duration; how to integrate self-supervised audio pre-training knowledge and combine local and global temporal features to complete refined multi-category pet voice emotion classification; and how to fully exploit the acoustic priors of the self-supervised model under the condition of a limited number of labeled audio samples, thereby alleviating the problem of model overfitting under small sample sizes.

[0011] For the sake of consistency and ease of reading, the main mathematical symbols and technical terms used in this application are defined as follows: B is the batch audio input tensor; B is the batch size. The maximum audio sampling frame within the batch; The actual number of valid audio frames for the b-th sample. ; : Frame-by-frame feature dimensions of the audio backbone network output; E: Dimension of the shared embedding space; F-frame audio feature sequence ; Temporal feature tensors after projection / temporal aggregation / recalibration; Channel-dimensional global statistics encoding operators (such as global average pooling); : Channel weight generation gating network with bottleneck structure; a: Learnable global aggregation token; P: Learnable positional encoding matrix; S: Input sequence containing aggregate token and position code; Q, K, V: Query, key, and value matrices in self-attention; Attention scaling factor (key vector dimension); Attention bias mask matrix corresponding to the valid frame indicator vector; O: Output sequence of the global timing encoder; : The fused audio-level feature representation; K: Number of emotion categories, in this invention K ≥ 10; : The set of trainable parameters for time-series modeling and classification heads; L: Training target loss function (cross-entropy or focusing loss); :Focus parameters and class balance weighting factors for focus loss; To achieve the above objectives, the present invention provides the following technical solution: a method for pet voice emotion recognition based on self-supervised learning: 1. System Overall Architecture This invention proposes an Audio-Temporal Decoupled Two-Stage Architecture: The first stage is an audio decoupling layer for sound source separation and noise suppression, which extracts pet vocalizations from the original audio, effectively decoupling the sound source from environmental noise; the second stage is a multi-scale temporal emotion recognition layer, which performs local-global multi-granular temporal modeling on the denoised pet vocalization audio sequence, outputting a multi-dimensional probability distribution of emotional states. The two stages are connected through a unified spectral feature normalization interface, forming an end-to-end trainable and deployable complete processing pipeline.

[0012] 2. First level: Audio decoupling layer for sound source separation and noise suppression (1) Sound source separation model training: A sound source separation network (preferably Conv-TasNet) was used to construct a dedicated sound source separation model for pet barking. The training dataset consisted of mixed audio samples: synthesized by superimposing various real environmental noises (indoor ambient sound, household appliance noise, human voice, and other animal sounds) onto pure pet barking samples. Data augmentation strategies were adopted during the training process, including volume scaling, random noise addition, and pitch perturbation, to improve the model's noise generalization ability.

[0013] (2) Audio preprocessing and source separation: The original audio is converted into mono with a preset sampling rate (preferably 16kHz). Long audio blocks are then fed into the trained source separation network to output the separated pet barking time-domain audio. ; In the formula, Represents the original input audio. This represents the time-domain signal of the pet's barking after separation and noise reduction.

[0014] (3) Acoustic feature extraction: The separated pet barks are converted into spectral features, preferably Mel-Spectrogram, and the time-domain signal is mapped into a two-dimensional time-frequency representation to obtain an audio frame sequence, thus completing the conversion from the original audio to the features available in the model.

[0015] (4) Optional enhancements: Audio Activity Detection (VAD) Deploys a speech activity detection module to filter silent segments, identify the effective start and end intervals of pet barks, and remove invalid silent frames; Set an energy threshold, and judge frames with energy below the threshold as silent and not send them to the subsequent time series model to reduce invalid calculations.

[0016] (5) Source Separation Loss Function: The source separation network uses the scale-invariant signal-to-noise ratio loss SI-SNR as the training loss: ; The aligned target pet vocal components, This represents the noise residual component.

[0017] (6) Post-processing filtering: Post-processing is performed on the output Mel spectrum sequence to filter out invalid frames with too low energy, output a standardized audio feature sequence, and send it to the downstream temporal emotion recognition layer.

[0018] 3. Second level: Audio emotion recognition model based on self-supervised learning This layer is the core of the invention, and is divided into a multi-level progressive pipeline of self-supervised audio coding, local temporal modeling, global temporal modeling, and emotion classification.

[0019] (1) Self-supervised pre-trained backbone extracts frame-by-frame audio features from the input batch audio Mel-spectrum tensor. The size of batch B is as follows: C represents the maximum number of audio frames in this batch, and C is the number of Mel-frequency channels. A self-supervised learning pre-trained audio backbone network (preferably Wav2Vec2 or HuBERT) is used to extract frame-by-frame acoustic features. During training, the weights of the self-supervised backbone network can be frozen to retain the acoustic prior knowledge learned from a large amount of unlabeled audio, training only the downstream temporal head network to reduce the need for labeled data. Frame-by-frame feature calculation: ; Output frame-by-frame feature sequence: .

[0020] (2) Feature projection and local temporal awareness use learnable linear mapping to map the backbone network output feature dimensions to a unified shared embedding space E: ; Temporal aggregation operations with limited receptive fields are performed in the time dimension to capture short-term acoustic change patterns between adjacent audio frames. These temporal aggregation operations can be implemented using one-dimensional convolution, dilated convolution, or gated linear units (GLUs), providing features containing short-term contextual information for subsequent global self-attention. .

[0021] (3) Channel attention recalibration mechanism is introduced after local temporal aggregation is completed. First, global statistical encoding (global average pooling) is performed in the embedding dimension to compress the temporal features into channel descriptors; the channel weights are mapped through a bottleneck-structured gating network; the original temporal features are multiplied with the channel weights channel by channel to adaptively amplify the emotion discrimination-related channels and suppress irrelevant acoustic channels.

[0022] Represents statistical coding operators, Represents a bottleneck gating network. Represents the Sigmoid activation function, recalibrated and recalculated: .

[0023] (4) Before the serialized representation and positional encoding are fed into the global temporal encoder, a learnable global aggregation token 'a' is inserted at the beginning of the sequence to aggregate the global semantics of all audio sequences; a learnable positional encoding is added to save the temporal order information of the audio frames: ; a is a learnable aggregate token, and P is a learnable location encoding matrix.

[0024] (5) Global temporal coding and dynamic attention masking employ a multi-layer stacked self-attention encoder to perform global temporal modeling of sequence S; the encoder consists of a multi-head self-attention sublayer and a feedforward network FFN stacked together, combined with layer normalization and residual connections; the number of encoder layers is preferably 4-8, with the number of attention heads and embedding dimension adapted accordingly. Single-head attention calculation: ; Q, K, V are the query, key, and value matrices; Scaling factor; This is the attention bias mask matrix.

[0025] Core Innovation 1: Dynamic Attention Masking Mechanism Based on Valid Frame Indicator Vector. For variable-length audio, a binary indicator mask is constructed, and the global aggregation token is always marked as valid. The position of the invalid frame is filled with a negative infinite bias, and after softmax, the attention weight of the invalid frame approaches 0.

[0026] Invalid frames obtained by zero padding do not participate in feature aggregation at all and do not pollute the acoustic temporal representation; Audio files of different durations can be mixed and processed in the same batch without the need for forced truncation or resampling alignment; There is no additional computational overhead during the inference phase; the mask is generated solely based on the actual number of valid frames in the sample.

[0027] (6) Adaptive temporal fusion and emotion classification global temporal encoder output sequence representation: ; Core Innovation 2: Global Token Guidance - Local Enhancement Adaptive Fusion Strategy Instead of using a single token output or simple mean pooling, we simultaneously acquire the global aggregated token output (global context) and feature statistics from all valid audio frames, fusing them to obtain a complete audio-level representation. ; For the fusion function, you can choose to concatenate, sum, or gated weighting; Pooling operations are performed on the set of valid frames, including mean pooling and attention-weighted pooling. Aggregate tokens capture long-range global sentiment cues, while valid frame pooling preserves local acoustic details. The two complement each other to improve representation robustness.

[0028] The audio-level representation is normalized and regularized, then fed into a classification projection layer to map onto the emotion category space, outputting a multidimensional emotion probability distribution prediction. .

[0029] (7) Multi-head attention and in-layer refinement of feedforward network Single-layer encoder input Forward flow: ; ; Multi-head attention projects the input to h parallel subspaces, computes the attention separately, and then concatenates them. ; ; The feedforward network (FFN) consists of two linear layers with GELU activation, and the hidden layer dimension is optimized to be 4 times the embedding dimension. ; The residuals and normalization can be performed using either Pre-Norm or Post-Norm. This invention prefers Post-Norm to ensure training stability.

[0030] (8) Model complexity analysis: actual sequence length T, embedding dimension E, number of attention heads h, number of encoder layers L, and output dimension of the self-supervised backbone. The computational overhead mainly comes from: ① the backbone network forward pass. ②Temporal self-attention Dynamic masking ensures that computation is only consumed within valid frames; ③ Feedforward network The backbone network is frozen, and only the temporal header is fine-tuned, allowing for a smaller number of trainable parameters. It is much smaller than the number of parameters required for full fine-tuning, enabling efficient transfer of self-supervised knowledge.

[0031] 4. Audio decoding, sampling, and robust data processing pipeline Real pet recordings suffer from engineering problems such as inconsistent sampling rates, file corruption, and huge differences in duration. This invention designs an adaptive sampling, multi-backend fallback audio decoding pipeline to ensure the robustness of training and inference data.

[0032] (1) Adaptive audio sampling strategy: Input audio total sampling frames N, target output frame count T. When Uniform sampling along the time axis; Index set: ; when All original sampled frames are retained, and the target length is completed by repeating existing frames to avoid truncating short audio files. This ensures uniform adaptation for both long and short audio samples.

[0033] (2) The priority of the multi-backend layer fallback decoding audio decoding modules is as follows: hardware accelerated audio decoding library → FFmpeg CPU decoding → framework built-in audio reading interface; the audio signal is output as soon as any backend reads successfully.

[0034] ; Multi-level rollback improves tolerance to damaged audio and various encoding formats.

[0035] (3) Before decoding, read the audio metadata to obtain the total number of frames, sampling rate and audio duration. If the audio stream is empty, the number of frames is 0, or the decoding throws an exception, the marked sample is skipped directly to prevent a single bad sample from causing the training batch to crash. At the same time, set the maximum duration threshold to filter out audio segments that are too long and have no labeling value.

[0036] (4) Normalization and Data Augmentation: Amplitude normalization is performed after audio conversion to Mel spectrum. Augmentation is enabled during the training phase. Frame-level spectral perturbation: Apply random gain, spectral mask, and time mask to the Mel spectrum to simulate real-world volume fluctuations; Segment-level overall perturbation: The entire audio stream is uniformly subjected to pitch shifting and time stretching to ensure temporal consistency and preserve the evolutionary characteristics of vocalizations. All enhancements are disabled during the inference phase to ensure reproducible results.

[0037] Compared with the prior art, the beneficial effects of the present invention are: Multi-granular temporal acoustic modeling capability: It adopts a multi-level progressive architecture of local temporal perception-channel attention recalibration-global self-attention, which can capture both short-term call spectrum changes and long-distance emotional evolution, overcoming the long-term gradient degradation problem of traditional RNN / LSTM.

[0038] Native variable-length audio support: Dynamic attention mask enables lossless batch processing of audio of arbitrary length without truncation or resampling, fully preserving the temporal information of the calls. There is no additional computational overhead during the inference stage, balancing training efficiency and engineering deployment performance.

[0039] Sound-temporal two-level decoupled noise suppression: The front-end sound source separation network removes environmental noise, while the back-end temporal model focuses on the acoustic features of the pet's barking itself; noise reduction and temporal emotion modeling are decoupled into two independent modules to avoid end-to-end training gradient conflicts, and the modules can be iterated and upgraded independently.

[0040] Multi-dimensional and refined emotion output: Outputs at least 10-dimensional probability distribution of pet voice emotions, covering aggression, curiosity, depression, disgust, excitement, fear, happiness, irritability, relaxation, sadness, etc., far exceeding the existing 2-4 class coarse-grained recognition solutions, supporting advanced applications such as pet health monitoring and abnormal emotion early warning.

[0041] Self-supervised efficient transfer learning reduces label dependence: Based on audio self-supervised pre-trained backbone network, the backbone is frozen and only the temporal head is fine-tuned. It makes full use of massive unlabeled pet audio to learn acoustic priors, and can converge stably even on small labeled datasets, reducing the cost of professional behavioral labeling.

[0042] Standardized deployment capability: The model supports exporting to ONNX format and can be deployed on server GPUs, edge embedded devices, and mobile devices to meet different latency requirements for offline audio analysis, real-time pet device monitoring, and cloud audio services.

[0043] Robust access to heterogeneous audio data: Multi-backend fallback audio decoding, compatible with multiple audio encoding formats, automatic filtering of damaged audio files, and improved availability of the entire system. This pet voice emotion recognition method and system settings based on self-supervised learning are as follows. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the overall system architecture of the present invention—a two-level hierarchical processing pipeline with spatial-temporal decoupling; Figure 2 This invention defines a chart for animal emotional states; Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] 1. Data Acquisition and Preprocessing (1) Audio data acquisition We collected audio recordings of cat and dog barks from real-life scenarios; the recording scenarios included indoor homes, pet hospitals, and pet boarding facilities; the recordings were mixed with real-world environmental noise; and each audio sample was labeled with emotion tags by animal behaviorists.

[0047] (2) Emotion Category Definition Table 2: Pet Voice Emotional State Definition Table, such as Figure 2 As shown.

[0048] (3) Labeling quality control Multi-annotator annotation aggregation: Summarizes multiple annotation results; Confidence filtering: Set confidence threshold The optimal range is 0.5-0.7, retaining only samples where the annotator's confidence level is higher than the threshold. ; Conflict label handling: Conflicting labels are handled through majority voting, expert review, or removal of controversial samples; Category balancing: Monitor the number of samples in each category and use oversampling and undersampling to alleviate class imbalance. In the example, at least 50 audio samples are retained for each category.

[0049] 2. Training of the sound source separation model Dataset construction: Pure pet barks superimposed with real environmental noise to synthesize mixed audio samples, and divide them into training set and validation set; Initialization uses pre-trained sound source separation network weights; training epochs: 100-250; batch size adjusted according to GPU memory; optimizer: Adam; learning rate: (1e-4); cosine annealing scheduling; loss function: SI-SNR loss. Save the optimal weights of the validation set separation metrics for audio noise reduction during the inference phase.

[0050] 3. Emotion Recognition Model Training (1) Dataset Construction: The original audio is fed into the trained sound source separation network to obtain the denoised pet barking, and the Mel spectrum features are extracted to divide the training, validation and test sets. (2) Hyperparameter Configuration Self-supervised backbone network: Hubert / Wav2Vec2, with backbone weights frozen during training; The temporal header embedding dimension is 256-1024; the number of self-attention encoder layers is 4-8; the number of attention heads ensures that the embedding dimension is divisible by the number of heads. The optimizer is Adam, with an initial learning rate of (1×10^{-4}), weight decay of (1×10^{-4}), and a cosine annealing learning rate. Loss function: Focal Loss, which alleviates class imbalance. ; Category balancing weights, focusing on parameters ; Degenerates to standard cross-entropy. An L2 weight decay regularization term is added to the overall training objective: ; These are trainable temporal header parameters. This is the weight decay coefficient.

[0051] The training run consists of 100-200 rounds, with verification performed at fixed intervals. The main monitoring metric is the macro-average F1 (Macro-F1), which saves the optimal weights.

[0052] (3) Variable-length audio batch processing training For audio samples in the same batch, zero padding is aligned to the maximum number of frames in the batch, and the number of valid frames for each sample is recorded. The padding position is masked by dynamic attention masking, and the padding frames are not included in the loss calculation.

[0053] (4) Evaluation indicator system Overall accuracy: ; Category-level precision, recall, and F1 score: ; Macro average F1: ; Simultaneously, the micro-average F1 score is calculated, a confusion matrix is ​​plotted, and easily confused emotion categories are analyzed to guide dataset iteration.

[0054] 4. Reasoning Deployment Offline inference: Load the weights of the sound source separation model and the emotion recognition model, and sequentially perform sound source separation and noise reduction, Mel spectrum feature extraction, and temporal emotion model inference on the original audio to output multi-dimensional emotion probability; Model Export: The entire model is exported in ONNX format, supporting deployment on mobile devices and C++ inference engines. It can be configured with dynamic sequence length to adapt to audio input of any duration.

[0055] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for pet voice emotion recognition based on self-supervised learning, characterized in that, Includes the following steps: S1. Obtain the original pet audio to be identified, and perform sound source separation and noise suppression on the original pet audio through the audio decoupling layer to obtain the noise-reduced pet barking time domain signal. S2. Convert the pet barking time-domain signal into a time-frequency domain audio feature sequence; S3. Input the audio feature sequence into the temporal emotion recognition layer, use a self-supervised pre-trained backbone network to extract frame-by-frame acoustic features, and obtain recalibrated temporal features through feature projection, local temporal perception, and channel attention recalibration. S4. Insert learnable global aggregate tokens into the recalibrated temporal features and superimpose positional codes to obtain sequence representations. Use a multi-layer self-attention encoder with dynamic attention mask to complete global temporal modeling. The dynamic attention mask is constructed based on the audio valid frame indicator vector, and a negative infinite bias is applied to zero-filled invalid audio frames; S5. An adaptive fusion strategy of global token guidance and local enhancement is adopted to fuse the output features of the global aggregated token with the pooled statistical features of all valid frames to obtain audio-level feature representation. S6. Input the audio-level feature representation into the classification projection layer and output at least a ten-dimensional probability distribution of pet sound emotional states.

2. The pet voice emotion recognition method based on self-supervised learning according to claim 1, characterized in that: In step S1, the audio decoupling layer includes a sound source separation network and an audio activity detection (VAD) module. The sound source separation network is Conv-TasNet, which uses pure pet barking plus environmental noise to construct mixed audio training samples during training, and uses the scale-invariant signal-to-noise ratio loss SI-SNR as the loss function to complete model training. The audio activity detection (VAD) module identifies and filters silent segments in the audio based on an energy threshold.

3. The pet voice emotion recognition method based on self-supervised learning according to claim 1, characterized in that: In step S3, the self-supervised pre-trained backbone network adopts Wav2Vec2 or HuBERT. During the model training phase, the weights of the self-supervised pre-trained backbone network are frozen, and only the downstream temporal modeling and classification head parameters are trained. The local temporal perception uses one-dimensional convolution, dilated convolution, or gated linear units to perform temporal aggregation operations with a limited receptive field, capturing short-term acoustic change patterns between audio frames; The channel attention recalibration is implemented as follows: global statistical encoding is performed on the local temporal aggregated output features to obtain channel descriptors, and channel weights are generated through a gated network with a bottleneck structure. The original temporal features are multiplied with the channel weights one by one, and the emotion discrimination-related feature channels are adaptively enhanced while irrelevant feature channels are suppressed.

4. The pet voice emotion recognition method based on self-supervised learning according to claim 1, characterized in that: The construction method of the dynamic attention mask includes: constructing a binary valid frame indicator vector, marking the position of the global aggregation token as valid, and setting a negative infinite bias at the position of the zero-padding invalid frame; during the self-attention calculation process, the attention score corresponding to the invalid frame is normalized by softmax and the weight approaches zero, and the padding frame does not participate in feature aggregation; achieving lossless batch processing of audio with different actual durations within a batch, with no additional computational overhead during the inference stage.

5. The pet voice emotion recognition method based on self-supervised learning according to claim 1, characterized in that: The adaptive fusion strategy of global token guidance and local enhancement is as follows: extract the global aggregated token output from the output sequence of the global temporal encoder as the global semantic feature, perform pooling operation on all valid frame outputs to obtain local temporal statistical features; use a fusion function of summation, concatenation or gated weighting to fuse the global semantic features and local temporal statistical features to obtain audio-level feature representation; the pooling operation is arithmetic average pooling, attention-weighted average pooling or max pooling.

6. The pet voice emotion recognition method based on self-supervised learning according to claim 1, characterized in that: It also includes an audio preprocessing step: adaptive sampling is performed on the audio to be processed, and when the total number of audio sampling frames N is greater than or equal to the target number of frames T, sampling is performed uniformly along the time axis; When the total number of audio sampling frames N is less than the target number of frames T, all original frames are retained and existing frames are repeated to fill in the target number of frames; Configure a multi-backend layer fallback audio decoding pipeline, and attempt decoding in order of priority: hardware-accelerated audio decoding library, FFmpeg CPU decoding, and framework built-in audio reading interface; read audio metadata before decoding, and automatically skip damaged audio samples with zero frame count, missing audio stream, or decoding abnormality; Data augmentation is performed during the training phase, including frame-level spectral perturbations and whole-segment-level pitch shifts and time stretching; All data augmentation operations are disabled during the inference phase.

7. The pet voice emotion recognition method based on self-supervised learning according to claim 1, characterized in that: The loss function used during the training phase is Focal Loss; ； in Predict probability vectors for the model. This is a real label; is the category balancing weight factor, and \(\gamma\) is the focusing parameter; The overall training objective is to add an L2 regularization term to the temporal head parameters to alleviate model overfitting; macro-average F1 is used as the main evaluation metric for model optimization, and accuracy, precision for each class, recall, and confusion matrix are calculated to complete performance evaluation.

8. The pet voice emotion recognition method based on self-supervised learning according to claim 1, characterized in that: The output pet sound emotion categories should include at least: aggression, curiosity, depression, disgust, excitement, fear, happiness, irritability, relaxation, and sadness.

9. A pet voice emotion recognition system based on self-supervised learning, characterized in that, To implement the method according to any one of claims 1-8, comprising: The audio decoupling module is used to receive the original pet audio, perform sound source separation and noise suppression, output the noise-reduced pet barking time-domain signal, and convert it into a time-frequency domain audio feature sequence. The temporal emotion recognition module includes a self-supervised feature extraction submodule, a local temporal modeling submodule, a channel attention recalibration submodule, a global temporal coding submodule, an adaptive fusion submodule, and a classification projection submodule. The self-supervised feature extraction submodule uses a self-supervised pre-trained backbone network with frozen parameters to extract frame-by-frame acoustic features; The local temporal modeling submodule performs feature projection and finite receptive field temporal aggregation; The channel attention recalibration submodule adaptively adjusts the weights of each feature channel; The global temporal coding submodule inserts a global aggregation token and overlays positional codes, and completes global temporal modeling based on multi-layer multi-head self-attention using dynamic attention masks; The adaptive fusion submodule fuses the global aggregated token output with the effective frame pooling features to obtain audio-level features; The classification projection submodule outputs a multidimensional probability distribution of pet emotions. The deployment export module is used to export the system model to ONNX format, which is compatible with heterogeneous platforms such as servers, edge devices, and mobile devices for inference.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pet voice emotion recognition method based on self-supervised learning as described in any one of claims 1-8.

Images