Unbiased animal behavior recognition method and system based on multi-scale feature fusion

Abstract

The present invention belongs to the field of animal behavior detection technology, and specifically relates to an unbiased animal behavior recognition method and system based on multi-scale feature fusion. The method includes: S1, through a multi-scale feature fusion module based on a hybrid expert system, adaptively fusing data of different sampling frequencies to capture the specific characterization of various unbiased animal behaviors; S2, through a classifier calibration module driven by neural collapse, maximizing the angle between weight vectors of classifiers of different categories, completing the construction of an unbiased animal behavior classification model, and realizing unbiased animal behavior classification under category imbalance; S3, using category-balanced focal loss as a loss function to optimize the unbiased animal behavior classification model. The present invention has the ability to accurately distinguish various animal behaviors by constructing a multi-scale sampling feature fusion network and combining it with a classifier dynamic calibration mechanism.

Description

Technical Field

[0001] The present invention belongs to the technical field of animal behavior detection, and in particular relates to an unbiased animal behavior recognition method and system based on multi-scale feature fusion. Background Art

[0002] Deep learning is driving the rapid development of refined animal activity recognition (AAR) technology based on wearable sensors. This technology enables real-time monitoring of animal behavior and early detection of diseases, thereby improving livestock management and animal welfare. Research has demonstrated that deep learning models perform well in distinguishing a wide range of animal behaviors. For example, researchers have developed a multiplayer perceptron (MLP) based on sensor data to identify cattle behaviors such as grazing, rumination, and resting, achieving over 90% accuracy. Currently, convolutional neural networks (CNNs) are the most widely used model for AAR tasks, achieving over 90% accuracy in identifying a variety of animal behaviors, such as standing and walking in horses, rumination and salt intake in cattle, grazing and activity levels in sheep, and feeding and lactation in pigs. Furthermore, recurrent neural networks (RNNs) are increasingly widely used due to their advantages in processing temporal information such as sensor data. Most studies have shown that by combining RNNs and CNNs to build a hybrid model, the performance of behavior recognition can be effectively improved, and its effect is significantly better than that of a single network structure.

[0003] Although deep learning has made significant progress in AAR tasks, practical applications still face the problem of low classification accuracy for certain behavioral categories. This problem mainly stems from the confusion between the classification of different animal behaviors, and its root causes can be attributed to the following two aspects:

[0004] One is the suboptimal sampling frequency strategy. Existing methods typically use a uniform sampling frequency to optimize overall classification performance. This "one-size-fits-all" strategy ignores the significant differences in the temporal scales of different behavioral patterns and fails to meet the specific sampling frequency requirements of various behaviors. For example, researchers compared the performance of sheep activity recognition based on triaxial accelerometer and gyroscope data at different sampling frequencies (8 Hz, 16 Hz, and 32 Hz) and found that a 32 Hz sampling frequency achieved the best classification results, with an overall accuracy of 95%. Other researchers evaluated the impact of different sampling frequencies (100 Hz, 50 Hz, 25 Hz, and 12.5 Hz) on horse behavior classification using triaxial acceleration and angular velocity data, and the results showed that a sampling frequency of 25 Hz achieved the best overall performance.

[0005] The second issue is class imbalance. Classifiers trained on class-imbalanced datasets often favor the majority class, resulting in lower accuracy for minority behavior classes. For example, researchers used an MLP network to identify five different daily behaviors of cattle, achieving an overall accuracy of 97.75%, but the accuracy for "walking" and "drinking" behaviors was only 67.69% and 48.08%, respectively. This was primarily due to the fact that the sample size for these two behaviors only accounted for 7.61% and 4.97% of the total. Similarly, researchers previously observed a similar phenomenon in horse behavior recognition: the overall accuracy was 93.37%, but the accuracy for the minority class "natural walking" was only 24.75%. Existing research has addressed this class imbalance issue by either using resampling techniques to balance the class distribution or by adjusting the loss function to increase the penalty for the minority class. However, these approaches focus primarily on optimizing external factors to improve classifier training performance, while ignoring the internal structural features that are essential for a well-trained classifier under class-balanced conditions.

[0006] To solve the above problems, the present invention develops an unbiased animal behavior recognition method based on multi-scale feature fusion, which enhances the model's recognition ability for each specific behavior by simultaneously customizing features and calibrating classifiers. Summary of the Invention

[0007] The present invention aims to overcome the problems of insufficient feature extraction (suboptimal sampling frequency) and uneven classification performance in the existing methods in the prior art in the recognition of specific behaviors. It provides an unbiased animal behavior recognition method and system based on multi-scale feature fusion, which can accurately distinguish various behaviors by constructing a multi-scale sampling feature fusion network and combining it with a dynamic calibration mechanism of the classifier.

[0008] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions:

[0009] The unbiased animal behavior recognition method based on multi-scale feature fusion includes the following steps:

[0010] S1, through a multi-scale feature fusion module based on a hybrid expert system, adaptively fuses data of different sampling frequencies to capture the specific representation of various behaviors;

[0011] S2, through the neural collapse-driven classifier calibration module, maximizes the angle between the weight vectors of classifiers of different categories, completing the construction of an unbiased animal behavior classification model, which is used to achieve unbiased animal behavior classification under class imbalance;

[0012] S3, class-balanced focal loss is used as the loss function to optimize the unbiased animal behavior classification model.

[0013] Preferably, step S1 includes the following steps:

[0014] S11, at the sampling frequency 、 、 Collect the corresponding data 、 、 ;

[0015] Among them, the values of the three sampling frequencies are set according to relationship;

[0016] S12, will 、 、 Input into respective feature extractors to capture features at different scales 、 、 ;

[0017] in and Represent the number of feature channels and the dimension of the coordinate axis respectively, ， , refers to the time dimension corresponding to the features obtained on each branch, and ;

[0018] S13, perform global average pooling operation on the features extracted from the three branches to generate three feature vectors, namely , and ;

[0019] In S14, the three generated feature vectors are input into a router-based soft weighted fusion layer for adaptive fusion.

[0020] Preferably, step S14 includes the following steps:

[0021] S141, generating a combined eigenvector by calculating the average of the three eigenvectors;

[0022] S142, the generated combined feature vector is processed by a two-layer multi-layer perceptron MLP to obtain a logical value ;

[0023] (1);

[0024] S143, the obtained logical value is subjected to SoftMax operation to obtain the contribution rate corresponding to the three different branch features, which is recorded as :

[0025] (2);

[0026] Among them, the parameters Indicates the degree of participation of features under different sampling frequencies;

[0027] S144, respectively, uses three projection layers, namely , used to align the feature vectors into a unified embedding space and obtain their respective projection features ;

[0028] S145, set the final fusion feature Expressed as the weighted sum of each projection feature, that is:

[0029] (3);

[0030] Among them, the weight The corresponding contribution rate .

[0031] Preferably, in step S2, the neural collapse-driven classifier calibration module includes an equiangular tight frame classifier with fixed parameters and a fully connected layer classifier with learnable parameters; the equiangular tight frame classifier with fixed parameters specifically includes the following process:

[0032] S211, randomly synthesize a classifier vector at the beginning of training, and ensure that the classifier vector conforms to the simplex equiangular tight frame property, as follows:

[0033] Set up a set of A collection of vectors ,in represents the feature dimension, Indicates the number of categories and satisfies the conditions ; The vector set is considered to be a simplex equiangular tight frame if the following conditions are met:

[0034] (4);

[0035] in, Allows rotation and satisfies ; is the identity matrix, Indicates the length is All vectors in a simplex equiangular tight frame have equal Norm and the same pairwise angle, that is: (5);

[0036] (6);

[0037] in, The paired angle representation Maximum equiangular separation between vectors;

[0038] S212, taking into account the normalized class feature prototype Represents the classifier weight vector for each category, normalizing the vector The fixed weight matrix is constructed as an equiangular tight frame classifier and remains unchanged throughout the training process;

[0039] S213, through the projection layer The fusion features obtained by formula (3) Project and get the features ;set up Perform dimensionality adjustment for achieving neural collapse in the classifier calibration module.

[0040] S214, the feature Normalized to :

[0041] (7);

[0042] S215, will and the classifier weights predefined in step S212 Multiply to get the logical value :

[0043] (8);

[0044] in, Represents the product used to scale the features The learnable temperature coefficient, and All have limited scope. Representation category Corresponding to the obtained logical value, where .

[0045] Preferably, the fully connected layer classifier whose parameters can be learned specifically includes the following process:

[0046] S221, randomly initialize the fully connected layer classifier parameters and allow these parameters to train normally during the training process;

[0047] S222, the weight of each category classifier is expressed as , used to fusion features Processed to generate logical values ,Right now: (9);

[0048] Preferably, step S2 further includes the following steps:

[0049] S23, the logical value output by the fixed parameter equiangular tight frame classifier Logical value output by the fully connected layer classifier with learnable parameters Perform linear combination to obtain the combined logical value :

[0050] (10);

[0051] in, Used to regulate the degree of fixedness of classifier parameters in the classification stage.

[0052] Preferably, step S3 includes the following steps:

[0053] S31, setting Represents the logical value of the final output, then the category-balanced focal loss Expressed as:

[0054] (11);

[0055] (12);

[0056] in, Representation category The number of samples; is the sample size adjustment factor, which is used to control the effective sample size. increased growth rate; is the weight smoothing factor, which is used to adjust the weight decay rate of easy-to-classify samples.

[0057] The present invention also provides an unbiased animal behavior recognition system based on multi-scale feature fusion, comprising:

[0058] A feature extraction module is used to adaptively fuse data of different sampling frequencies through a multi-scale feature fusion module based on a hybrid expert system to capture the specific representation of various behaviors;

[0059] The behavioral classification module is used to maximize the angle between the weight vectors of classifiers of different categories through the neural collapse-driven classifier calibration module, completing the construction of an unbiased animal behavior classification model to achieve unbiased animal behavior classification under class imbalance;

[0060] A model optimization module is used to optimize the unbiased animal behavior classification model using class-balanced focal loss as a loss function.

[0061] Compared with the existing technology, the present invention has the following advantages: (1) The present invention proposes an unbiased animal behavior recognition method based on multi-scale feature fusion, which for the first time shifts the research focus to the recognition accuracy of a single specific behavior. By simultaneously optimizing the dual mechanism of feature customization and classifier calibration, it breaks through the limitation of traditional methods that only focus on the overall behavior classification accuracy;

[0062] (2) In response to the differentiated demands of different behaviors for sampling frequency, the present invention designs a multi-scale feature fusion module based on a hybrid expert system. This module realizes the adaptive selection and fusion of data with different sampling frequencies by constructing a multi-branch feature extraction network. Unlike existing methods, the present invention can determine the optimal sampling frequency combination without relying on prior behavior labels. By optimizing the criterion of maximizing the feature expression ability, it effectively captures the specific representation of various behaviors and solves the technical bottleneck that the traditional single sampling frequency is difficult to adapt to the extraction of diversified behavior features.

[0063] (3) To address the classifier bias problem caused by category imbalance, the present invention proposes a neural collapse-driven classifier calibration module; this module introduces an equiangular tight frame constraint during the classifier training process to maximize the angle between the weight vectors of classifiers of different categories, significantly improving the inter-class separation; this design draws on the ideal internal structural characteristics of the classifier under equilibrium conditions, alleviates the bias of the classifier under category imbalance, and thus improves the recognition accuracy of minority behavior categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A flow chart of the unbiased animal behavior recognition method based on multi-scale feature fusion in the present invention;

[0065] Figure 2 A schematic diagram of the architecture of the router-based soft weighted fusion layer in the present invention;

[0066] Figure 3 Schematic diagram of activity category distribution of a sheep dataset (a) and a cattle dataset (b) provided in an embodiment of the present invention;

[0067] Figure 4 A comparison of confusion matrices corresponding to three different variants: an unbiased animal behavior recognition method based on multi-scale feature fusion (c), a variant without a hybrid expert system-based multi-scale feature fusion module (b), and a variant without both a hybrid expert system-based multi-scale feature fusion module and a neural collapse-driven classifier calibration module (a);

[0068] Figure 5 A t-SNE visualization diagram of single sampling frequency features and multi-sampling frequency fusion features;

[0069] Figure 6 A schematic diagram comparing the impact of the neural collapse-driven classifier calibration module on the cosine similarity distribution between weight vectors of each category. DETAILED DESCRIPTION

[0070] To more clearly illustrate the embodiments of the present invention, specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings and other embodiments can be obtained based on these drawings without inventive efforts.

[0071] like Figure 1 and Figure 2 As shown, the present invention provides an unbiased animal behavior recognition method based on multi-scale feature fusion, which specifically includes the following steps:

[0072] Step 1, feature extraction stage: adaptively fuse data of different sampling frequencies through a multi-scale feature fusion module based on a hybrid expert system to capture the specific representation of various behaviors.

[0073] Step 2, behavioral classification stage: Through the neural collapse-driven classifier calibration module, the angle between the weight vectors of classifiers of different categories is maximized to complete the construction of an unbiased animal behavior classification model to achieve unbiased animal behavior classification under category imbalance.

[0074] Step 3, model optimization stage: class-balanced focal loss is used as the loss function to optimize the unbiased animal behavior classification model.

[0075] For step 1, the specific steps include:

[0076] 1-1, at sampling frequency 、 、 Collect the corresponding data 、 、 ;

[0077] Among them, the values of the three sampling frequencies are set according to relationship;

[0078] 1-2, will 、 、 Input into respective feature extractors to capture features at different scales 、 、 ;

[0079] in and Represent the number of feature channels and the dimension of the coordinate axis respectively, ， , refers to the time dimension corresponding to the features obtained on each branch, and ;

[0080] 1-3, perform global average pooling operations on the features extracted from the three branches to generate three feature vectors, namely , and ;

[0081] 1-4, the three generated feature vectors are input into a router-based soft weighted fusion layer for adaptive fusion.

[0082] Another example Figure 2 As shown, steps 1-4 include the following steps:

[0083] 1-4-1, generate a combined eigenvector by calculating the average of the three eigenvectors;

[0084] 1-4-2, the generated combined feature vector is processed through a two-layer multi-layer perceptron MLP to obtain the logical value ;

[0085] (1);

[0086] 1-4-3, perform SoftMax operation on the obtained logical value to obtain the contribution rate corresponding to the three different branch features, recorded as :

[0087] (2);

[0088] Among them, the parameters Indicates the degree of participation of features under different sampling frequencies;

[0089] 1-4-4, using three projection layers, namely , used to align the feature vectors into a unified embedding space and obtain their respective projection features ;

[0090] 1-4-5, set the final fusion feature Expressed as the weighted sum of each projection feature, that is:

[0091] (3);

[0092] Among them, the weight The corresponding contribution rate .

[0093] For step 2, the neural collapse-driven classifier calibration module proposed in this embodiment includes an equiangular tight frame classifier with fixed parameters and a fully connected layer classifier with learnable parameters.

[0094] Among them, the parameter-fixed equiangular tight frame classifier specifically includes the following process:

[0095] 2-1-1, randomly synthesize the classifier vector at the beginning of training, and ensure that the classifier vector conforms to the simplex equiangular tight frame property, as follows:

[0096] Set up a set of A collection of vectors ,in represents the feature dimension, Indicates the number of categories and satisfies the conditions ; The vector set is considered to be a simplex equiangular tight frame if the following conditions are met:

[0097] (4);

[0098] in, Allows rotation and satisfies ; is the identity matrix, Indicates the length is All vectors in a simplex equiangular tight frame have equal Norm and the same pairwise angle, that is:

[0099] (5);

[0100] (6);

[0101] in, The paired angle representation Maximum equiangular separation between vectors;

[0102] 2-1-2, considering the normalized class feature prototype Represents the classifier weight vector for each category, normalizing the vector The fixed weight matrix is constructed as an equiangular tight frame classifier and remains unchanged throughout the training process;

[0103] 2-1-3, through the projection layer The fusion features obtained by formula (3) Project and get the features ;set up Perform dimensionality adjustment for achieving neural collapse in the classifier calibration module.

[0104] 2-1-4, the characteristics Normalized to :

[0105] (7);

[0106] 2-1-5, The classifier weights predefined in step 2-1-2 Multiply to get the logical value :

[0107] (8);

[0108] in, Represents the product used to scale the features The learnable temperature coefficient, and All have of limited scope, Representation category Corresponding to the obtained logical value, where ;

[0109] In addition, the fully connected layer classifier with learnable parameters specifically includes the following process:

[0110] 2-2-1, randomly initialize the fully connected layer classifier parameters and allow these parameters to train normally during the training process;

[0111] 2-2-2, the weights of each category classifier are expressed as , used to fusion features Processed to generate logical values ,Right now:

[0112] (9);

[0113] 2-3, finally, the logical value output by the fixed parameter equiangular tight frame classifier Logical value output by the fully connected layer classifier with learnable parameters Perform linear combination to obtain the combined logical value :

[0114] (10);

[0115] in, Used to regulate the degree of fixedness of classifier parameters in the classification stage.

[0116] Step 3 specifically includes the following steps:

[0117] set up Represents the logical value of the final output, then the category-balanced focal loss Expressed as:

[0118] (11);

[0119] (12);

[0120] in, Representation category The number of samples; is the sample size adjustment factor, which is used to control the effective sample size. increased growth rate; is the weight smoothing factor, which is used to adjust the weight decay rate of easy-to-classify samples. Based on experimental verification, the present invention will and Set to 0.9999 and 0.5.

[0121] The remarkable technical effects of the present invention are further illustrated below in conjunction with experiments.

[0122] 1. Experimental Data and Settings

[0123] 1-1. Experimental Data

[0124] The present invention uses two open-source animal behavior recognition datasets to verify the proposed method, which are collected from the behavior of sheep and cattle respectively.

[0125] Sheep Dataset: This dataset collects data from five sheep on two farms, including three small domestic pygmy sheep from one farm and two larger wild sheep from another farm. Each sheep wears a collar equipped with a triaxial accelerometer and a triaxial gyroscope in six different orientations, sampling at a frequency of 100 Hz. This paper screened a total of 42,943 data samples, each corresponding to a 2-second behavioral segment, represented as a tensor of dimensions 1 × 36 × 200. Figure 3 (a) shows the class distribution of these samples, covering five major activities, including standing (43.15%), galloping (0.87%), eating (35.35%), trotting (0.44%), and walking (20.19%). Obviously, this dataset has a serious class imbalance problem, with an imbalance ratio of 98.05.

[0126] Cattle Dataset: This dataset is collected from six freely moving dairy cows. Each cow has a triaxial accelerometer mounted on its collar, sampling at 25 Hz. We segment the raw acceleration time series data using a 2-second random sliding window, generating 10,429 data samples, each represented as a tensor of 1 × 3 × 50 dimensions. Figure 3 (b) shows the class distribution of samples, covering five main activities, including eating (6.10%), moving (16.29%), resting (54.25%), rumination (19.32%), and salt supplementation (4.04%). Obviously, this dataset also has a class imbalance problem, with an imbalance ratio of 13.44.

[0127] Experimental Setup

[0128] The present invention uses precision, recall, F1 score, and accuracy as metrics to evaluate the overall performance of the classification network. To verify the generalization capability of the present invention, a leave-one-out cross-validation method was used on a sheep dataset, where "one" represents the data of a single animal. Because not all individuals in the cattle dataset cover all behaviors, another commonly used validation method, stratified five-fold cross-validation, was employed. In each round, samples from the 3-fold, 1-fold, and 1-fold samples were used as the training set, validation set, and test set, respectively. This was repeated for five rounds, ensuring that the validation and test sets were inconsistent in each round.

[0129] In order to prevent the model from overfitting, the present invention adds an L2 regularization term to the loss function, where the weight decay coefficients of the sheep dataset and the cattle dataset are set to 1×10 -4 and 6×10 -2 The Adam optimizer is used in the optimization process, and the initial learning rate is set to 1×10-4 , set to 5×10 on the cattle dataset -4 , and adopts a step-by-step learning rate decay strategy, decaying to 0.1 times the original value every 20 training rounds. In the present invention, the model training is carried out for a total of 100 rounds, and the batch size is set to 256. During the training process, the performance of the validation set is continuously monitored and the model parameters with the highest accuracy are saved. Finally, an independent test set is used to evaluate the performance of the saved best model. Based on the conclusions of previous studies, the sheep dataset uses 12.5Hz as the sampling frequency of the baseline model, which has been proven to achieve optimal recognition performance. For the cattle dataset, through preliminary experimental comparative analysis, it was determined that 25 Hz was the optimal sampling frequency, so it was used as the sampling frequency of the baseline model.

[0130] To validate the effectiveness of the proposed method, comprehensive comparative experiments were designed. Based on preliminary experimental results, the sheep dataset used a sampling frequency combination of 50 Hz, 25 Hz, and 12.5 Hz, while the cattle dataset used a sampling frequency combination of 25 Hz, 12.5 Hz, and 5 Hz. This is the first time a multi-sampling frequency fusion strategy has been introduced for animal behavior recognition. The comparative experiments were conducted at two levels: first, a comparison with a baseline method that uses only a single optimal sampling frequency; and second, a comparison with several advanced class imbalance treatment methods, including resampling methods such as K-means synthetic minority oversampling (KMS) and random undersampling (RUS), as well as reweighting methods such as cost-sensitive cross-entropy loss (CSCE), class-balanced focal loss (CBFL), and adaptive class suppression loss (ACSL). All testing was performed using the PyTorch platform on an NVIDIA GeForce RTX 3090 graphics processing unit.

[0131] 2. Performance comparison between this method and existing methods

[0132] Table 1 and Table 2 show the performance comparison results of the proposed method and the existing methods on the sheep and cattle datasets respectively. and Set as: Sheep Dataset ( =0.4, = 0.3), cattle dataset ( =0.8, =0.1). Experimental results show that the proposed method achieves optimal performance on both datasets: 93.17% accuracy, 87.17% F1 score, and 91.71% recall on the sheep dataset; and 91.42% accuracy, 90.62% F1 score, 88.06% precision, and 93.64% recall on the cattle dataset. In particular, compared with the baseline method, the proposed method achieves significant improvement in recall, i.e., 16.98% and 15.78% on the sheep and cattle datasets, respectively. These improvements are in line with the original intention of the present invention to achieve high recall and maximize the accuracy of classifying various behaviors. In addition, the performance improvement of the KMS method is limited, which may be due to the fact that its oversampling strategy leads to over-representation of minority class samples, thereby causing the model overfitting problem. In contrast, the RUS method loses key information due to downsampling on a dataset with limited sample size, resulting in a significant performance degradation. It is worth noting that CB_FL outperforms other methods in improving recall rate. This result verifies the rationality of choosing category-balanced focal loss as the optimization objective in this paper.

[0133] Table 1 Performance comparison of the proposed method and existing methods on the sheep dataset

[0134]

[0135] Table 2 Performance comparison of the proposed method and existing methods on the cattle dataset

[0136]

[0137] 3. Ablation Experiment

[0138] 3.1. Evaluation of the Hybrid Expert System-Based Multi-Scale Feature Fusion Module and the Neural Collapse-Driven Classifier Calibration Module:

[0139] In order to verify the contribution of the newly proposed hybrid expert system-based multi-scale feature fusion module and neural collapse-driven classifier calibration module, the present invention conducted experiments on the sheep dataset to test the animal behavior recognition method with and without these two modules. The experimental results are shown in Table 3 and are summarized in Table 3. Figure 4 The recall confusion matrix is presented in Table 3. Table 3 shows that after adding the classifier calibration module, the F1 score, precision, and recall rates are slightly improved by 0.51%, 0.29%, and 0.99%, respectively. Correspondingly, the classification accuracy of most behaviors has improved to varying degrees (e.g. Figure 4 (b) and Figure 4(a) in the figure), including the minority classes "gallop" and "trot" which increased by 3.48% and 1.06% respectively. This shows that the classifier calibration module proposed in this invention helps to improve the classification performance of minority classes. After further adding the multi-scale feature fusion module based on the hybrid expert system, the overall performance is significantly improved, that is, the accuracy, F1 score, precision and recall rate increased by 1.81%, 4.78%, 5.33% and 3.37% respectively. Figure 4 As shown in (c), all behaviors achieve ideal recall rates, with most exceeding 90% accuracy. This is attributed to the feature fusion module's ability to capture the optimal motion pattern for each behavior by adaptively fusing features from multiple sampling frequencies, thereby ensuring accurate classification of each type of behavior.

[0140] Table 3 Comparison of behavior recognition performance with and without the hybrid expert system-based multi-scale feature fusion module and the neural collapse-driven classifier calibration module

[0141]

[0142] 3.2. Analysis of multi-scale feature fusion module based on hybrid expert system

[0143] Analysis of the advantages of the soft-weighted fusion mechanism: The multi-scale feature fusion module proposed in this invention innovatively adopts a hybrid expert system architecture. Its core advantage lies in the integrated soft routing mechanism that can dynamically and adaptively assign contributions to features extracted at different sampling frequencies and use these contributions as weights for soft feature fusion. To evaluate the effectiveness of this method, the present invention conducted comparative experiments with traditional fusion methods (including addition, averaging, multiplication, and splicing), and the results are shown in Table 4. Experimental data show that the soft-weighted fusion method proposed in this invention outperforms other methods in all evaluation indicators. This advantage mainly stems from the following two aspects: first, the dynamic routing mechanism of the hybrid expert system can automatically identify the optimal sampling frequency for specific behavior patterns; second, the soft weighting strategy can adaptively enhance the feature representation at the optimal sampling frequency, thereby accurately capturing customized feature patterns for different behavior categories.

[0144] Table 4 Performance comparison of different feature fusion methods

[0145]

[0146] Analysis of the advantages of multi-scale feature fusion: Since a single sampling frequency is difficult to meet the feature extraction requirements of each type of behavior category, the present invention proposes a multi-scale feature fusion module based on a hybrid expert system to solve this problem by integrating data of multiple sampling frequencies. To verify its effectiveness, the present invention conducted a comparative experiment with the method that only uses a single sampling frequency (50Hz, 25Hz, 12.5Hz). As shown in Table 5, multi-scale feature fusion showed significant improvement in all evaluation indicators, verifying its robustness and generalization ability. In addition, the present invention also visualized the feature distribution before and after fusion through t-distributed Stochastic Neighbor Embedding (t-SNE). As Figure 5 As shown in the figure, compared to the independent features extracted by each expert network before fusion, the fused features show higher clustering density on similar behavior samples, and the separability between different categories is significantly enhanced. This result shows that multi-scale feature fusion can not only adaptively integrate discriminative information at different frequencies, but also effectively enhance the intra-class consistency and inter-class differentiation of features, thereby improving the accuracy and robustness of behavior recognition.

[0147] Table 5 Performance comparison of the model under multi-sampling frequency fusion and single sampling frequency

[0148]

[0149] 3.3. Analysis of the neural collapse-driven classifier calibration module

[0150] Hyperparameters Impact analysis: The hyperparameters in formula (10) It is used to control the constraint strength of the classifier weight during the training process. Its value directly affects the optimization dynamics and final performance of the model. In order to explore its influence law, the present invention fixes the parameter is 0.4, and the system evaluates The model performance is shown in Table 6. When , the method reaches its peak performance, with accuracy, F1 score, precision and recall of 93.17%, 87.17%, 83.80% and 91.71% respectively. When it exceeds 0.3, the model performance gradually decreases; especially when , the classifier weights are completely fixed in the initial equiangular tight frame state, causing the model to lose its optimization ability, verifying the necessity of dynamic adjustment of classifier parameters.

[0151] Table 6 Data table of the impact of different values on model performance

[0152]

[0153] Visualization of cosine similarity of classifier vectors: The cosine similarity between classifier weight vectors directly determines the geometric characteristics of the decision boundary and is a key factor affecting the performance of class discrimination. To verify the effectiveness of the neural collapse-driven classifier calibration module, this paper systematically compared and analyzed the changes in the cosine similarity distribution of classifier weight vectors before and after the module was introduced. Figure 6 As shown in the figure. Experimental results show that without the classifier calibration module, the cosine similarity distribution range is relatively wide (-0.5 to +0.5), with a significant proportion of sample similarities exceeding 0.4, indicating a small angle between classifier vectors. This leads to blurred decision boundaries and increased risk of misclassification. In contrast, after introducing the neural collapse-driven classifier calibration module, the cosine similarity distribution range is significantly narrowed to -0.2 to +0.2, demonstrating that the module can effectively force the classifier vectors to form a nearly equiangular distribution in the feature space. This optimization of geometric properties significantly improves the classifier's discriminative performance under conditions of class imbalance and effectively alleviates the model's bias towards the majority class.

[0154] 3.4. Model robustness evaluation under class imbalance conditions

[0155] The proposed unbiased animal behavior recognition method based on multi-scale feature fusion effectively maximizes the angle between the weight vectors of different classifiers by introducing an equiangular tight frame classifier as a regularization constraint. To systematically evaluate the robustness of the method under varying degrees of class imbalance, we constructed a progressively imbalanced experimental scenario by downsampling the minority class samples of a sheep dataset: reducing the number of minority class samples to 1 / 2 and 1 / 5 of the original number, respectively, gradually improving the class imbalance ratio from a baseline of 98.05 to 196.1 and 490.25. As shown in Table 7, the proposed method maintains significant advantages in key metrics such as accuracy, F1 score, and precision under varying degrees of imbalance. Notably, the performance advantage of the proposed method over the baseline becomes even more pronounced as the imbalance increases (from 196.1 to 490.25), fully demonstrating the strong robustness of the proposed equiangular tight frame regularization mechanism under extreme imbalance scenarios.

[0156] Table 7 Performance comparison data of the proposed method and the baseline method at different imbalance rates

[0157]

[0158] The present invention develops an unbiased animal behavior recognition method based on multi-scale feature fusion, which enhances the recognition ability of the unbiased animal behavior classification model proposed by this method for each specific behavior by simultaneously customizing features and calibrating classifiers. Specifically, considering that different behaviors require different sampling frequencies to achieve the best classification effect of their respective categories, the present invention designs a multi-scale feature fusion module based on a hybrid expert system. This module can adaptively fuse data of multiple different sampling frequencies, thereby extracting personalized features of specific behavior categories. At the same time, in order to alleviate the problem of classifier bias towards the majority class due to category imbalance, the present invention proposes a neural collapse-driven classifier calibration module. This module introduces a fixed equiangular tight frame classifier in the classification stage, which effectively alleviates the bias of the classifier by maximizing the angle between the weight vectors of different classifiers, thereby improving the classification performance of minority categories.

[0159] The above description is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, based on the ideas provided by the present invention, there may be changes in the specific implementation methods, and these changes should also be considered as the scope of protection of the present invention.

Claims

1. An unbiased animal behavior recognition method based on multi-scale feature fusion, characterized by: The steps include: S1, through a multi-scale feature fusion module based on a hybrid expert system, adaptively fuses data of different sampling frequencies to capture the specific representation of various behaviors; S2, through the neural collapse-driven classifier calibration module, maximizes the angle between the weight vectors of classifiers of different categories, completing the construction of an unbiased animal behavior classification model, which is used to achieve unbiased animal behavior classification under class imbalance; S3, optimizing the unbiased animal behavior classification model using class-balanced focal loss as a loss function; In step S2, the neural collapse-driven classifier calibration module includes an equiangular tight frame classifier with fixed parameters and a fully connected layer classifier with learnable parameters; the equiangular tight frame classifier with fixed parameters specifically includes the following process: S211, randomly synthesize a classifier vector at the beginning of training, and ensure that the classifier vector conforms to the simplex equiangular tight frame property, as follows: Suppose there is a set of M vectors V = [v1, v2, ..., v M ]∈R d×M , where d represents the feature dimension, M represents the number of categories, and satisfies the condition d ≥ M-1; if the following conditions are met, the vector set is regarded as a simplex equiangular tight frame: Where U∈R d×M Rotation is allowed and satisfies U T U=I M ;I M is the identity matrix, 1 M represents a vector of length M and all values are 1; all vectors in the simplex equiangular tight frame have equal l2 norm and the same pairwise angle; S212, normalize the vector The fixed weight matrix is constructed as an equiangular tight frame classifier and remains unchanged throughout the training process; S213, the fusion feature e obtained by the projection layer g of formula (3) f ∈R C Project and get the features Set d = M for dimensionality adjustment, which is used to achieve neural collapse in the classifier calibration module later; S214, the feature Normalized to S215, will and the classifier weights predefined in step S212 Multiply to get the logical value Among them, μ represents the function used to scale the feature product The learnable temperature coefficient, and All have a limited range of [-1,1]; Represents the logical value obtained corresponding to category m, where 1≤m≤M.

2. The unbiased animal behavior recognition method based on multi-scale feature fusion according to claim 1 is characterized in that: Step S1 includes the following steps: S11, collects corresponding data X1, X2, X3 at sampling frequencies S1, S2, S3 respectively; The values of the three sampling frequencies are set to follow the relationship of S1>S2>S3; S12, input X1, X2, and X3 into their respective feature extractors to capture features of different scales Where C and H represent the number of feature channels and the dimension of the coordinate axis respectively, W1, W2, and W3 refer to the time dimension corresponding to the features obtained on each branch, and W1>W2>W3; S13, perform global average pooling operation on the features extracted from the three branches to generate three feature vectors, namely e1∈R C , e2∈R C and e3∈R C ; In S14, the three generated feature vectors are input into a router-based soft weighted fusion layer for adaptive fusion.

3. The unbiased animal behavior recognition method based on multi-scale feature fusion according to claim 2 is characterized in that: Step S14 includes the following steps: S141, generating a combined eigenvector by calculating the average of the three eigenvectors; S142, the generated combined feature vector is processed by a two-layer multi-layer perceptron MLP to obtain the logical value o∈R 3 ; o=MLP(Avg(e1,e2,e3)),o∈R 3 (1) S143, the obtained logical value is subjected to a SoftMax operation to obtain the contribution rates corresponding to the three different branch features, which are recorded as {r1, r2, r3}: Among them, the parameter τ represents the degree of participation of features under different sampling frequencies; S144, three projection layers, namely P1, P2, and P3, are used to align the feature vectors into a unified embedding space, and obtain their respective projected features P1(e1), P2(e2), and P3(e3); S145, set the final fusion feature e f ∈R C Expressed as the weighted sum of each projection feature, that is: And f =∑ i∈{1,2,3} r i *P i (And i ) (3) Among them, the weight r i is the corresponding contribution rate {r1, r2, r3}.

4. The unbiased animal behavior recognition method based on multi-scale feature fusion according to claim 3 is characterized in that: The fully connected layer classifier with learnable parameters specifically includes the following process: S221, randomly initializing the fully connected layer classifier parameters, and allowing the parameters to be trained normally during the training process; S222, the weight of each category classifier is expressed as W = [w1, w2, ..., w M ], used to fusion feature e f ∈R C Processed to generate logical values Right now:

5. The unbiased animal behavior recognition method based on multi-scale feature fusion according to claim 4 is characterized in that: Step S2 also includes the following steps: S23, the logical value output by the fixed parameter equiangular tight frame classifier Logical value output by the fully connected layer classifier with learnable parameters Perform linear combination to obtain the combined logical value Z: Among them, k is used to regulate the degree of fixity of parameters in the classifier during the classification stage.

6. The unbiased animal behavior recognition method based on multi-scale feature fusion according to claim 5 is characterized in that: Step S3 includes the following steps: S31, set Z = {z1, z2, ..., z M } represents the logical value of the final output, then the category-balanced focal loss Expressed as: Among them, n y represents the number of samples of category y; β∈[0,1) is the sample size adjustment factor, which is used to control the effective sample size with n y The growth rate of increase; γ≥0 is the weight smoothing factor, which is used to adjust the weight decay rate of easy-to-classify samples.

7. An unbiased animal behavior recognition system based on multi-scale feature fusion, used to implement the unbiased animal behavior recognition method based on multi-scale feature fusion according to any one of claims 1 to 6, characterized in that: The unbiased animal behavior recognition system based on multi-scale feature fusion includes: A feature extraction module is used to adaptively fuse data of different sampling frequencies through a multi-scale feature fusion module based on a hybrid expert system to capture the specific representation of various behaviors; The behavioral classification module is used to maximize the angle between the weight vectors of classifiers of different categories through the neural collapse-driven classifier calibration module, completing the construction of an unbiased animal behavior classification model to achieve unbiased animal behavior classification under class imbalance; A model optimization module is used to optimize the unbiased animal behavior classification model using class-balanced focal loss as a loss function.

Images