Animal behavior recognition enhancement method and system based on self-supervised pre-training

Abstract

The invention belongs to the technical field of animal behavior detection, and particularly relates to an animal behavior recognition enhancement method and system based on self-supervised pre-training. The method comprises the following steps: S1, constructing a reference model based on a PatchTST model, partitioning a time sequence, and introducing channel independence processing and Transform architecture to complete local feature extraction and long-range dependence modeling of time sequence data; s2, adopting a self-supervised pre-training framework based on time-frequency consistency TF-C, carrying out time-frequency consistency contrast learning on the large-scale cross-species unlabeled data set, and enabling the reference model to capture time-frequency domain invariant features through time domain and frequency domain enhancement and contrast loss and consistency loss optimization; and S3, migrating the pre-trained encoder weight to a PatchTST model, performing initialization and fine tuning on a specific species annotation data set, and combining channel attention and feature recombination convolution optimization to complete cross-species knowledge migration and target task adaptation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of animal behavior detection, and particularly relates to an animal behavior recognition enhancement method and system based on self-supervised pre-training. BACKGROUND

[0002] Animal behavior, as a manifestation of the interaction between animals and the environment, is an important basis for evaluating the internal psychological state and external physiological health state. For example, behaviors such as eating, sleeping, and activity can reflect the health status of animals. When animals show signs of loss of appetite, abnormal sleep, and reduced activity, these may be signals of health problems. In real life, people can observe these behavioral changes to discover possible health problems in animals in a timely manner, and thus take appropriate treatment measures. Traditional animal behavior monitoring largely relies on manual observation, which is not only time-consuming and labor-intensive, but also prone to errors due to the subjective factors of the observer. Therefore, in the field of animal health and welfare evaluation, it is urgent to develop an automated, quantifiable, and high-precision animal behavior monitoring system. For example, through intelligent technology, administrators or owners can remotely and real-time monitor animal behavior changes, thereby discovering abnormal behaviors or diseases in a timely manner, reducing economic losses, and improving management efficiency.

[0003] Currently, animal behavior automated monitoring methods mainly rely on computer vision technology, wearable sensors, and sound signal recognition systems. Among them, wearable sensors are widely used in animal behavior monitoring due to their small size, low power consumption, easy integration, resistance to external environmental interference, and ability to provide accurate motion data. In practical applications, wearable sensors such as accelerometers, gyroscopes, magnetometers, pressure sensors, and global navigation satellite systems (GNSS) are usually installed on different parts of animals (such as ears, necks, legs) to accurately collect motion signals of various behaviors (such as standing, trotting, walking, drinking water, running, etc.). After signal preprocessing, the data is input into various model networks to learn data features, and finally the results (various behaviors of animals) are output by the classifier, thereby achieving the evaluation of animal health and welfare status.

[0004] For motion signals (time series) obtained from animals, advanced data processing and learning techniques must be used to obtain accurate analysis. Machine learning, as a method that relies on computers to automatically learn data patterns and rules, can effectively learn and analyze data features.

[0005] However, traditional machine learning methods rely on manual feature extraction, are susceptible to expert knowledge, and have problems such as difficulty in feature extraction and low generalization ability. In contrast, deep learning technology can learn features directly from raw sensor data in an end-to-end manner due to its multi-layer neural network, and can achieve hierarchical abstraction from low to high. In addition, with its high-dimensional unstructured data processing, the model's adaptability to data is greatly improved. These advantages make deep learning models more flexible and robust, thereby improving the performance of animal behavior monitoring.

[0006] However, existing animal behavior recognition technologies based on deep learning and motion sensors still have the following shortcomings in actual training and deployment:

[0007] 1. Deep learning has high requirements for labeled data, but the labeling process is costly and time-consuming, and errors are prone to occur during the labeling process, resulting in a lack of current labeled data, which is difficult to meet the model training requirements;

[0008] 2. Although existing technologies use a variety of methods to alleviate the problem of insufficient sample size, they generally ignore the rich feature information contained in unlabeled data. How to effectively exploit and utilize the potential value of unlabeled data has become a key technical problem to be solved.

[0009] To solve the above problems, the present application provides an animal behavior recognition enhancement method driven by self-supervised pre-training, which relieves the problem of lack of labeled data by exploiting the features of unlabeled data. SUMMARY

[0010] The present application is to overcome the problem of limited animal behavior classification performance caused by insufficient labeled data in the prior art, and provides an animal behavior recognition enhancement method based on self-supervised pre-training that uses a contrast learning strategy to pre-train the model using large-scale unlabeled data across species, enabling the model to learn effective feature representations autonomously.

[0011] To achieve the above application purposes, the present application adopts the following technical solutions:

[0012] The animal behavior recognition enhancement method based on self-supervised pre-training comprises the following steps:

[0013] S1, a benchmark model is constructed based on the PatchTST model, which introduces channel independence processing and Transformer architecture by dividing the time series into blocks, for completing local feature extraction and long-range dependency modeling of time series data;

[0014] S2, using a self-supervised pre-training framework based on time-frequency consistency (TF-C), performing time-frequency consistency contrastive learning on a large-scale cross-species unlabeled dataset, and optimizing through time-domain and frequency-domain enhancement, contrastive loss, and consistency loss to enable the benchmark model constructed in step S1 to capture time-frequency invariant features;

[0015] S3, migrating the pre-trained encoder weights to the PatchTST model, initializing and fine-tuning on a specific species labeled dataset, and combining channel attention and feature reorganization convolution optimization to complete cross-species knowledge transfer and target task adaptation.

[0016] Preferably, in step S1, the time series block specifically includes the following process:

[0017] S11, for an input single-variable time series X = [x1, x2, ··· xL] of length L, using a fixed block length P and step size S to segment, generating an overlapping or non-overlapping block sequence; before block, fill the end of the sequence with S repeated last values xL to ensure that the sequence length meets the block requirement, the formula is: L L

[0018]

[0019] where N is the number of blocks, each block can retain local time sequence fragments unified representation of real space, used to define or represent data space, parameter space, loss function space, etc.), by block, the number of input tokens is reduced from L to At the same time, mark the block order through position embedding to retain time dependence.

[0020] Preferably, in step S1, the channel independence processing specifically includes the following process:

[0021] S12, for multivariate time series data, separate each channel as an independent input;

[0022] For the i-th channel sequence through a learnable linear projection and position encoding mapping to a latent space with dimension D, the specific formula is as follows:

[0023]

[0024] where, denotes the block sequence mapping to the input of the Transformer encoder.

[0025] ​​As preferred, in step S1, the modeling of the Transformer architecture specifically comprises the following process:

[0026] S13, multi-head attention is calculated for the block sequence of each channel to establish global dependence across time steps; for the hth attention head, first convert the input into a query matrix key matrix value matrix

[0027]

[0028] wherein, denotes the query projection matrix, denotes the key projection matrix, denotes the value projection matrix; d k denotes the feature dimension of the key and query vectors.

[0029] The attention output is obtained using scaled dot-product is:

[0030]

[0031] As preferred, in step S1, the channels of the Transformer architecture share the same number of embedding layers and Transformer encoder weights.

[0032] As preferred, step S1 further comprises the following steps:

[0033] S14, training the baseline model on the labeled dataset of the goat species, inputting the block and channel encoding, extracting global features through the Transformer encoder, and outputting the classification result through the fully connected layer to verify the recognition ability of the animal basic behavior; the basic behavior includes standing and eating.

[0034] As preferred, in step S2, the time-frequency consistency contrast learning specifically comprises the following steps:

[0035] S21, constructing a time-domain contrast loss function L T,i :

[0036] Setting a time series sample x i , generating an enhanced sample set from which an enhanced sample is randomly selected a time-domain encoder G T mapping the original sample and the enhanced sample into embedding vectors and maximizing the similarity in the positive sample pair and suppressing the negative sample pair through normalized temperature cross-entropy loss:

[0037]

[0038] where denotes cosine similarity, indicator function to ensure excluding self-contrast, τ is temperature hyper-parameter; refers to different time series samples or corresponding augmented samples; refers to the unlabeled dataset used for pre-training, which acts as a negative sample in the loss function.

[0039] S22, constructing a frequency domain contrast loss function L F,i :

[0040] performing Fourier transform on the sample x i to obtain the frequency spectrum generating a set of perturbed samples through a frequency domain augmentation strategy a frequency domain encoder G F mapping the original frequency spectrum and the perturbed frequency spectrum to and The frequency domain contrast loss function is defined as:

[0041]

[0042] S23, constructing a time-frequency domain contrast consistency loss function L C,i :

[0043] Setting the time domain and frequency domain representations from the same sample to remain aligned in the latent space, introducing a cross-domain projector R T and R F , embedding the time domain and the frequency domain into a shared time-frequency space to obtain the time embedding projected representation and the time embedding projected representation Based on the idea of triplet loss, construct the time-frequency consistency loss:

[0044]

[0045] where, denotes the distance between the original time domain and frequency domain embeddings, and in each pair, covers the cross-domain distance between augmented samples, and δ is a boundary hyper-parameter.

[0046] S24, constructing the total loss function:

[0047] Integrating the three loss functions in steps S21, S22 and S23, and balancing contrast learning and time-frequency consistency by adding a weighting coefficient λ:

[0048]

[0049] where λ controls the relative importance of contrastive and consistency loss, and the total loss is calculated by summing over all pre-training samples.

[0050] As preferred, step S3 comprises the following steps:

[0051] S31, migrating the TF-C pre-trained time domain encoder weights to the PatchTST model, initializing the backbone network of the benchmark model, and retaining the modeling ability of “local block features + cross-species commonality”; the time domain encoder weights include block layers and Transformer encoders;

[0052] S32, classification module adaptation and enhancement, the specific process is as follows:

[0053] S321, setting the feature after inputting through the PatchTST backbone network and global average pooling as where B is the batch size, C is the channel number, and D is the feature dimension; the calculation of channel attention is actually the calculation of multi-head attention, first transpose the feature to adapt to the input of the multi-head attention layer, and get the transposed feature tensor F t :

[0054]

[0055] where transpose(F,0,1) represents exchanging the positions of the 0th and 1st dimensions of the feature tensor F.

[0056] Then the attention weight is calculated, and after weighted aggregation, the outputs of all heads are spliced and linearly projected, and finally residual connection and transposition are performed to obtain the final feature F attn :

[0057] F attn = F + transpose(Attention(F t ),0,1) (12)

[0058] where Attention(F t ) is the splicing of the output of multi-head attention.

[0059] S322, constructing a feature reorganization convolutional layer for enhancing the representation ability of the internal features of each channel; the feature reorganization convolutional layer uses a depth separable convolution, which consists of a channel grouping convolution and a channel fusion convolution; the channel grouping convolution uses a one-dimensional convolution for each channel c∈{1,2,…,C} independently, and obtains a single-channel convolution output Y c :

[0060] ​Y c =GELU(W c *F attn +b c ) (13)

[0061] Wherein, GELU is Gaussian error linear unit activation function, is a convolution kernel (k=3), * represents a convolution operation, b c is a bias term;

[0062] The convolution result output Y of all channels c The complete output Y after splicing along the channel dimension, the channel fusion convolution uses 1x1 convolution to fuse channels to obtain the fused features:

[0063] Z=W point *Y+b point (14)

[0064] Wherein, is the weight of 1x1 convolution, is the feature after feature recombination fusion, b point is a bias term;

[0065] Finally, the feature after feature recombination fusion is input into the next layer through layer normalization, which is used to flatten the feature and classify by using the classifier.

[0066] The application also provides an animal behavior recognition enhancement system based on self-supervised pre-training, comprising:

[0067] A model construction module is configured to construct a benchmark model based on a PatchTST model, introduce channel independence processing and a Transformer architecture by dividing a time sequence into blocks, and complete local feature extraction and long-range dependence modeling of the time sequence data.

[0068] A pre-training module is configured to use a self-supervised pre-training framework based on time-frequency consistency TF-C to perform time-frequency consistency comparison learning on a large-scale cross-species unlabeled data set, and optimize through time domain and frequency domain enhancement, comparison loss and consistency loss, so that the constructed benchmark model captures time-frequency domain invariant features.

[0069] A migration module is configured to migrate the pre-trained encoder weight to the PatchTST model, initialize and fine-tune on a specific species labeled data set, and combine channel attention and feature recombination convolution optimization to complete cross-species knowledge migration and target task adaptation.

[0070] Compared with the prior art, the present application has the beneficial effects that: (1) the present application first constructs a large-scale pre-training data set (576,897 samples) across species (sheep, horses, cattle, etc.), breaks through the limitation of traditional pre-training based on single-species data, utilizes the complementarity of multi-species data, reduces the dependence on target species labeled data, and provides an efficient migration solution for small sample scenarios; in addition, through the self-supervised learning method based on time-frequency consistency, the behavior common characteristics among species are effectively mined, and the enhancement method (time domain: scaling, time shift, etc.; frequency domain: spectrum disturbance, increasing and decreasing frequency components, etc.) used in constructing positive and negative sample pairs avoids the destruction of traditional data enhancement to the behavior time sequence information; (2) the present application proposes a hierarchical time sequence modeling architecture, which divides the time sequence into a local block (Patch) sequence to extract fine-grained behavior characteristics, adopts a channel-independent calculation strategy to improve the feature extraction efficiency; at the same time, a Transformer-based architecture is designed to model the long-range time sequence dependence relationship of behavior, and a deformable convolution and channel attention mechanism are combined to realize dynamic feature adaptive fusion of local fine-grained features and global behavior patterns. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1 A flowchart of the animal behavior recognition enhancement method based on self-supervised pre-training in the present application;

[0072] Figure 2 A schematic diagram of the animal behavior recognition network based on PatchTST in the present application;

[0073] Figure 3 A pre-training framework diagram based on time-frequency consistency in the present application;

[0074] Figure 4 A schematic diagram of the feature distribution before and after pre-training provided by the embodiment of the present application;

[0075] Figure 5 A curve schematic diagram of the time-frequency consistency loss in the pre-training provided by the embodiment of the present application;

[0076] Figure 6 A comparative schematic diagram of the training accuracy and training loss curve provided by the embodiment of the present application;

[0077] Figure 7 A confusion matrix comparison diagram before and after using the pre-training weight initialization provided by the embodiment of the present application;

[0078] Figure 8 A feature distribution comparison diagram before and after using the pre-training weight initialization provided by the embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to more clearly illustrate the embodiments of the present application, the specific embodiments of the present application will be described below with reference to the drawings. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art, and other embodiments can also be obtained.

[0080] In the process of constructing the animal behavior recognition model pre-training and fine-tuning framework, in view of the contradiction between the time sequence dependence of the data itself and the traditional data enhancement technology, and the generalization-specificity balance dilemma faced in the small sample domain adaptation, the present application adopts a "three-stage" training strategy, combines the time-frequency contrastive learning pre-training framework and the classification model fine-tuning, realizes the enhanced learning of the characteristics of each species sample, and simultaneously learns the common characteristics from the cross-species data and migrates to the specific species behavior to realize accurate classification.

[0081] As shown in Figures 1 to 3 , the present application provides an animal behavior recognition enhancement method based on self-supervised pre-training, comprising the following steps:

[0082] S1, constructing a benchmark model based on a PatchTST model, introducing channel independence processing and a Transformer architecture by dividing the time sequence into blocks, for completing local feature extraction and long-range dependence modeling of the time sequence data;

[0083] S2, using a self-supervised pre-training framework based on time-frequency consistency TF-C, performing time-frequency consistency contrastive learning on a large-scale cross-species unlabeled data set, and optimizing through time domain and frequency domain enhancement, contrastive loss and consistency loss, so that the benchmark model constructed in step S1 captures time-frequency domain invariant features;

[0084] S3, migrating the pre-trained encoder weight to the PatchTST model, initializing and fine-tuning on a specific species labeled data set, and combining channel attention and feature reorganization convolution optimization to complete cross-species knowledge migration and target task adaptation.

[0085] Through the above process, the problem of labeled data scarcity is effectively alleviated, and the generalization performance and classification accuracy of the model in cross-species behavior recognition are improved.

[0086] Further, as shown in Figure 2 , in step S1, the time sequence blocking specifically includes the following process:

[0087] S11, for an input single-variable time sequence X=[x1,x2,···x L ] with a length of L, fixed block length P and step S are used for segmentation to generate overlapping or non-overlapping block sequences; before blocking, the last value xL , to ensure that the sequence length meets the block requirement, the formula is:

[0088]

[0089] where N is the number of blocks, each block Local timing fragments can be retained (in this application Uniformly represent real space, used to define or represent data space, parameter space, loss function space, etc.), reduce the input token number from L to Reduce the amount of attention calculation, and mark the block order through position embedding to retain time dependence.

[0090] Further, in step S1, the channel independence process specifically includes the following process:

[0091] S12, for multivariate time series data (such as three-axis acceleration X = [X 1 , X 2 , X 3 ], X i is a single channel sequence), separate each channel into an independent input to avoid interference between channels;

[0092] Map the i-th channel sequence to a latent space with dimension D through a learnable linear projection and position encoding , the specific formula is as follows:

[0093]

[0094] wherein, indicates that the block sequence is mapped to the input of the Transformer encoder.

[0095] Further, in step S1, the modeling of the Transformer architecture specifically includes the following process:

[0096] S13, calculate the multi-head attention of the block sequence of each channel to establish global dependence across time steps; for the h-th attention head, first convert the input into a query matrix key matrix value matrix

[0097]

[0098] wherein, indicates the query projection matrix, indicates the key projection matrix, indicates the value projection matrix; d k indicates the feature dimension of the key and query vectors.

[0099] Attention output using scaled dot-product For:

[0100]

[0101] Each channel shares the same number of embedding layers and Transformer encoder weights, and multi-head attention enables multi-dimensional feature interaction through parallel attention heads, capturing global dependencies at different time steps. Feedforward networks and layer normalization: enhance non-linear representation through Gaussian Error Linear Unit (GELU) activation function and residual connection, stabilize training, and alleviate gradient vanishing.

[0102] Further, step S1 further includes the following steps:

[0103] S14, training the baseline model on the annotated dataset of the goat species, inputting the block and channel encoding, extracting global features through the Transformer encoder, and outputting the classification result through the fully connected layer, verifying the recognition ability of the animal basic behavior; the basic behavior includes standing and eating.

[0104] Further, as shown in step S2, the time-frequency consistency contrast learning specifically includes the following steps: Figure 3

[0105] S21, constructing a time-domain contrast loss function L T,i :

[0106] Set the time series sample x i , generate an enhanced sample set from it by a time-domain enhancement strategy Randomly select an enhanced sample T from it and Map the original sample and the enhanced sample to embedding vectors

[0107]

[0108] where represents the cosine similarity, and the indicator function ensures to exclude self-contrast, and τ is a temperature hyperparameter; refers to different time series samples or corresponding enhanced samples; refers to the unlabeled dataset used for pre-training, which provides negative samples in the loss function.

[0109] ​S22, constructing a frequency domain contrast loss function L F,i :

[0110] Performing Fourier transform on the sample x i to obtain the frequency spectrum Generating a set of perturbed samples through a frequency domain enhancement strategy A frequency domain encoder G F Mapping the original frequency spectrum and the perturbed frequency spectrum to and Then the frequency domain contrast loss function is defined as:

[0111]

[0112] S23, constructing a time-frequency domain contrast consistency loss function L C,i :

[0113] Setting that the time domain and frequency domain representations from the same sample remain aligned in the latent space, introducing a cross-domain projector R T and R F , embedding the time domain and the frequency domain into a shared time-frequency space, obtaining the time embedding projected representation and the time embedding projected representation Based on the triplet loss idea, constructing a time-frequency consistency loss:

[0114]

[0115] wherein, represents the distance between the original time domain and frequency domain embeddings, in each pair, covers the cross-domain distance between the enhanced samples, and δ is a boundary hyperparameter.

[0116] S24, constructing a total loss function:

[0117] Integrating the three loss functions in steps S21, S22 and S23, balancing contrast learning and time-frequency consistency by adding a weighting coefficient λ:

[0118]

[0119] wherein, λ controls the relative importance of contrast and consistency loss, and the total loss is calculated by summing of all pre-training samples.

[0120] Further, step S3 includes the following steps:

[0121] S31, migrate the TF-C pre-trained time domain encoder weights (including block layer and Transformer encoder) to the PatchTST model, initialize the backbone network of the benchmark model, and retain the modeling ability of "local block features + cross-species commonality";

[0122] S32, classification module adaptation and enhancement, the specific process is as follows:

[0123] S321, set the feature after inputting through the PatchTST backbone network and global average pooling as Where B is the batch size, C is the channel number, and D is the feature dimension; the calculation of channel attention is actually the calculation of multi-head attention. First, the feature is transposed to adapt to the input of the multi-head attention layer, and the transposed feature tensor F is obtained t :

[0124]

[0125] Where transpose(F,0,1) represents exchanging the positions of the 0th and 1st dimensions of the feature tensor F.

[0126] Then the attention weight is calculated, and after weighted aggregation, the outputs of all heads are spliced and linearly projected, and finally residual connection and transposition are performed to obtain the final feature F of the attention operation attn :

[0127] F attn =F+transpose(Attention(F t ),0,1) (12)

[0128] Where Attention(F t ) is the concatenation of the outputs of multi-head attention.

[0129] S322, construct a feature reorganization convolutional layer to enhance the representation ability of the features within each channel; the feature reorganization convolutional layer uses a depth separable convolution, which consists of a channel grouping convolution and a channel fusion convolution. The channel grouping convolution uses a one-dimensional convolution for each channel c∈{1,2,…,C} independently to obtain a single-channel convolution output Y c :

[0130] Y c =GELU(W c *F attn +b c ) (13)

[0131] Where GELU is the Gaussian error linear unit activation function, is the convolution kernel (k=3), and * represents the convolution operation, bc is a bias term;

[0132] Convolution result output Y of all channels c Complete output Y after splicing along the channel dimension, the channel fusion convolution uses 1x1 convolution to fuse channels to obtain fused features:

[0133] Z=W point *Y+b point (14)

[0134] wherein, is the weight of 1x1 convolution, is the feature after feature recombination and fusion, b point is a bias term;

[0135] Finally, the feature after feature recombination and fusion is input to the next layer through layer normalization, which is used to flatten the feature and classify it using a classifier.

[0136] In addition, the present application also provides an animal behavior recognition enhancement system based on self-supervised pre-training, comprising:

[0137] The model construction module is used to construct a benchmark model based on the PatchTST model, introduce channel independence processing and a Transformer architecture by dividing the time series into blocks, and is used to complete local feature extraction and long-range dependence modeling of time series data.

[0138] The pre-training module is used to adopt a self-supervised pre-training framework based on time-frequency consistency TF-C, perform time-frequency consistency comparison learning on a large-scale cross-species unlabeled data set, and optimize through time domain and frequency domain enhancement, comparison loss and consistency loss, so that the constructed benchmark model captures time-frequency domain invariant features.

[0139] The migration module is used to migrate the pre-trained encoder weight to the PatchTST model, initialize and fine-tune on a specific species labeled data set, and combine channel attention and feature recombination convolution optimization to complete cross-species knowledge migration and target task adaptation.

[0140] The following further illustrates the significant technical effects of the present application through experiments.

[0141] 1. Experimental data and settings

[0142] 1-1. Experimental data description and preprocessing

[0143] As shown in Table 1, the present application first constructs a cross-species pre-training data set based on four open source data sets for model pre-training, as follows:

[0144] Cross-species pre-training dataset: This dataset integrates multiple public datasets from three species of cattle, sheep and horses, containing a total of 576,897 samples, covering 13 typical animal behaviors. In view of the differences existing in the original data (including differences in sensor models, sampling frequencies, etc.), the present application carries out a systematic data processing procedure: first, the different sources of public datasets are cleaned, and the data with high sampling rate (such as 100Hz) is down-sampled, and the data with low sampling rate (such as 12.5Hz) is up-sampled by nonlinear interpolation to unify the frequency to 25Hz; at the same time, only the three-axis acceleration data common to each dataset are retained, a sliding window with a length of 2s is used to divide the signal, and Z-score standardization operation is performed to generate a tensor with dimensions of 576897x50x3. Note that this dataset is a unlabeled dataset, which is used for self-supervised learning pre-training.

[0145] Target species fine-tuning dataset: This dataset contains 42,943 labeled samples of 5 sheep, covering 5 typical behaviors: standing (43.15%), running (0.87%), eating (35.35%), trotting (0.44%), and walking (20.19%). The class imbalance rate is 98.05, among which the minority class "trotting" contains only 189 samples. The present application uses a 2s sliding window to divide the signal, and performs Z-score standardization operation to generate a tensor with dimensions of 42943x50x3. This dataset is a labeled dataset, which is used for fine-tuning of the pre-trained model.

[0146] Table 1 Wearable sensor-based animal behavior recognition public dataset table

[0147]

[0148]

[0149] In Table 1, IMU: Inertial Measurement Unit, represents inertial measurement unit; GNSS: Global Navigation Satellite System, represents global navigation satellite system.

[0150] 1-2. Experimental setup

[0151] The present application uses accuracy (Accuracy), recall (Recall), F1 score (F1 score), macro-average (Macro-average) and weighted average (Weighted-average) as indicators to evaluate the overall performance of the classification network.

[0152] To verify the effectiveness of the method, the present application designs a comprehensive comparative experiment. The comparative experiment revolves around the core issue of "whether cross-species pre-training can improve the performance of specific species behavior recognition", and adopts strict single variable control principle in setting to ensure that all conditions are completely consistent except for the model initialization method: first, in model construction, the baseline classification model uses PatchTST architecture, and all parameters (including embedding layer, time sequence feature extraction layer and classification head) are randomly initialized. The pre-trained classification model loads the pre-trained weights based on the same architecture on the large-scale cross-species dataset (including cattle, sheep, horses and other species, 576,897 samples, 13 behaviors), which extracts general behavior features through the self-supervised learning method based on TF-C, and maximizes the consistency of time and frequency domain features of the same sample. The target species dataset (including 5 sheep, 42,943 samples, 5 behaviors) for fine-tuning process is divided into training set (60%, data of 3 sheep), validation set (20%, data of 1 sheep) and test set (20%, data of 1 sheep) according to the proportion of the number of species. During the training process, both of them use AdamW optimizer (learning rate 1e -4 , weight decay 0.01, batch size 32), train for 100 rounds and enable early stopping mechanism (terminate if the validation set loss does not decrease for 10 consecutive rounds); In order to make the pre-trained model better adapt to new data, the "layer-by-layer unfreezing" training strategy is adopted in the fine-tuning experiment to prevent unstable training caused by directly unfreezing all layers; During evaluation, the loss and accuracy of the training set and the validation set are recorded every round, and finally the accuracy, macro average, weighted average, F1 score, precision and recall are calculated on the test set, the confusion matrix is generated, and the last layer feature vector of the model (dimensionality reduction to 2D, perplexity = 30, iteration 500 times) is visualized through T-SNE (t-distributed Stochastic Neighbor Embedding) to analyze the feature separation degree of "walking-eating", "jogging-running" and other easily confused categories; The experiment strictly controls the variables such as hardware environment and random seed (fixed at 42), repeats 3 independent experiments to take the average value to eliminate randomness, and ensures the reliability and repeatability of the results. The core logic is to clearly reveal the substantial gains of cross-species pre-training in feature learning efficiency, generalization ability and discriminative representation through standardized process and variable control.

[0153] 2. Performance comparison of the method of the present application and existing methods

[0154] Figure 4 and Figure 5The performance comparison results of the proposed method and existing methods on sheep and cattle datasets are presented. In comparison, the proposed method for animal behavior recognition enhancement driven by self-supervised pre-training demonstrates significant advantages: by integrating public datasets of multiple species, such as cattle, sheep, and horses, for self-supervised pre-training, and using time-frequency consistency contrastive learning (TF-C) to force the model to align time-domain waveforms with frequency-domain spectral features, the time-frequency loss after pre-training is reduced from 10.3291 to 4.6732, demonstrating the overall effectiveness of the pre-trained model.

[0155] Figure 4 In [1], t-SNE (t-Distributed Stochastic Neighbor Embedding) refers to t-distributed stochastic neighbor embedding, which is a nonlinear machine learning algorithm for dimensionality reduction and visualization of high-dimensional data.

[0156] In terms of convergence speed, Figure 6 As shown in the figure, the training loss of the classification model with reinitialized pre-trained weights (hereinafter referred to as the pre-trained classification model) decreases and converges faster than the baseline classification model. The training accuracy of the pre-trained classification model also converges faster and has higher precision. In both graphs, the pre-trained classification model reaches convergence after only about 35 rounds of training, and finally shows better results than the baseline classification model. This shows that the model after loading the pre-trained weights can quickly adapt to the training data, and quickly learn and use data features for classification prediction.

[0157] In terms of overall performance on the test set, experimental analysis shows that the pre-training framework effectively improves animal behavior recognition performance. As shown in Table 2, the TF-C pre-trained model achieves higher accuracy, F1 score, precision, and recall than the baseline model. The accuracy reaches 92.55%, a 4.79% improvement over the baseline model (87.76%). In terms of overall classification performance, the pre-trained model's F1 score improves by 4.57%, demonstrating its improved generalization performance in class imbalance scenarios.

[0158] Table 2 Comparison of model performance before and after pre-training

[0159]

[0160] In terms of classification accuracy, the pre-trained model performs better than the baseline model in the test set confusion matrix. Figure 7As can be seen, firstly, the accuracy of the trotting category, which has the fewest categories, is improved from 66.7% to 76.9%, an increase of 10.2%; the accuracy of the grazing and walking categories is also improved by 0.6% and 15.2%, respectively; the accuracy of the remaining categories is slightly reduced, but still remains at about 90%; from Figure 8 The feature distribution also shows that the feature overlap between walking and grazing, trotting and running is reduced, indicating that the discriminative representation ability of the model for behavior timing patterns has been significantly improved after pre-training.

[0161] 3. Ablation experiment

[0162] By systematically removing different modules of the model, the necessity and effectiveness of each part are verified, the contribution of each part to the performance is evaluated, the structure is optimized to avoid redundancy, and the model interpretability is enhanced. The ablation experiment is designed to compare the comprehensive performance of the following model variants: 1) PatchTST base model, 2) PatchTST (feature convolution) model, 3) PatchTST (channel attention) model, 4) PatTST (feature convolution layer + channel attention) model.

[0163] The influence of different modules on the performance of the model is shown in Table 3:

[0164] Table 3: Influence of each module on the performance of the model

[0165]

[0166] The addition of the channel multi-head attention mechanism in the base PatchTST improves the F1 score by 1.36%, and the macro average accuracy is improved from 75.23% to 85.37%, an increase of 8.34%, indicating that it may improve the discrimination ability of the minority classes; after introducing the feature convolution alone, the model accuracy is improved by 0.61%, and the F1 score is improved by 1.73%, indicating that the convolution has improved the ability to capture local timing features in detail; after joint debugging experiments, the accuracy reaches 87.76%, achieving the optimal F1 score (77.75%), which is 3.61% higher than the baseline model, and the weighted average accuracy also reaches the optimal value (90.51%), and it is worth noting that the model maintains a high accuracy while the macro average is only slightly lower than the attention model, indicating that the combination of the two performs well in handling global information and local details.

[0167] The method of the present application firstly constructs a self-supervised pre-training model based on time-frequency consistency (TF-C), trains on a large-scale cross-species unlabeled data set (containing sheep, horses, cattle and other species, a total of 576,897 samples), and captures general features by reducing the time-frequency domain representation difference through the time-frequency consistency loss; then the pre-trained time domain encoder weight is migrated to the animal behavior recognition network based on Patch Time Series Transformer (PatchTST), and fine-tuned on the sheep data set (42,943 samples), thereby improving the animal behavior recognition performance under limited labeled data.

[0168] The above only details the preferred embodiments and principles of the present application, and for those of ordinary skill in the art, the specific implementation will change according to the idea provided by the present application, and these changes should also be considered as the protection scope of the present application.

Claims

1. Animal behavior recognition enhancement method based on self-supervised pre-training, characterized by: The steps include: S1, a benchmark model is built based on the PatchTST model. By dividing the time series into blocks, channel independence processing and the Transformer architecture are introduced to complete local feature extraction and long-range dependency modeling of time series data; S2 uses a self-supervised pre-training framework based on time-frequency consistency TF-C to conduct time-frequency consistency comparative learning on a large-scale cross-species unlabeled dataset. Through time and frequency domain enhancement, contrast loss, and consistency loss optimization, the baseline model constructed in step S1 captures the invariant features in the time and frequency domains. S3 migrates the pre-trained encoder weights to the PatchTST model, initializes and fine-tunes it on a species-specific annotated dataset, and combines channel attention and feature reorganization convolution optimization to complete cross-species knowledge transfer and target task adaptation.

2. The method for enhancing animal behavior recognition based on self-supervised pre-training according to claim 1, characterized in that: In step S1, the time series segmentation specifically includes the following process: S11, for the input univariate time series X=[x1,x2,···x L ], using fixed block length P and step size S for segmentation, generating overlapping or non-overlapping block sequences; filling the end of the block forward sequence with S repeated last value x L , used to ensure that the sequence length meets the block requirements, the formula is: Among them, N is the number of blocks, each block Keep the local time sequence fragments and reduce the number of input tokens from L to At the same time, the block order is marked by position embedding, preserving the temporal dependency; Represents the real number space.

3. The method for enhancing animal behavior recognition based on self-supervised pre-training according to claim 2, characterized in that: In step S1, the channel independence processing specifically includes the following process: S12, for multivariate time series data, separates each channel into independent inputs; For the i-th channel sequence Through learnable linear projection and positional encoding Mapped to a latent space of dimension D, the specific formula is as follows: in, Represents the mapping of a block sequence to the input of the Transformer encoder.

4. The method for enhancing animal behavior recognition based on self-supervised pre-training according to claim 3, characterized in that: In step S1, the modeling of the Transformer architecture specifically includes the following process: S13, calculate the multi-head attention for the block sequence of each channel to establish the global dependency across time steps; for the h-th attention head, first convert the input into a query matrix Bond Matrix Value Matrix in, represents the query projection matrix, represents the key projection matrix, Represents the value projection matrix; d k Represents the feature dimension of the key and query vector; Use scaled dot product to get attention output as follows:

5. The method for enhancing animal behavior recognition based on self-supervised pre-training according to claim 4, characterized in that: In step S1, each channel of the Transformer architecture shares the same number of embedding layers and Transformer encoder weights.

6. The method for enhancing animal behavior recognition based on self-supervised pre-training according to claim 5, characterized in that: Step S1 also includes the following steps: S14, train a baseline model on a labeled dataset of goat species. After input block and channel encoding, extract global features through the Transformer encoder, and output classification results through the fully connected layer to verify the ability to recognize basic animal behaviors; the basic behaviors include standing and eating.

7. The method for enhancing animal behavior recognition based on self-supervised pre-training according to claim 6, characterized in that: In step S2, the time-frequency consistency contrast learning specifically includes the following steps: S21, construct the time domain contrast loss function L T,i : Set the time series sample x i , generate enhanced sample sets through time domain enhancement strategy Randomly select augmented samples from Time domain encoder G T Map the original sample and the enhanced sample into embedding vectors respectively and Maximize the similarity among positive pairs and suppress negative pairs by normalized temperature-scale cross entropy loss: in Represents cosine similarity, indicator function Make sure to exclude self-contrast,τ is the temperature hyperparameter; Refers to different time series samples or corresponding enhanced samples; Represents the unlabeled dataset used for pre-training, which is used to provide negative samples in the loss function. S22, construct the frequency domain contrast loss function L F,i : For sample x i Perform Fourier transform to get the spectrum Generate perturbation sample sets through frequency domain enhancement strategy Frequency domain encoder G F Map the original spectrum and the perturbation spectrum into and Then the frequency domain contrast loss function is defined as: S23, construct the time-frequency domain contrast consistency loss function L C,i : Assuming that the time domain and frequency domain representations from the same sample are aligned in the latent space, a cross-domain projector R is introduced. T and R F , embed the time domain into and frequency domain embedding Mapped to the shared time-frequency space, the time-embedded projection representation is obtained Representation after projection with frequency embedding Based on the triplet loss idea, the time-frequency consistency loss is constructed: in, Represents the distance between the original time domain and the frequency domain embedding. In each pair, It covers the cross-domain distance between enhanced samples, and δ is the boundary hyperparameter. S24, construct the total loss function: Integrate the three loss functions in step S21, step S22, and step S23, and balance contrastive learning and time-frequency consistency by adding a weighting coefficient λ: Among them, λ controls the relative importance of contrast and consistency loss, which is obtained by Sum them to calculate the total loss.

8. The method for enhancing animal behavior recognition based on self-supervised pre-training according to claim 7, characterized in that: Step S3 includes the following steps: S31: Migrate the TF-C pre-trained temporal encoder weights to the PatchTST model to initialize the baseline model backbone network, retaining the ability to model "local block features + cross-species commonality"; the temporal encoder weights include the block layer and Transformer encoder; S32, classification module adaptation and enhancement, the specific process is as follows: S321, set the input feature after passing through the PatchTST backbone network and global average pooling to be Among them, B is the batch size, C is the number of channels, and D is the feature dimension; the calculation of channel attention is actually the calculation of multi-head attention. First, the features are transposed to adapt to the input of the multi-head attention layer, and the transposed feature tensor F is obtained. t : Among them, transpose(F,0,1) means exchanging the positions of the 0th dimension and the 1st dimension of the feature tensor F; Then the attention weight is calculated, and after weighted aggregation, the output of all heads is spliced ​​and linearly projected, and finally the residual link and transposition are performed to obtain the final feature F of the attention operation attn : F attn =F+transpose(Attention(F t ),0,1) (12) Among them, Attention(F t ) That is, concatenate the outputs of multiple attention heads; S322, construct a feature reorganization convolution layer to enhance the representation capability of the internal features of each channel; the feature reorganization convolution layer adopts depthwise separable convolution, which consists of two parts: channel grouping convolution and channel fusion convolution; channel grouping convolution uses one-dimensional convolution for each channel c∈{1,2,…,C} independently to obtain a single-channel convolution output Y c : Y c =GELU(W c *F attn +b c ) (13) Among them, GELU is the Gaussian error linear unit activation function, is the convolution kernel (k=3), * represents the convolution operation, b c is the bias term; the convolution result of all channels outputs Y c After concatenating the complete output Y along the channel dimension, the channel fusion convolution uses 1×1 convolution to perform channel fusion and obtain the fused features: Z=W point *Y+b point (14) in, is the weight of the 1×1 convolution, is the feature after feature reorganization and fusion, b point is the bias term; Finally, the features after feature reorganization and fusion are normalized and input into the next layer to flatten the features for classification using the classifier.

9. An animal behavior recognition enhancement system based on self-supervised pre-training, used to implement the animal behavior recognition enhancement method based on self-supervised pre-training according to any one of claims 1 to 8, characterized in that: The animal behavior recognition enhancement system based on self-supervised pre-training includes: The model building module is used to build a benchmark model based on the PatchTST model. By partitioning the time series into blocks, introducing channel independence processing and the Transformer architecture, it is used to complete local feature extraction and long-range dependency modeling of time series data; A pre-training module uses a self-supervised pre-training framework based on time-frequency consistency (TF-C) to perform time-frequency consistency comparative learning on a large-scale, cross-species, unlabeled dataset. By optimizing time and frequency domain enhancement, contrast loss, and consistency loss, the constructed baseline model captures the invariant features in the time and frequency domains. The migration module is used to migrate the pre-trained encoder weights to the PatchTST model, initialize and fine-tune it on a species-specific annotated dataset, and combine channel attention and feature reorganization convolution optimization to complete cross-species knowledge transfer and target task adaptation.

Images