A beef cattle abnormal behavior video time sequence positioning method based on time offset loss

Abstract

The application provides a beef cattle abnormal behavior video timing positioning method based on time offset loss, comprising the following steps: S1, constructing a beef cattle abnormal behavior timing positioning dataset; S2, extracting beef cattle abnormal behavior video features; S3, realizing a beef cattle abnormal behavior timing positioning method; S4, training a beef cattle abnormal behavior timing positioning model by using a center point-start and end time offset loss function and a focal loss function; and S5, beef cattle abnormal behavior timing positioning. In view of the problem that the existing timing positioning method is only for a single target and has poor accuracy, the loss function for calculating the start and end time offset of the behavior, i.e., the center point-start and end time offset loss function, is used to construct the beef cattle abnormal behavior video timing positioning method based on time offset loss, so that the timing positioning of the beef cattle abnormal behavior in the monitoring video is realized, and the demand for obtaining the abnormal behavior occurrence time in the real breeding scene for precise management of beef cattle is met.

Description

Technical Field

[0001] This invention belongs to the field of video image understanding technology, and relates to beef cattle behavior, specifically to a video temporal localization method for abnormal beef cattle behavior based on time offset loss. Background Technology

[0002] In beef cattle farming, feeders need to constantly monitor and promptly detect any abnormal conditions in the cattle, and then adjust the cattle pen environment and feed accordingly. Traditional farming relies on manual patrols to monitor abnormal behavior. This method is time-consuming, labor-intensive, and costly, and it is difficult to achieve 24 / 7 monitoring of the cattle pens.

[0003] With the popularization of monitoring facilities in livestock farms and the development of computer vision technology, combined with deep learning technology, a method for temporal localization of abnormal behaviors in beef cattle has been constructed, making it possible to monitor abnormal behaviors of beef cattle around the clock. In recent years, animal behavior recognition methods based on deep learning technology and computer vision have developed rapidly. For example, Wang Shaohua et al. proposed a method for recognizing estrus behavior in dairy cows based on an improved YOLO v3 model in their paper "Research on Recognition of Estrus Behavior in Dairy Cows Based on an Improved YOLO v3 Model". This method introduces dense modules and uses a bounding box loss function that fuses center point distance with intersection-union ratio to achieve the recognition of mounting behavior in dairy cows in complex farming scenarios. For example, Wang et al. proposed a method for recognizing basic behaviors of dairy cows based on a three-dimensional convolutional neural network in their paper "E3D: An efficient 3D CNN for the recognition of dairy cow's basic motion behavior". This method extracts the spatiotemporal features of individual cow behavior videos by fusing deep separable convolutions and efficient channel modules, achieving the recognition of behaviors such as lying down, standing, walking, drinking, and eating. The above methods only involve cow behavior recognition; methods for temporal localization of cow behavior are still in the preliminary research stage. Temporal localization methods for cattle behavior not only need to predict which abnormal behaviors are included in the video, but also the start and end times of the behaviors. This is more in line with the needs of precise management of beef cattle in real-world farm scenarios, where accurate timing of behavior occurrences is crucial. For example, Ma et al., in their paper "Basic motion behavior recognition of single-dairy cow based on improved Rexnet 3D network," proposed a method based on an improved 3D convolutional neural network of Rexnet to recognize the lying, standing, and walking behaviors of a single cow. Based on this, they used a sliding window strategy to achieve temporal localization of the lying, standing, and walking behaviors of a single cow.

[0004] Existing methods for temporal localization of cattle behavior using a sliding window strategy only target a single target, resulting in poor accuracy and failing to meet the needs of large-scale beef cattle farming. Therefore, it is essential to design a method for temporal localization of abnormal beef cattle behavior within the field of view of farm monitoring videos to achieve high-precision temporal localization of abnormal beef cattle behavior. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a temporal localization method for abnormal behavior videos of beef cattle based on time offset loss, thereby solving the technical problem that existing temporal localization methods only target a single target and have poor accuracy.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A temporal localization method for abnormal behavior in beef cattle videos based on time offset loss, the method includes the following steps:

[0008] Step S1, construct a temporal localization dataset of abnormal behavior in beef cattle:

[0009] Collect videos of abnormal behaviors of beef cattle from the perspective of real farm monitoring. The abnormal behaviors of beef cattle include mounting behavior, running behavior and fighting behavior. Label the time sequence of abnormal behaviors in each video. Each video contains multiple time sequence tags of abnormal behaviors. Each time sequence tag of abnormal behavior includes the start time of behavior, the end time of behavior and the behavior category.

[0010] Step S2, extract features from videos of abnormal behavior in beef cattle:

[0011] The video of length L frames is sampled every 4 frames and input into the dual-branch spectral channel spatiotemporal aggregation and excitation model to extract the features of the video frames. Each video frame is represented as a feature vector of length 2048, and finally each video is represented as a feature vector of dimension L / 4×2048.

[0012] Step S3, implement the method for temporal localization of abnormal behavior in beef cattle:

[0013] A temporal localization model for abnormal behavior in beef cattle is constructed using a multi-head temporal localization model based on an aggregated scalable granularity pyramid. The model includes a backbone network, an aggregated scalable granularity perception pyramid, and a boundary localization multi-branch prediction head. The temporal localization method for abnormal behavior in beef cattle is implemented using this model.

[0014] Step S4: Based on the beef cattle abnormal behavior temporal localization dataset obtained in Step S1, train the beef cattle abnormal behavior temporal localization model obtained in Step S3 using the center point-start and end time offset loss function and the focus loss function.

[0015] Specifically, step S4 includes:

[0016] Step S401: Calculate the loss between the model's time-series predicted values ​​and the true values ​​using the center-start-end time offset loss function.

[0017]

[0018] In the formula:

[0019] IoU is the intersection-union ratio calculated between the predicted sequence and the true sequence;

[0020] c is the minimum bounding sequence length between the predicted sequence and the true sequence;

[0021] cp represents the center moment of the anomalous behavior predicted by the model;

[0022] cg represents the central moment of the actual abnormal behavior;

[0023] lp is the start time of the abnormal behavior predicted by the model;

[0024] lg represents the actual start time of the abnormal behavior;

[0025] rop represents the end time of the anomalous behavior predicted by the model;

[0026] rg represents the end time of the actual abnormal behavior.

[0027] Step S402: Calculate the classification loss L of the model using the focus loss function. C :

[0028] L C =-α t (1-p t ) γ log(p t )

[0029] In the formula:

[0030] α t Indicates the weight of different categories;

[0031] p t This represents the class probability predicted by the model;

[0032] γ represents the loss contribution used to suppress simple samples and promote the loss contribution of hard samples;

[0033] Step S403, L CSEDand L C The loss L is obtained by summing the results. The parameters of the beef cattle abnormal behavior temporal localization model are adjusted by backpropagation L until the number of iterations reaches the preset initial value and the training is completed.

[0034] Step S5, Temporal localization of abnormal behavior in beef cattle:

[0035] Input the real-world beef cattle monitoring video into the beef cattle abnormal behavior time-series localization model trained in step S4 to identify abnormal beef cattle behavior and locate the occurrence and end time of the abnormal behavior.

[0036] The present invention also has the following technical features:

[0037] Preferably, in step S1, the duration of the video covers 30 seconds, 60 seconds, 120 seconds, and 300 seconds.

[0038] Preferably, in step S3, the preset maximum length L of the input feature vector in the time-series localization model for abnormal behavior in beef cattle is... max The value is 2304, when L / 4 ≤ L max When, in the index interval [L / 4, L max [Padded with 0s; when L / 4 > L] max When, discard the index range [L] max The features of [L / 4] are used to obtain a feature vector with shape [2048, 2304].

[0039] Preferably, in step S3, the backbone network is composed of two encoded convolutional modules and one aggregated scalable granularity-aware module stacked together.

[0040] Preferably, in step S3, the aggregated scalable granularity-aware pyramid consists of 6 layers of aggregated scalable granularity-aware modules and layer normalization operations.

[0041] Preferably, in step S3, the boundary localization multi-branch prediction head includes a classification head, a start time prediction head, a center time prediction head, and an end time prediction head; the video features extracted by the aggregated scalable granularity-aware pyramid... Input the values ​​into the classification header, start time prediction header, center time prediction header, and end time prediction header respectively to complete the prediction of the abnormal behavior category, start time, center time, and end time.

[0042] Compared with the prior art, the present invention has the following technical effects:

[0043] (I) This invention proposes a loss function for calculating the offset of the occurrence, end, and center time of behavior—the center point-start-end time offset loss function—and constructs a multi-head temporal localization model based on an aggregated scalable granularity pyramid to achieve a temporal localization method for abnormal behavior in beef cattle videos based on time offset loss. This method constructs an aggregated scalable granularity perception multi-head abnormal behavior temporal localization model. First, video frame features are used as input and initially extracted through a backbone network. Then, features are fused through an aggregated scalable granularity perception pyramid to fully perceive the abnormal behavior features of beef cattle at different channels and scales. Next, a boundary localization multi-branch prediction head is used to predict the start time, end time, center time, and behavior category of the abnormal behavior. During model training, the center point-start-end time offset loss function is used to calculate the temporal localization loss, thereby achieving more accurate temporal boundary prediction of abnormal behavior. Finally, after training, the model achieves temporal localization of abnormal behavior in beef cattle in livestock pen monitoring videos.

[0044] (II) To address the problem that existing time-series localization methods only target a single target and have poor accuracy, this invention proposes a loss function for calculating the offset between the occurrence and end times of behavior—the center point-start and end time offset loss function. This constructs a time-series localization method for abnormal behavior videos of beef cattle based on time offset loss, enabling the time-series localization of abnormal behavior of beef cattle in monitoring videos, so as to meet the need to obtain the occurrence time of abnormal behavior in the precise management of beef cattle in real breeding scenarios. Attached Figure Description

[0045] Figure 1 This is a structural diagram of the time-series localization model of abnormal behavior in beef cattle.

[0046] Figure 2 This is a structural diagram of the aggregated scalable granularity-aware module.

[0047] Figure 3 This is a schematic diagram illustrating the calculation of the center point-start-end time offset loss.

[0048] Figure 4(a) is a comparison of the temporal location results of abnormal behavior of beef cattle in a 30-second daytime monitoring video using the method in this embodiment of the invention and other existing methods.

[0049] Figure 4(b) is a comparison of the temporal location results of abnormal behavior of beef cattle in a 60-second daytime monitoring video using the method in this embodiment of the invention and other existing methods.

[0050] Figure 4(c) is a comparison of the temporal location results of abnormal beef cattle behavior in a 120-second daytime monitoring video using the method in this embodiment of the invention and other existing methods.

[0051] Figure 4(d) is a comparison of the temporal location results of abnormal behavior of beef cattle in a 300-second daytime monitoring video using the method in this embodiment of the invention and other existing methods.

[0052] Figure 4(e) is a comparison of the temporal location results of abnormal behavior of beef cattle in a 30-second nighttime monitoring video using the method in this embodiment of the invention and other existing methods.

[0053] Figure 4(f) is a comparison of the temporal location results of abnormal behavior of beef cattle in a 60-second nighttime monitoring video using the method in this embodiment of the invention and other existing methods.

[0054] Figure 4(g) is a comparison of the temporal localization results of abnormal behavior of beef cattle in a 120-second nighttime monitoring video using the method in this embodiment of the invention and other existing methods.

[0055] Figure 4(h) is a comparison of the temporal location results of abnormal behavior of beef cattle in a 300-second nighttime monitoring video using the method in this embodiment of the invention and other existing methods.

[0056] The specific content of the present invention will be further explained in detail below with reference to the embodiments. Detailed Implementation

[0057] It should be noted that, unless otherwise specified, all algorithms in this invention are algorithms known in the prior art.

[0058] To meet the needs of monitoring and early warning of abnormal behavior in beef cattle pens during the breeding process, and addressing the limitations of existing cattle behavior temporal localization methods which can only target individual cattle and have poor accuracy, this invention proposes a video temporal localization method for abnormal beef cattle behavior based on time offset loss. This method includes:

[0059] First, a time-series dataset of abnormal beef cattle behavior was constructed, consisting of monitoring videos of cattle pens in a real-world farming environment. Each video segment contains multiple time-series labels, and each time-series label consists of the start time of the behavior, the end time of the behavior, and the behavior category.

[0060] Then, an aggregated scalable granularity-aware multi-head abnormal behavior temporal localization model is constructed to predict abnormal behaviors in beef cattle. This model includes a backbone network, a scalable granularity-aware pyramid, and boundary localization multi-branch heads. The backbone network consists of an encoding convolutional module and an aggregated scalable granularity convolutional module, used to extract video features and adjust the weights of different channels, which are then input into the aggregated scalable granularity-aware pyramid. The aggregated scalable granularity-aware pyramid extracts and fuses abnormal behavior features of different durations. The multi-branch prediction heads utilize the features of each layer of the aggregated scalable granularity-aware pyramid to predict the start time, end time, and category of the behavior.

[0061] Finally, the model training uses the center point-start-end time offset loss function to calculate the loss of the true label and the predicted label, so as to obtain more accurate temporal boundary prediction results.

[0062] This invention can aggregate more significant abnormal behavior features in videos and perform more detailed loss calculations during model training, resulting in more accurate temporal boundary predictions of abnormal behavior in beef cattle.

[0063] Following the above technical solutions, specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent modifications made based on the technical solutions of this application fall within the protection scope of the present invention.

[0064] Example:

[0065] This embodiment presents a method for temporal localization of abnormal behavior in beef cattle videos based on time offset loss, including the following steps:

[0066] Step S1, construct a temporal localization dataset of abnormal behavior in beef cattle:

[0067] We collected videos of abnormal behaviors in beef cattle from the perspective of real farm monitoring. These abnormal behaviors included mounting, running, and fighting. We used manual observation to label the time sequence of abnormal behaviors in each video. Each video contained multiple time sequence tags for abnormal behaviors, and each tag included the start time, end time, and category of the behavior.

[0068] Step S1: The video duration covers 30 seconds, 60 seconds, 120 seconds, and 300 seconds.

[0069] Specifically, this embodiment collects monitoring videos of cattle pens under real-world farming conditions, with durations of 30 seconds, 60 seconds, 120 seconds, and 300 seconds. Each video segment is manually labeled with a time-series tag, including the start and end times of the behavior and the behavior category. This beef cattle abnormal behavior time-series dataset includes 1337 videos, with 1067 in the training set and 270 in the test set. There are 534 videos of 30 seconds, 366 videos of 60 seconds, 313 videos of 120 seconds, and 124 videos of 300 seconds. It contains a total of 956 time-series tags for mounting behavior, 596 for running behavior, and 434 for fighting behavior. Among these, 923 videos have one time-series tag, 270 videos have two, 96 videos have three, and 48 videos have four or more, with an average of 1.49 time-series tags per video.

[0070] Step S2, extract features from videos of abnormal behavior in beef cattle:

[0071] The video of length L frames is sampled every 4 frames and input into the dual-branch spectral channel spatiotemporal aggregation and excitation model to extract the features of the video frames. Each video frame is represented as a feature vector of length 2048, and finally each video is represented as a feature vector of dimension L / 4×2048.

[0072] In this specific embodiment, a video of length L frames containing M time-series tags is input. Video frames are extracted at intervals of 4. Utilizing the spatiotemporal features of the dual-branch spectral channel spatiotemporal aggregation and excitation model, each video frame is represented as a 1×2048 feature vector. Finally, the video is represented by a feature vector X0 of size L / 4×2048. The M time-series tags of this video can be represented as...

[0073] Step S3, implement the method for temporal localization of abnormal behavior in beef cattle:

[0074] A temporal localization model for abnormal behavior in beef cattle is constructed using a multi-head temporal localization model based on an aggregated scalable granularity pyramid. The model includes a backbone network, an aggregated scalable granularity perception pyramid, and a boundary localization multi-branch prediction head. The temporal localization method for abnormal behavior in beef cattle is implemented using this model.

[0075] In this specific embodiment, in the time-series localization model for abnormal behavior in beef cattle, the preset maximum length L of the input feature vector is... max The value is 2304, when L / 4 ≤ L max When the eigenvector X0 is in the index interval [L / 4, L... max Fill the position with 0, when L / 4 > L max When, discard the index range [L] max The features of [L / 4] are used to obtain a feature vector X1 with a size of 2304×2048. For example... Figure 1 As shown, a method for locating abnormal behaviors in beef cattle is implemented by constructing a multi-head abnormal behavior temporal localization model based on scalable granularity awareness. Feature vector X1 is extracted into feature vector X2 with size 2304×512 through the backbone network. Feature vector X2 is then aggregated into a scalable granularity awareness pyramid with size 2304 / 2 through a 6-layer structure to obtain a feature vector X2 with size 2304 / 2. i-1 The eigenvector of ×512 (i∈[1,6]) For each feature vector Inputting the data into the boundary localization multi-branch prediction head yields temporal localization prediction results at different scales.

[0076] Specifically, the method for constructing the temporal localization model of abnormal behavior in beef cattle is as follows:

[0077] Step S301, construct the backbone network:

[0078] The backbone network mainly consists of two encoding convolutional modules and one aggregation scalable granularity-aware module stacked together. The input feature vector X1 with a size of 2304×2048 is assigned different channel attention weights by the backbone network to highlight effective features and weaken ineffective features, resulting in a feature vector X2 with a size of 2304×512.

[0079] In this embodiment, the specific structure of the aggregated scalable granularity awareness module is as follows: Figure 2 As shown. The aggregated scalable granular perception module mainly consists of three branches: distance branch, window branch, and global perception branch. The goal of the distance branch is to enhance the difference between abnormal and non-abnormal behaviors; the goal of the window branch is to introduce semantic information with a longer span; and the goal of the global perception branch is to capture global features of the video sequence. Finally, the features extracted from the three branches are aggregated to achieve the identification and temporal localization of abnormal behaviors in beef cattle. The calculation process of the aggregated scalable granular perception module is as follows:

[0080] f ASGP =PReLU(FC(AvgPool(FC(x))))+Conv w (x)(Conv w (x)+Conv kw (x))+Conv(x)

[0081] In the formula:

[0082] PReLU is the activation function;

[0083] FC stands for fully connected layer;

[0084] AvgPool is a global equal pooling pool;

[0085] Conv w A convolutional layer of depth with a window size of w;

[0086] k is a scalability factor;

[0087] Conv is a depthwise convolution operation.

[0088] Step S302, construct the aggregated scalable granularity-aware pyramid:

[0089] The Aggregated Scalable Granularity-Aware Pyramid consists of 6 layers of Aggregated Scalable Granularity-Aware modules and layer normalization operations. Features X2 extracted from the backbone network are input to... Figure 1 The aggregated scalable granularity-aware pyramid shown has an output size of 2304 / 2. i-1 The eigenvector of ×512 (i∈[1,6]) The multi-branch prediction head for boundary localization is used to predict the results, thereby enhancing the model's ability to predict abnormal behaviors of different durations.

[0090] Step S303, construct the boundary localization multi-branch prediction head:

[0091] The boundary localization multi-branch prediction head mainly consists of a classification head, a start-time prediction head, a center-time prediction head, and an end-time prediction head. Video features are extracted through aggregation and scalable granularity-aware pyramid. Input the values ​​into the classification header, start time prediction header, center time prediction header, and end time prediction header respectively to complete the prediction of the start time, center time, end time, and abnormal behavior category.

[0092] Specifically, each prediction head consists of three coded convolutional layers, and to reduce the computational cost of the model, the prediction heads used to predict features at each level of the aggregated scalable granularity-aware pyramid share weights.

[0093] Step S4: Based on the beef cattle abnormal behavior temporal localization dataset obtained in Step S1, train the beef cattle abnormal behavior temporal localization model obtained in Step 3 using the center point-start-end time offset loss function and the focus loss function.

[0094] Step S4 includes:

[0095] Step S401, as follows Figure 3 As shown, the Center-Start-EndDistance Loss (CSED Loss) function is used to calculate the time-series predicted values ​​and the true values ​​of the model:

[0096]

[0097] In the formula:

[0098] IoU is the intersection-union ratio calculated between the predicted sequence and the true sequence;

[0099] c is the minimum bounding sequence length between the predicted sequence and the true sequence;

[0100] cp represents the center moment of the anomalous behavior predicted by the model;

[0101] cg represents the central moment of the actual abnormal behavior;

[0102] lp is the start time of the abnormal behavior predicted by the model;

[0103] lg represents the actual start time of the abnormal behavior;

[0104] rp is the end time of the abnormal behavior predicted by the model;

[0105] rg represents the end time of the actual abnormal behavior.

[0106] Step S402: Calculate the classification loss L of the model using the Focal Loss function. C :

[0107] L C =-α t (1-p t ) γ log(p t )

[0108] In the formula:

[0109] α t Indicates the weight of different categories;

[0110] p t This represents the class probability predicted by the model;

[0111] γ represents the loss contribution used to suppress simple samples and promote the loss contribution of hard samples.

[0112] Step S403, L CSsD and L C The loss L is obtained by summing the results. The parameters of the beef cattle abnormal behavior temporal localization model are adjusted by backpropagation L until the number of iterations reaches the preset initial value and the training is completed.

[0113] Step S5, Temporal localization of abnormal behavior in beef cattle:

[0114] Input the real-world beef cattle monitoring video into the beef cattle abnormal behavior time-series localization model trained in step S4 to identify abnormal beef cattle behavior and locate the occurrence and end time of the abnormal behavior.

[0115] Specifically, in this embodiment, given a video of abnormal behavior in beef cattle, it is input into a multi-head temporal localization model based on an aggregated scalable granular pyramid. The model outputs temporal prediction labels for the abnormal behavior. The evaluation metrics are the mean average accuracy (mAP) at different intersection-over-union (IoU) thresholds (0.3-0.7) and the overall mean average accuracy (Average mAP).

[0116] Comparative Example 1:

[0117] This comparative example compares the proposed method with three state-of-the-art behavioral temporal localization methods: TriDet (three-branch behavioral localization method), TemporalMaxer (temporal maximization method), and ActionFormer (behavioral modeling method).

[0118] Experimental Verification: The results of this comparative example are shown in Table 1. The embodiments of the present invention achieved mean average accuracy of 93.93%, 93.73%, 93.22%, 92.29%, and 87.46% and an overall mean average accuracy of 92.13% at cross-union ratio (CUN) thresholds of 0.3-0.7, respectively. The results show that compared to TriDet, this method improves the mean average accuracy by 3.15%, 3.13%, 3.61%, 5.18%, and 4.24% at CUN thresholds of 0.3-0.7, respectively, with an overall improvement of 3.87%. The accuracy improvement is more significant at CUN thresholds of 0.6 and 0.7, indicating that the time series of abnormal behaviors in beef cattle predicted by this method are more accurate. Green represents mounting behavior, blue represents fighting behavior, orange represents running behavior, and the length of the color block represents the length of the abnormal behavior segment; for example... Figures 4(a) to 4(h) Using visualization methods, the predicted time-series labels of abnormal behaviors from various methods for daytime and nighttime monitoring videos of beef cattle behavior (30 seconds, 60 seconds, 120 seconds, and 300 seconds) were compared with the actual values. As shown in Figure 4(a), this video is 30 seconds long and contains two actual time-series labels: the first label indicates a stepping behavior occurring between 0 and 7 seconds, and the second label indicates a stepping behavior occurring between 16 and 25 seconds. The method in this embodiment predicted stepping behavior occurring between 0.32 and 7.56 seconds, and between 16.06 and 24.98 seconds. The most competitive method, TriDet, predicted stepping behavior occurring between 0.30 and 8.84 seconds, and between 15.85 and 25.13 seconds. The results show that the time-series labels of abnormal behaviors predicted by the method in this embodiment are closer to the actual values.

[0119] Table 1. Abnormal Behaviors and Temporal Localization Results of Beef Cattle Using Different Methods

[0120]

[0121] Comparative Example 2:

[0122] This comparative example presents three different methods to verify the effectiveness of the aggregated scalable granularity-aware module and the center-to-start-to-end time offset loss function in the embodiments of the present invention. Method 1 in this comparative example uses the scalable granularity-aware module in step S302 and trains the model using the distance intersection-union loss function in step S304; all other steps are the same as in the embodiments. Method 2 uses the aggregated scalable granularity-aware module in step S302 and trains the model using the distance intersection-union loss function in step S304; all other steps are the same as in the embodiments. Method 3 uses the scalable granularity-aware module in step S302 and trains the model using the center-to-start-to-end time offset loss function in step S304; all other steps are the same as in the embodiments.

[0123] Experimental Verification: The results of this comparative example are shown in Table 2. Overall, the average accuracy of the embodiments of the present invention is improved under different cross-union ratio (CUI) thresholds. In this comparative example, Method 1 serves as the baseline model, with an average average accuracy of 88.26%. Method 2 improves the average average accuracy by 3.53% compared to Method 1, verifying the effectiveness of the aggregated scalable granularity-aware module. Method 3 improves the average average accuracy by 3.31% compared to Method 1, verifying the effectiveness of the center point-start / end time offset loss function. Finally, the embodiment of the present invention that simultaneously uses the aggregated scalable granularity-aware module and the center point-start / end time offset loss function achieves the best results, with an average average accuracy improvement of 3.87% compared to Method 1.

[0124] Table 2. Abnormal Behaviors and Temporal Localization Results of Beef Cattle Using Different Methods

[0125]

[0126] Comparative Example 3:

[0127] This comparative example presents the results of temporal localization of abnormal behavior in beef cattle under different video acquisition times during the day and night, in accordance with the embodiments of the present invention, to verify that the method in this case can meet the needs of farms to achieve all-weather monitoring of abnormal behavior in beef cattle.

[0128] Experimental verification: The results of this comparative example are shown in Table 3. The embodiments of this invention achieved temporal localization of abnormal behavior in daytime beef cattle behavior videos collected under an intersection-union (IU) threshold of 0.3-0.7, with an average mean accuracy of 96.63%. Although the recognition results decreased for beef cattle behavior videos collected at night, the average mean accuracy still reached 79.68%, meeting the needs of nighttime monitoring in livestock farms. The experimental results demonstrate that the embodiments of this invention can meet the needs of all-weather monitoring of abnormal behavior in beef cattle pens in livestock farms.

[0129] Table 3. Abnormal Behavior and Temporal Localization Results of Beef Cattle at Different Video Acquisition Times

[0130]

[0131] Comparative Example 4:

[0132] This comparative example presents the results of temporal localization of abnormal beef cattle behavior in beef cattle behavior videos of different durations of 30 seconds, 60 seconds, 120 seconds, and 300 seconds, in accordance with the embodiments of the present invention, to verify that the method in this case can meet the needs of farms for long-term monitoring of abnormal beef cattle behavior.

[0133] Experimental Verification: The results of this comparative example are shown in Table 4. For videos of different durations (30 seconds, 60 seconds, 120 seconds, and 300 seconds), the average mean accuracy of the embodiments of the present invention was 90.80%, 96.87%, 91.39%, and 78.64%, respectively. The embodiments of the present invention performed best with a video length of 60 seconds and worst with a video length of 300 seconds, but the average mean accuracy was still greater than 75%. This comparative example demonstrates that the embodiments of the present invention have a certain robustness in the temporal localization of abnormal behavior in beef cattle and can meet the needs of monitoring abnormal behavior in beef cattle pens in livestock farms.

[0134] Table 4. Abnormal Beef Cattle Behavior and Temporal Localization Results for Different Video Lengths

[0135]

[0136] Comparative Example 5:

[0137] This comparative example presents the identification results of the present invention under time-series labels of abnormal behavior of beef cattle containing 1, 2, 3, 4 or more different numbers in a single video, in order to verify the robustness and practicality of the method in this case.

[0138] Experimental verification: The results of this comparative example are shown in Table 5. The recognition results of the embodiment of the present invention with 1, 2, and 3 time tags are 91.90%, 94.70%, and 92.32%, respectively. In videos with 4 or more time tags, the results of abnormal behavior and time localization decrease, with an average accuracy of 84.92%. Although the accuracy of the embodiment of the present invention decreases as the number of time tags increases, based on the analysis of the dataset constructed in step S1, the number of time tags in real aquaculture scenarios is relatively small, and the embodiment of the present invention can meet the actual needs.

[0139] Table 5. Abnormal Beef Cattle Behavior and Temporal Localization Results in Videos with Different Numbers of Temporal Tags

[0140]

[0141] Comparative Example 6:

[0142] This comparative example presents the identification and temporal localization results of different abnormal behaviors of beef cattle, such as running, mounting, and fighting, in accordance with the embodiments of the present invention, to verify that the method in this case can be applied to a variety of abnormal behaviors of beef cattle.

[0143] Experimental Verification: The results of this comparative example are shown in Table 6. This invention's embodiments identify and temporally locate abnormal behaviors of beef cattle in monitoring videos, such as running, mounting, and fighting. The accuracy of each embodiment in identifying different behaviors is evaluated. The results show that this invention's embodiments can achieve the identification and temporal localization of various abnormal behaviors of beef cattle in real-world farming scenarios. Specifically, the embodiments of this invention achieve the highest accuracy in identifying and temporally locating mounting behavior, with an average mean accuracy of 96.51% at an intersection-over-union (IoU) threshold of 0.3-0.7. This is followed by running behavior identification and temporal localization, with an average mean accuracy of 91.25%, and finally fighting behavior identification and temporal localization, with an average mean accuracy of 88.62%.

[0144] Table 6. Results of Localization of Different Abnormal Behaviors in Beef Cattle

[0145]

Claims

1. A method for temporal localization of abnormal behavior in beef cattle based on time offset loss, characterized in that, The method includes the following steps: Step S1, construct a temporal localization dataset of abnormal behavior in beef cattle: Collect videos of abnormal behaviors of beef cattle from the perspective of real farm monitoring. The abnormal behaviors of beef cattle include mounting behavior, running behavior and fighting behavior. Label the time sequence of abnormal behaviors in each video. Each video contains multiple abnormal behavior time sequence tags. Each abnormal behavior time sequence tag includes the start time of the behavior, the end time of the behavior and the behavior category. Step S2, extract features from videos of abnormal behavior in beef cattle: The length is L The video is sampled every 4 frames and input into a dual-branch spectral channel spatiotemporal aggregation and excitation model to extract features from the video frames. Each video frame is represented as a feature vector of length 2048. Finally, each video is represented as a feature vector with a dimension of... eigenvectors; Step S3, implement the method for temporal localization of abnormal behavior in beef cattle: A temporal localization model for abnormal behavior in beef cattle is constructed using a multi-head temporal localization model based on an aggregated scalable granularity pyramid. The model includes a backbone network, an aggregated scalable granularity perception pyramid, and a boundary localization multi-branch prediction head. The method for temporal localization of abnormal behavior in beef cattle is implemented using this model. In step S3, the backbone network is composed of two encoded convolutional modules and one aggregated scalable granularity-aware module stacked together. In step S3, the aggregated scalable granularity-aware pyramid consists of 6 layers of aggregated scalable granularity-aware modules and layer normalization operations. The aforementioned aggregated scalable granular perception module mainly comprises three branches: distance branch, window branch, and global perception branch. The goal of the distance branch is to enhance the difference between abnormal and non-abnormal behaviors. The goal of the window branch is to introduce semantic information with a longer span. The goal of the global perception branch is to capture global features of the video sequence. Finally, the features extracted from the three branches are aggregated to achieve the identification and temporal localization of abnormal behaviors in beef cattle. Step S4: Based on the beef cattle abnormal behavior time-series localization dataset obtained in Step S1, train the beef cattle abnormal behavior time-series localization model obtained in Step 3 using the center point-start-end time offset loss function and the focus loss function. Step S4 includes: Step S401: Calculate the loss between the model's time-series predicted values ​​and the true values ​​using the center-start-end time offset loss function. In the formula: The intersection-union ratio (IUU) is calculated for the predicted sequence and the true sequence. The minimum bounding sequence length between the predicted and true sequences; The central moment of the anomalous behavior predicted by the model; This is the central moment of the actual abnormal behavior; The starting time of the abnormal behavior predicted by the model; This represents the starting point of the actual abnormal behavior; This represents the end time of the abnormal behavior predicted by the model. This is the moment when the actual abnormal behavior ends; Step S402: Calculate the classification loss of the model using the focus loss function. : In the formula: Indicates the weight of different categories; This represents the class probability predicted by the model; This indicates the loss contribution used to suppress simple samples and promote the loss contribution of hard samples; Step S403, will and Summing up results in loss Through backpropagation Adjust the parameters of the time-series localization model for abnormal behavior in beef cattle until the number of iterations reaches the preset initial value to complete the training; Step S5, Temporal localization of abnormal behavior in beef cattle: Input the real-world beef cattle monitoring video into the beef cattle abnormal behavior time-series localization model trained in step S4 to identify abnormal beef cattle behavior and locate the occurrence and end time of the abnormal behavior.

2. The method for temporal localization of abnormal behavior in beef cattle based on time offset loss as described in claim 1, characterized in that, In step S1, the duration of the video covers 30 seconds, 60 seconds, 120 seconds, and 300 seconds.

3. The method for temporal localization of abnormal behavior in beef cattle based on time offset loss as described in claim 1, characterized in that, In step S3, the preset maximum length of the input feature vector in the time-series localization model for abnormal behavior in beef cattle is... For 2304, when At that time, in the subscript range Position is filled with 0; when When, discard the subscript range. The characteristics, thereby obtaining a shape as eigenvectors.

4. The method for temporal localization of abnormal behavior in beef cattle based on time offset loss as described in claim 1, characterized in that, In step S3, the boundary localization multi-branch prediction head includes a classification head, a start time prediction head, a center time prediction head, and an end time prediction head; the video features extracted by the aggregated scalable granularity-aware pyramid are... Input the values ​​into the classification header, start time prediction header, center time prediction header, and end time prediction header respectively to complete the prediction of the abnormal behavior category, start time, center time, and end time.

Images