A cow health evaluation method based on a Transformer light-weight model

Abstract

The present application relates to the technical field of cattle health assessment, and discloses a cattle health assessment method based on a light-weighted Transformer model, comprising the following steps: step 1, synchronously collecting a visible light video stream and an infrared thermal imaging data stream of a target cattle, and processing the visible light video stream to identify, track and obtain a continuous posture skeleton point coordinate sequence of the target cattle; step 2, based on the continuous posture skeleton point coordinate sequence, the visible light video stream and the infrared thermal imaging data stream, extracting and generating a posture feature sequence, a visual feature sequence and a temperature feature sequence. The evaluation technical scheme based on multi-modal data fusion and automatic analysis achieves the technical effect of all-weather, standardized and uninterrupted monitoring of the health status of the cattle, and compared with the technical scheme of relying on manual experience and visual assessment in the prior art, the present application solves the problems of strong subjectivity of the evaluation results and inability to timely capture subtle health changes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cattle health assessment, in particular to a cattle health assessment method based on a light-weight Transformer model. BACKGROUND

[0002] The safety of milk has always been a problem for milk consumers, and how to promote milk food safety from the source is a major issue for scientific researchers. The safety of milk is closely related to the health of dairy cows. Changes in physiological indicators of dairy cows are an important basis for timely detection of diseases and timely prevention. The traditional way to understand the health status of dairy cows is for the breeder to periodically patrol the barn and observe the dairy cow's appearance, feeding, urine, body temperature, milk yield, nasal mirror and other clinical indicators to determine whether the dairy cow is abnormal.

[0003] For example, the Chinese invention application with publication number CN107485412A discloses a cattle health monitoring system and method, which belongs to the technical field of Internet of Things. The monitoring system and method propose to install a collection device at the milking passage and the carousel type milking hall to cooperatively detect and obtain various physical data of the cattle such as body temperature, back texture, abdominal circumference, height at different positions of the back, shoulder height, gait, etc. based on two modes of motion and stillness, so as to evaluate the health status of the cattle according to the obtained data.

[0004] The above patent has the following disadvantages:

[0005] The cattle health assessment method relying on manual inspection in the prior art consumes a large amount of manpower, and the evaluation results completely depend on personal experience, which is highly subjective and lacks uniform objective standards. Therefore, it is not easy to make timely and accurate judgments on the subtle changes in the health status of the cattle, often leading to missed opportunities for early intervention of diseases and causing unnecessary economic losses.

[0006] The prior art scheme of analyzing pasture video data by using a cloud server has high requirements for the local network bandwidth of the pasture due to the continuous uploading of massive video data, resulting in high data transmission and cloud computing costs. Meanwhile, the inherent delay in the process of uploading data from the local to the remote server seriously damages the real-time performance of health assessment, and cannot meet the immediate response needs of the pasture to sudden conditions.

[0007] The current algorithm model applied to cattle behavior recognition has poor performance and recognition stability in the actual environment where the cattle in the cowshed are mutually blocked, the lighting conditions are complex and changeable, and the monitoring shooting distances are different. Moreover, the model itself cannot effectively extract weak behavior features that can indicate the deterioration of the health status from the visual information full of interference, resulting in a serious lack of early health risk warning ability for the cattle.

[0008] To address these issues, this invention proposes a cattle health assessment method based on a lightweight Transformer model. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a method for assessing bovine health based on a lightweight Transformer model, thereby resolving the problems mentioned in the background section.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a method for assessing bovine health based on a lightweight Transformer model, comprising the following steps:

[0011] Step 1: Simultaneously acquire visible light video stream and infrared thermal imaging data stream of the target cattle, and process the visible light video stream to identify, track and obtain the continuous posture skeleton point coordinate sequence of the target cattle;

[0012] Step 2: Based on the continuous attitude skeleton point coordinate sequence, the visible light video stream, and the infrared thermal imaging data stream, extract and generate attitude feature sequence, visual feature sequence, and temperature feature sequence;

[0013] Step 3: Perform feature vectorization processing on the posture feature sequence, the visual feature sequence, and the temperature feature sequence, and inject temporal location information and modality type information to construct a multimodal fusion input feature sequence;

[0014] Step 4: Input the multimodal fusion input feature sequence into a preset lightweight Transformer model for processing. The lightweight Transformer model has a built-in dynamic attention switching mechanism. The dynamic attention switching mechanism uses local attention calculation in normal state, and switches to sparse global attention calculation when a preset key event is detected to generate an enhanced feature sequence.

[0015] Step 5: Decode the enhanced feature sequence, calculate the health status index of the target cattle, and output a health assessment conclusion based on the comparison result of the health status index with the preset health baseline.

[0016] Preferably, in step 1, processing the visible light video stream to identify, track, and obtain the continuous posture skeleton point coordinate sequence of the target cow specifically includes: using a target detection algorithm to identify the cow in the image, and using a multi-target tracking algorithm to assign a unique identity ID to the target cow to achieve continuous tracking.

[0017] Preferably, in step 2, extracting and generating the temperature feature sequence specifically includes: performing temperature sampling in the infrared thermal imaging data stream synchronized with time on the core area of ​​the cow's body determined by the continuous posture skeleton point coordinate sequence, and using the sampled values ​​as temperature features.

[0018] Preferably, in step 5, the health status indicators include at least: an activity frequency index calculated based on the frequency of the target cattle's dynamic behavior, a standing-to-lying ratio calculated by statistically analyzing the proportion of standing and lying postures during the time period, and a body temperature fluctuation trend obtained by trend analysis of the temperature feature sequence.

[0019] Preferably, in step 5, the output of the health assessment conclusion specifically means: when the health status indicator continuously deviates from the normal range of the preset health baseline, the system automatically triggers an alarm mechanism and pushes a notification containing the identity of the target cattle. Warning information related to abnormal indicators.

[0020] Preferably, step 1 further includes:

[0021] Sub-step 1.1: Synchronous acquisition and timestamp annotation of multi-source data

[0022] Visible light cameras and infrared thermal imaging cameras are deployed to synchronously acquire visible light video streams of the target cattle at a fixed frame rate. and infrared thermal imaging data stream ,

[0023] in, For time step index;

[0024] For each frame and Mark the same timestamp Forming time-aligned cross-modal data pairs ,

[0025] in, It is a visible light video stream. The first in the visible light video stream time step Image frame, For infrared thermal imaging data stream, The first in the infrared thermal imaging data stream Thermal imaging image frames at time steps For the first The timestamp of the time step For time-aligned cross-modal data pairs;

[0026] Sub-step 1.2: Target cattle detection and identification tracking

[0027] For the visible light video stream Each frame in The target detection algorithm outputs a set of detection boxes. ,

[0028] in, For the first The coordinates and confidence level of the bounding box. For the first The set of detection boxes at each time step. For the first Time step Parameters of the bit detection box, and The coordinates of the top-left corner pixel of the detection box. and The coordinates of the bottom right corner of the detection box. To test the confidence level;

[0029] Multi-target tracking algorithm for consecutive frames Perform cross-frame association to assign a unique identity to the detected cattle. Generate a set of tracking trajectories ,

[0030] in, For the first A sequence of consecutive detection frames for each cow. For tracking the set of trajectories, For the first The first cow was only in the first Parameters of the detection box at each time step;

[0031] Sub-step 1.3: Extraction of continuous attitude skeleton point coordinates

[0032] For the set of tracking trajectories Each target in At each time step detection box Inside, a pre-trained cattle pose estimation model is used. Extracting the key skeleton point coordinate sequence ,

[0033] in, For the first The coordinates of the skeleton points of the time step. For the first The sequence of coordinates of the skeleton of a cow;

[0034] Will With timestamp Binding, forming a time-series skeleton point coordinate sequence:

[0035] , For the first A time-series skeletal point coordinate sequence for a single cow.

[0036] Preferably, step 2 further includes:

[0037] Sub-step 2.1: Generation of pose feature sequences

[0038] Based on the continuous attitude skeleton point coordinate sequence Calculate time steps Skeletal point motion velocity characteristics The expression is:

[0039] ,

[0040] in, For time step interval, For the first The coordinates of the skeleton points of the time step. For the first The coordinates of the skeleton points of the time step. For the first The timestamp of the time step For the first The timestamp of the time step;

[0041] Will and Concatenate to form a pose feature vector ;

[0042] Sub-step 2.2: Visual feature sequence generation

[0043] For the visible light video stream with timestamp Aligned cattle image regions Input to a lightweight convolutional neural network Perform feature extraction to generate visual feature vectors. The expression is:

[0044] ,

[0045] in, For visual feature vectors, Visible light video stream The cropped area of ​​the cow image. For lightweight convolutional neural networks;

[0046] Sub-step 2.3: Generation of temperature feature sequences

[0047] Based on the continuous attitude skeleton point coordinate sequence In timestamp Corresponding infrared thermal imaging data In the middle, the spinal skeleton point Generate a rectangular sampling area centered on the target region:

[0048] ,

[0049] in, The preset sampling radius;

[0050] calculate Average temperature of all pixels Forming a temperature feature vector .

[0051] Preferably, step 3 further includes:

[0052] Sub-step 3.1: Multimodal Feature Vectorization Mapping

[0053] The pose feature sequence Visual feature sequences and temperature feature sequence Dimensionality reduction is performed using separate linear projection layers:

[0054] ,

[0055] ,

[0056] ,

[0057] in, For the first The pose feature vector at each time step. For the first Visual feature vectors at time steps For the first Temperature feature vector at time step , , For trainable projection matrices, , , For bias terms;

[0058] Sub-step 3.2: Timing Position Encoding Injection

[0059] For each time step Projection features , , Injecting sinusoidal absolute position code The calculation formula is:

[0060] ,

[0061] ,

[0062] in, For sinusoidal position coding components, For cosine position coding components, For dimensional indexing, For the first The timestamp of the time step;

[0063] Add the positional encoding to the projected features:

[0064] ,

[0065] ,

[0066] ,

[0067] in, To enhance features by fusing attitude features with temporal location information, To enhance features by fusing visual features with temporal and location information, Enhanced features for temperature characteristics and temporal location information are used for fusion.

[0068] Sub-step 3.3: Modality type embedding and fusion

[0069] Enhancing features for the location encoding , , Add learnable modality embeddings Generate the final multimodal fusion input features:

[0070] ,

[0071] ,

[0072] ,

[0073] The three components are concatenated according to time steps to construct a complete input feature sequence. :

[0074] ,

[0075] in, This is the final attitude mode feature vector. This is the final visual modality feature vector. This represents the final temperature mode eigenvector.

[0076] Preferably, step 4 further includes:

[0077] Sub-step 4.1: Local attention calculation

[0078] For the multimodal fusion input feature sequence Under normal conditions, each encoder layer of the lightweight Transformer model processes each feature vector. Perform local attention calculations, limiting the attention scope to a window. ,in, Set the default window size;

[0079] Local attention weights The calculation formula is:

[0080] ,when ,

[0081] in, , , For a trainable parameter matrix, For feature dimension, For the first Feature vector For the Feature vector The level of attention, For the first Feature vector linear projection, for Feature vector Linear projection;

[0082] Sub-step 4.2: Critical event detection and attention mode switching

[0083] Real-time monitoring of the multimodal fusion input feature sequence Attention mode switching is triggered when any of the following key events are detected:

[0084] Drastic attitude change events: Calculating the velocity characteristics of skeleton points in continuous time steps of Norm change ,like This was determined to be a drastic change in posture.

[0085] in, and For the first A cow only in time steps and The speed of the posture movement;

[0086] Temperature anomaly events: Calculate current temperature characteristics With sliding window mean deviation ,like ,in, For preset multiples, The standard deviation of the temperature within the window is used to determine a temperature anomaly.

[0087] Sub-step 4.3: Sparse Global Attention Calculation

[0088] When a critical event is detected in sub-step 4.2, the time step of the event is set. All corresponding modal feature vectors Expanding the attention scope to the global level, we calculate the global attention weights:

[0089] ,

[0090] Output features ,in, , For a trainable parameter matrix, For the first eigenvector pair The degree of global attention on feature vectors.

[0091] Preferably, step 5 further includes:

[0092] Sub-step 5.1: Health Indicator Decoding and Calculation

[0093] The enhanced feature sequence Input to multilayer perceptron decoder Generate predicted values ​​for health indicators:

[0094] ,in, ,

[0095] in, This is an activity frequency index. The ratio of standing to lying down is given. For the trend of body temperature fluctuation, Corresponding time step Temperature mode enhancement features;

[0096] Sub-step 5.2: Sliding window health baseline calculation

[0097] For each health indicator Calculate within the sliding time window Baseline mean and standard deviation within:

[0098] , ,

[0099] in, For window length, The mean of the healthy baseline, Standard deviation of the healthy baseline;

[0100] Sub-step 5.3: Anomaly Detection and Early Warning Trigger

[0101] When any health indicator Satisfying the continuous deviation condition:

[0102] continuous The test was successful.

[0103] in, The deviation coefficient, The duration threshold;

[0104] Trigger the alarm mechanism and generate a generator containing the identity of the target cattle. Abnormal indicator types Warning information on the degree of deviation ,

[0105] in, .

[0106] This invention provides a method for assessing bovine health based on a lightweight Transformer model. It offers the following advantages:

[0107] 1. This invention adopts an assessment technology based on multimodal data fusion and automated analysis to achieve the technical effect of all-weather, standardized and uninterrupted monitoring of cattle health status. Compared with the existing technology that relies on manual experience and visual assessment, it solves the shortcomings of strong subjectivity in assessment results and inability to capture subtle health changes in a timely manner.

[0108] 2. This invention adopts a technical solution of deploying a lightweight Transformer model and a dynamic attention switching mechanism at the edge, so as to achieve the technical effect of completing real-time data processing and analysis on local devices. Compared with the existing technical solution that relies on cloud servers for video analysis, it solves the shortcomings of high network bandwidth requirements, large data transmission latency and high cloud computing costs.

[0109] 3. The present invention adopts a technical solution of multimodal feature complementarity and gated sparse attention mechanism to achieve the technical effect of stable extraction of key health features under interference conditions such as occlusion and lighting changes. Compared with the existing single visual model technical solution, it solves the shortcomings of the model being susceptible to environmental interference and not easy to capture weak abnormal behavior signals. Attached Figure Description

[0110] Figure 1This is a flowchart of the present invention. Detailed Implementation

[0111] To enable those skilled in the art to understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the scope of protection of the present invention.

[0112] The present invention will now be described in detail with reference to the accompanying drawings:

[0113] Example:

[0114] Please see the appendix Figure 1 This invention provides a method for assessing bovine health based on a lightweight Transformer model, comprising the following steps:

[0115] Step 1: Simultaneously acquire visible light video stream and infrared thermal imaging data stream of the target cattle, and process the visible light video stream to identify, track and obtain the continuous posture skeleton point coordinate sequence of the target cattle;

[0116] In step 1, the visible light video stream is processed to identify, track and obtain the continuous posture skeleton point coordinate sequence of the target cow. Specifically, this includes: using a target detection algorithm to identify the cow in the picture, and using a multi-target tracking algorithm to assign a unique identity ID to the target cow to achieve continuous tracking.

[0117] Sub-step 1.1: Synchronous acquisition and timestamp annotation of multi-source data

[0118] Visible light cameras and infrared thermal imaging cameras are deployed to synchronously acquire visible light video streams of the target cattle at a fixed frame rate. and infrared thermal imaging data stream ,

[0119] in, For time step index;

[0120] For each frame and Mark the same timestamp Forming time-aligned cross-modal data pairs ,

[0121] in, It is a visible light video stream. The first in the visible light video stream time step Image frame, For infrared thermal imaging data stream, The first in the infrared thermal imaging data stream Thermal imaging image frames at time steps For the first The timestamp of the time step For time-aligned cross-modal data pairs;

[0122] Sub-step 1.2: Target cattle detection and identification tracking

[0123] Visible light video stream Each frame in The target detection algorithm outputs a set of detection boxes. ,

[0124] in, For the first The coordinates and confidence level of the bounding box. For the first The set of detection boxes at each time step. For the first Time step Parameters of the bit detection box, and The coordinates of the top-left corner pixel of the detection box. and The coordinates of the bottom right corner of the detection box. To test the confidence level;

[0125] Multi-target tracking algorithm for consecutive frames Perform cross-frame association to assign a unique identity to the detected cattle. Generate a set of tracking trajectories ,

[0126] in, For the first A sequence of consecutive detection frames for each cow. For tracking the set of trajectories, For the first The first cow was only in the first Parameters of the detection box at each time step;

[0127] Sub-step 1.3: Extraction of continuous attitude skeleton point coordinates

[0128] For the set of tracking trajectories Each target in At each time step detection box Inside, a pre-trained cattle pose estimation model is used. Extracting the key skeleton point coordinate sequence ,

[0129] in, For the first The coordinates of the skeleton points of the time step. For the first The sequence of coordinates of the skeleton of a cow;

[0130] Will With timestamp Binding, forming a time-series skeleton point coordinate sequence:

[0131] , For the first A time-series sequence of skeletal coordinates of a single cow;

[0132] Step 2: Based on the continuous attitude skeleton point coordinate sequence, visible light video stream and infrared thermal imaging data stream, extract and generate attitude feature sequence, visual feature sequence and temperature feature sequence;

[0133] In step 2, temperature feature sequences are extracted and generated, specifically including: temperature sampling of the core area of ​​the cow's body determined by the continuous posture skeleton point coordinate sequence, in the time-synchronized infrared thermal imaging data stream, and the sampled values ​​are used as temperature features.

[0134] Sub-step 2.1: Generation of pose feature sequences

[0135] Based on continuous attitude skeleton point coordinate sequence Calculate time steps Skeletal point motion velocity characteristics The expression is:

[0136] ,

[0137] in, For time step interval, For the first The coordinates of the skeleton points of the time step. For the first The coordinates of the skeleton points of the time step. For the first The timestamp of the time step For the first The timestamp of the time step;

[0138] Will and Concatenate to form a pose feature vector ;

[0139] Sub-step 2.2: Visual feature sequence generation

[0140] For visible light video streams and timestamps Aligned cattle image regions Input to a lightweight convolutional neural network Perform feature extraction to generate visual feature vectors. The expression is:

[0141] ,

[0142] in, For visual feature vectors, Visible light video stream The cropped area of ​​the cow image. For lightweight convolutional neural networks;

[0143] Sub-step 2.3: Generation of temperature feature sequences

[0144] Based on the continuous attitude skeleton point coordinate sequence In timestamp Corresponding infrared thermal imaging data In the middle, the spinal skeleton point Generate a rectangular sampling area centered on the target region:

[0145] ,

[0146] in, The preset sampling radius;

[0147] calculate Average temperature of all pixels Forming a temperature feature vector ;

[0148] Step 3: Perform feature vectorization on the pose feature sequence, visual feature sequence, and temperature feature sequence, and inject temporal location information and modality type information to construct a multimodal fusion input feature sequence;

[0149] Sub-step 3.1: Multimodal Feature Vectorization Mapping

[0150] pose feature sequence Visual feature sequences and temperature feature sequence Dimensionality reduction is performed using separate linear projection layers:

[0151] ,

[0152] ,

[0153] ,

[0154] in, For the first The pose feature vector at each time step. For the first Visual feature vectors at time steps For the first Temperature feature vector at time step , , For trainable projection matrices, , , For bias terms;

[0155] Sub-step 3.2: Timing Position Encoding Injection

[0156] For each time step Projection features , , Injecting sinusoidal absolute position code The calculation formula is:

[0157] ,

[0158] ,

[0159] in, For sinusoidal position coding components, For cosine position coding components, For dimensional indexing, For the first The timestamp of the time step;

[0160] Add the positional encoding to the projected features:

[0161] ,

[0162] ,

[0163] ,

[0164] in, To enhance features by fusing attitude features with temporal location information, To enhance features by fusing visual features with temporal and location information, Enhanced features for temperature characteristics and temporal location information are used for fusion.

[0165] Sub-step 3.3: Modality type embedding and fusion

[0166] Enhancing features for location encoding , , Add learnable modality embeddings Generate the final multimodal fusion input features:

[0167] ,

[0168] ,

[0169] ,

[0170] The three components are concatenated according to time steps to construct a complete input feature sequence. :

[0171] ,

[0172] in, This is the final attitude mode feature vector. This is the final visual modality feature vector. This represents the final temperature mode eigenvector;

[0173] Step 4: Input the multimodal fusion input feature sequence into the preset lightweight Transformer model for processing. The lightweight Transformer model has a built-in dynamic attention switching mechanism. Under normal conditions, the dynamic attention switching mechanism uses local attention calculation, and when a preset key event is detected, it switches to sparse global attention calculation to generate an enhanced feature sequence.

[0174] Sub-step 4.1: Local attention calculation

[0175] For multimodal fusion input feature sequences Under normal conditions, the encoder layers of the lightweight Transformer model process each feature vector... Perform local attention calculations, limiting the attention scope to a window. ,in, Set the default window size;

[0176] Local attention weights The calculation formula is:

[0177] ,when ,

[0178] in, , , For a trainable parameter matrix, For feature dimension, For the first Feature vector For the Feature vector The level of attention, For the first Feature vector linear projection, for Feature vector Linear projection;

[0179] Sub-step 4.2: Critical event detection and attention mode switching

[0180] Real-time monitoring of multimodal fusion input feature sequences Attention mode switching is triggered when any of the following key events are detected:

[0181] Drastic attitude change events: Calculating the velocity characteristics of skeleton points in continuous time steps of Norm change ,like This was determined to be a drastic change in posture.

[0182] in, and For the first A cow only in time steps and The speed of the posture movement;

[0183] Temperature anomaly events: Calculate current temperature characteristics With sliding window mean deviation ,like ,in, For preset multiples, The standard deviation of the temperature within the window is used to determine a temperature anomaly.

[0184] Sub-step 4.3: Sparse Global Attention Calculation

[0185] When a critical event is detected in sub-step 4.2, the time step of the event is set. All corresponding modal feature vectors Expanding the attention scope to the global level, we calculate the global attention weights:

[0186] ,

[0187] Output features ,in, , For a trainable parameter matrix, For the first eigenvector pair The degree of global attention on feature vectors;

[0188] Step 5: Decode the enhanced feature sequence, calculate the health status index of the target cattle, and output the health assessment conclusion based on the comparison results between the health status index and the preset health baseline.

[0189] In step 5, the minimum health status indicators include: the activity frequency index calculated based on the frequency of the target cattle's dynamic behavior, the standing-to-lying ratio, which is the ratio of standing to lying posture time during the statistical analysis period, and the body temperature fluctuation trend obtained by trend analysis of the temperature feature sequence.

[0190] In step 5, the output of the health assessment conclusion is as follows: when the health status indicators continuously deviate from the normal range of the preset health baseline, the system automatically triggers an alarm mechanism and pushes a notification containing the identity of the target cattle. Early warning information for abnormal indicators;

[0191] Sub-step 5.1: Health Indicator Decoding and Calculation

[0192] Enhance feature sequences Input to multilayer perceptron decoder Generate predicted values ​​for health indicators:

[0193] ,in, ,

[0194] in, This is an activity frequency index. The ratio of standing to lying down is given. For the trend of body temperature fluctuation, Corresponding time step Temperature mode enhancement features;

[0195] Sub-step 5.2: Sliding window health baseline calculation

[0196] For each health indicator Calculate within the sliding time window Baseline mean and standard deviation within:

[0197] , ,

[0198] in, For window length, The mean of the healthy baseline, Standard deviation of the healthy baseline;

[0199] Sub-step 5.3: Anomaly Detection and Early Warning Trigger

[0200] When any health indicator Satisfying the continuous deviation condition:

[0201] continuous The test was successful.

[0202] in, The deviation coefficient, The duration threshold;

[0203] Trigger the alarm mechanism and generate a generator containing the identity of the target cattle. Abnormal indicator types Warning information on the degree of deviation ,

[0204] in, .

[0205] Step 1 utilizes simultaneous acquisition of dual-modal data from visible light and infrared thermal imaging, combined with target detection and attitude estimation techniques, to achieve accurate identification and continuous attitude tracking of individual cattle. The advantages are: strict alignment of multi-source data through timestamps ensures spatiotemporal consistency for subsequent analysis; quantification of cattle behavior patterns using attitude skeleton point coordinate sequences provides structured motion features for health assessment; and the multi-target tracking algorithm effectively solves the herd occlusion problem, ensuring the continuity of individual data and overcoming the limitations of traditional manual inspections in continuous monitoring.

[0206] Step 2 constructs a multidimensional health representation system by jointly extracting postural velocity features, visual appearance features, and core body temperature features. The value lies in the fact that postural velocity features capture subtle behavioral anomalies, visual features extract robust environmental appearance information, and temperature features reflect changes in physiological state. These three features complement each other at the feature level, enhancing the model's ability to perceive early health risks and significantly improving the comprehensiveness and robustness of feature expression compared to single-modal analysis.

[0207] Step 3 unifies the feature dimensions through linear projection and injects temporal position encoding and modality type embedding to achieve deep fusion of multimodal data. The innovation is reflected in the following aspects: temporal encoding preserves the chronological relationship of events, modality embedding distinguishes the semantic differences of different data sources; feature vectorization eliminates the scale differences between modalities, provides standardized input for the Transformer model, and solves the problem of model convergence difficulties caused by feature heterogeneity in traditional methods.

[0208] Step 4 utilizes a dynamic switching mechanism between local and global attention to achieve deep correlation analysis of key events with low computational overhead. The technological breakthrough lies in: local attention reduces computational complexity under normal conditions, adapting to the computing power limitations of edge devices; when sudden changes in posture or abnormal body temperature are detected, sparse global attention captures long-range dependencies, enhancing the model's ability to interpret sudden events and overcoming the traditional model's trade-off between efficiency and accuracy.

[0209] Step 5 achieves individualized and adaptive health status assessment through sliding window baseline calculation and continuous deviation detection. Its advantages include: a dynamic baseline reflects the historical health status of individual cattle, avoiding false alarms caused by fixed thresholds; the multilayer perceptron decoder maps abstract features into interpretable indicators, improving the reliability of the results; and continuous deviation conditions ensure the reliability of alarms, solving the problem of traditional methods being sensitive to instantaneous noise, and providing accurate decision support for ranch management.

[0210] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method based on A lightweight model for cattle health assessment, characterized by: Includes the following steps: step Simultaneously acquire visible light video stream and infrared thermal imaging data stream of the target cattle, and process the visible light video stream to identify, track and obtain the continuous posture skeleton point coordinate sequence of the target cattle; step Based on the continuous attitude skeleton point coordinate sequence, the visible light video stream and the infrared thermal imaging data stream, attitude feature sequence, visual feature sequence and temperature feature sequence are extracted and generated; step The posture feature sequence, the visual feature sequence, and the temperature feature sequence are processed into feature vectors, and temporal position information and modality type information are injected to construct a multimodal fusion input feature sequence. step The multimodal fusion input feature sequence is input into a preset lightweight input. The model is processed, the lightweighting The model has a built-in dynamic attention switching mechanism. Under normal conditions, the dynamic attention switching mechanism uses local attention calculation, and switches to sparse global attention calculation when a preset key event is detected to generate an enhanced feature sequence. The steps Further including: Sub-step Local attention computation For the multimodal fusion input feature sequence Under normal conditions, the lightweight Each encoder layer of the model processes each feature vector. Perform local attention calculations, limiting the attention scope to a window. ,in, Set the default window size; Local attention weights The calculation formula is: ,when , in, , , For a trainable parameter matrix, For feature dimension, For the first Feature vector For the Feature vector The level of attention, For the first Feature vector linear projection, for Feature vector Linear projection; Sub-step Critical event detection and attention mode switching Real-time monitoring of the multimodal fusion input feature sequence Attention mode switching is triggered when any of the following key events are detected: Drastic attitude change events: Calculating the velocity characteristics of skeleton points in continuous time steps of Norm change ,like ,in, A preset speed change threshold is used to determine if a sudden change in attitude has occurred. in, and For the first A cow only in time steps and The characteristics of the motion velocity of the skeleton points; Temperature anomaly events: Calculate the current temperature characteristic sequence With sliding window mean deviation ,like ,in, For preset multiples, The standard deviation of the temperature within the window is used to determine a temperature anomaly. Sub-step Sparse Global Attention Computation When sub-step When a critical event is detected, the time step of the event is determined. All corresponding modal feature vectors Expanding the attention scope to the global level, we calculate the global attention weights: ， Output features ,in, , For a trainable parameter matrix, For the first eigenvector pair The degree of global attention on feature vectors; step The enhanced feature sequence is decoded to calculate the health status index of the target cattle, and a health assessment conclusion is output based on the comparison result of the health status index with the preset health baseline.

2. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps In this process, the visible light video stream is processed to identify, track, and obtain a continuous sequence of pose skeleton point coordinates of the target cow. Specifically, this includes: using a target detection algorithm to identify the cow in the image, and using a multi-target tracking algorithm to assign a unique identity to the target cow. To achieve continuous tracking.

3. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps In the process, the temperature feature sequence is extracted and generated, specifically including: temperature sampling is performed on the core area of ​​the cow's body determined according to the continuous posture skeleton point coordinate sequence, in the infrared thermal imaging data stream synchronized with time, and the sampled values ​​are used as temperature features.

4. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps The health status indicators include at least the following: an activity frequency index calculated based on the frequency of the target cattle's dynamic behavior, a standing-to-lying ratio calculated by statistically analyzing the proportion of standing and lying postures during the period, and a body temperature fluctuation trend obtained by trend analysis of the temperature feature sequence.

5. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps Specifically, the output health assessment conclusion is as follows: when the health status indicator continuously deviates from the normal range of the preset health baseline, the system automatically triggers an alarm mechanism and pushes a notification containing the identity of the target cattle. Warning information related to abnormal indicators.

6. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps Further including: Sub-step Synchronous acquisition and timestamp annotation of multi-source data Visible light cameras and infrared thermal imaging cameras are deployed to synchronously acquire visible light video streams of the target cattle at a fixed frame rate. and infrared thermal imaging data stream , in, For time step index; For each frame and Mark the same timestamp Forming time-aligned cross-modal data pairs , in, It is a visible light video stream. The first in the visible light video stream time step Image frame, For infrared thermal imaging data stream, The first in the infrared thermal imaging data stream Thermal imaging image frames at time steps For the first The timestamp of the time step For time-aligned cross-modal data pairs; Sub-step Target cattle detection and identification tracking For the visible light video stream Each frame in The target detection algorithm outputs a set of detection boxes. , in, For the first The coordinates and confidence level of the bounding box. For the first The set of detection boxes at each time step. For the first Time step Parameters of the bit detection box, and The coordinates of the top-left corner pixel of the detection box. and The coordinates of the bottom right corner of the detection box. To test the confidence level; Multi-target tracking algorithm for consecutive frames Perform cross-frame association to assign a unique identity to the detected cattle. Generate a set of tracking trajectories , in, For the first A sequence of consecutive detection frames for each cow. For tracking the set of trajectories, For the first The first cow was only in the first Parameters of the detection box at each time step; Sub-step Extraction of continuous attitude skeleton point coordinates For the set of tracking trajectories Each target in At each time step detection box Inside, a pre-trained cattle pose estimation model is used. Extracting the key skeleton point coordinate sequence , in, For the first The coordinates of the skeleton points of the time step. For the first The sequence of coordinates of the skeleton of a cow; Will With timestamp Binding, forming a continuous sequence of attitude skeleton point coordinates: , For the first A sequence of continuous posture skeleton point coordinates for a single cow.

7. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps Further including: Sub-step Based on the continuous attitude skeleton point coordinate sequence Calculate time steps Skeletal point motion velocity characteristics The expression is: ， in, For time step interval, For the first The coordinates of the skeleton points of the time step. For the first The coordinates of the skeleton points of the time step. For the first The timestamp of the time step For the first The timestamp of the time step; Will and Concatenate to form a pose feature vector ; Sub-step For the visible light video stream with timestamp Aligned cattle image regions Input to a lightweight convolutional neural network Perform feature extraction to generate visual feature vectors. The expression is: ， in, For visual feature vectors, Visible light video stream The cropped area of ​​the cow image. For lightweight convolutional neural networks; Sub-step According to the continuous attitude skeleton point coordinate sequence In timestamp Corresponding infrared thermal imaging data In the middle, the spinal skeleton point Generate a rectangular sampling area centered on the target region: ， in, The preset sampling radius; calculate Average temperature of all pixels Forming a temperature feature vector .

8. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps Further including: Sub-step Multimodal feature vectorization mapping pose feature sequence Visual feature sequences and temperature feature sequence Dimensionality reduction is performed using a separate linear projection layer: ， ， ， in, For the first The pose projection feature vector at the time step. For the first Visual projection feature vectors at time steps For the first Temperature projection feature vector at time step , , For trainable projection matrices, , , For bias terms; Sub-step Timing Position Encoding Injection For each time step pose projection feature vector Visual projection feature vector Temperature projection eigenvector Injecting sinusoidal absolute position code The calculation formula is: ， ， in, For sinusoidal position coding components, For cosine position coding components, For dimensional indexing, For the first The timestamp of the time step; The positional encoding is added to each projected feature vector to generate the positional encoding enhanced feature: ， ， ， in, To enhance features by fusing attitude features with temporal location information, To enhance features by fusing visual features with temporal and location information, Enhanced features that fuse temperature characteristics with temporal location information; Sub-step Modal type embedding fusion Location-encoded enhanced features , , Add learnable modality embeddings Generate the final multimodal fusion input features: ， ， ， The three components are concatenated according to time steps to construct a complete input feature sequence. : ， in, This is the final attitude mode feature vector. This is the final visual modality feature vector. This represents the final temperature mode eigenvector.

9. A method based on claim 1 A lightweight model for cattle health assessment, characterized by: The steps Further including: Sub-step Health Indicator Decoding and Calculation The enhanced feature sequence Input to multilayer perceptron decoder Generate predicted values ​​for health indicators: ,in, , in, This is an activity frequency index. The ratio of standing to lying down is given. For the trend of body temperature fluctuation, Corresponding time step Temperature mode enhancement features; Sub-step Sliding window health baseline calculation For each health indicator Calculate within the sliding time window Baseline mean and standard deviation within: ， ， in, For window length, The mean of the healthy baseline, Standard deviation of the healthy baseline; Sub-step Anomaly detection and early warning triggering When any health indicator Satisfying the continuous deviation condition: continuous The test was successful. in, The deviation coefficient, The duration threshold; Trigger the alarm mechanism and generate a generator containing the identity of the target cattle. Abnormal indicator types Warning information on the degree of deviation ,in, .

Images