Intelligent sheep growth state prediction method based on multi-modal data fusion

Abstract

The invention discloses an intelligent sheep growth state prediction method based on multi-modal data fusion. The method comprises the following steps: 1) calculating sheep body size parameters through an RGB-D camera and a three-dimensional dynamic calibration algorithm; 2) fusing the visible light / thermal infrared image and the activity amount to construct a wool thickness evaluation model, and 3) predicting a future growth trend based on a multi-mode Transform model, and outputting a weight prediction value.

Description

Technical Field

[0001] The present invention relates to the field of smart animal husbandry technology, and in particular to a method for intelligently predicting sheep growth status based on multimodal data fusion, which specifically includes non-contact phenotypic monitoring, dynamic evaluation of wool thickness, and multimodal growth prediction. Background Art

[0002] The existing traditional breeding has the following technical defects:

[0003] 1. Strong dependence on experience: Traditional farming relies on manual observation to assess growth status and cannot quantitatively predict future trends.

[0004] 2. Single monitoring method: Existing systems mostly use a single weight sensor or manual measurement, which cannot obtain key phenotypic parameters such as body size and wool thickness in real time.

[0005] 3. Data isolation and inefficiency: Growth data, breed information and behavioral data are not effectively integrated, the accuracy rate of identifying inferior sheep is less than 70%, and the feed waste rate is >20%.

[0006] 4. Wool evaluation is delayed: it relies on manual palpation or actual shearing measurement, and cannot dynamically monitor changes in wool quality without contact. Summary of the Invention

[0007] The present invention aims to overcome the deficiencies of the prior art and provide a method for intelligently predicting sheep growth status based on multimodal data fusion.

[0008] In order to achieve the above object, the technical solution provided by the present invention is:

[0009] The method for intelligently predicting sheep growth status based on multimodal data fusion comprises the following steps:

[0010] 1) Calculate sheep body size parameters using an RGB-D camera and a 3D dynamic calibration algorithm;

[0011] 2) Fusing visible light / thermal infrared images, behavioral activity, and metabolic data to construct a three-modal fusion model of wool thickness;

[0012] 3) Predict future growth trends based on the multimodal Transformer model and output weight prediction values.

[0013] Preferably, the three-dimensional dynamic calibration algorithm adopts ICP point cloud alignment and OpenPose posture estimation, and the body measurement error is ≤1.5cm.

[0014] Preferably, the wool thickness trimodal fusion model includes combining existing long-term tracking data before and after shearing to predict the wool coverage level and shearing timing of sheep in different production cycles; the implementation involves time series analysis and feature calibration, specifically: taking the "shearing event" as the key time node to construct a central time window, normalizing the data of different cycles, extracting the wool thickness change trend and body condition fluctuation pattern before shearing, and calibrating "wool coverage" or "shearing maturity" as supervised learning labels; in the model architecture, a two-channel convolutional neural network is adopted, in which the first channel extracts visible light image texture features through ResNet50, and the second channel obtains the temperature zone feature map by segmenting the thermal infrared temperature distribution through U-Net. Finally, the fusion layer combines the two-channel image features and activity time series data to output the predicted wool thickness level (for example, level 0-5).

[0015] Preferably, the multimodal Transformer model is as follows: a body size parameter encoder, a wool thickness encoder, and a growth curve encoder are set, a cross-modal attention mechanism is used to fuse spatiotemporal features, and the decoder includes a temporal convolution layer and an LSTM prediction unit, which outputs the predicted weight value and confidence interval for the next 30 days.

[0016] More preferably, the growth curve encoder integrates the food intake, movement trajectory and environmental temperature and humidity data of the past 60 days, and constructs a time series embedding vector through time position coding.

[0017] The present invention will be further described below:

[0018] The core principle of this invention is to build a three-layer intelligent architecture of "perception-fusion-prediction":

[0019] The non-contact phenotyping layer uses 3D visual odometry (RGB-D camera + ICP point cloud alignment) and a posture estimation and compensation algorithm to eliminate measurement errors caused by animal motion. This allows for dynamic collection of body measurements such as body length and chest circumference. The formula is as follows: Weight = 0.0235 × L (length) + 0.715 × H (height) + 0.425 × C (chest circumference) + 0.891 × ρbreed, where ρbreed is the breed density coefficient (e.g., Hulunbuir sheep = 1.02, Suffolk sheep = 0.98). OpenPose skeleton tracking technology corrects the sheep's standing posture in real time, ensuring a point cloud registration error of ≤0.8mm.

[0020] Multi-physics fusion layer: Wool thickness assessment utilizes a combined optical-thermodynamic model. ResNet50 is used to extract hair texture features from visible light images (GLCM energy values ​​> 0.3 indicate thick hair areas). U-Net is used to segment subcutaneous temperature conduction characteristics from thermal infrared images (each 0.5°C decrease in temperature gradient corresponds to a one-level increase in wool thickness). This is combined with activity monitoring using a triaxial accelerometer (recording steps, rest time, and feeding duration), ketone body concentrations, and energy metabolism indicators collected at key time points (such as before and after shearing), along with annotation of shearing time, wool yield, and current feeding strategy. This constructs a temperature-light-dynamic trimodal feature space.

[0021] Spatiotemporal prediction layer: A multi-headed cross-modal Transformer is designed, using a self-attention mechanism to establish a dynamic coupling relationship between body size, hair thickness, and environmental parameters. The temporal convolution layer extracts the periodic pattern of 60 days of historical data (7 days), while the LSTM unit learns the nonlinear characteristics of growth inflection points. The final output confidence interval is calculated using the Monte Carlo Dropout method.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] Improved measurement accuracy: Compared with traditional manual measurement, the efficiency of body parameter detection is increased by 30 times, and the error is reduced from ±3cm to ≤1.5cm; wool thickness assessment achieves hourly dynamic monitoring, and the recognition accuracy rate is increased to 91.7%.

[0024] Data fusion innovation: By establishing multi-dimensional feature associations through a cross-modal attention mechanism, the accuracy rate of identifying inferior sheep reached 89.3% (an increase of 25% compared to traditional methods), and the feed utilization rate was increased to 87.5%.

[0025] Optimized prediction performance: The mean absolute error (MAE) of 30-day weight prediction is 0.82 kg, which is 42% lower than that of the single-modal LSTM model; the confidence interval coverage exceeds 95%.

[0026] Breakthrough in automation: Achieve all-weather unmanned monitoring, reduce manual inspection workload by 75%, and provide 60-day advance warning for wool quality assessment. DETAILED DESCRIPTION

[0027] A total of 80 6-month-old Hulunbuir sheep (half male and half female) were selected from an intensive ranch in Inner Mongolia with a scale of 10,000 sheep. A traditional manual monitoring control group (n=80) was set up at the same time.

[0028] 1. Hardware deployment and data collection

[0029] Sensing equipment:

[0030] 3D perception module: Hikvision MV-CA016-10GC RGB-D camera (resolution 1920×1080, frame rate 30fps, depth accuracy ±2mm@1m), installed 1.2m above the sheep channel, with two cameras arranged symmetrically with a spacing of 2m.

[0031] Thermal imaging module: FLIR T865 thermal imager (resolution 640×480, thermal sensitivity <40mK), coaxially mounted with the RGB-D camera, equipped with an automatic defog lens.

[0032] Behavior monitoring module: DS-TCG206 three-axis accelerometer (sampling rate 50 Hz, range ±16 g), fixed on the inside of the sheep's collar.

[0033] Environmental Control:

[0034] The channel is paved with anti-slip mats to eliminate motion artifacts, and LED fill lights (color temperature 5600K, illumination 500lux) are used to ensure all-weather imaging consistency.

[0035] 2. 3D dynamic calibration

[0036] Point cloud preprocessing:

[0037] Use bilateral filtering (σ_color=10, σ_space=15) to remove hair reflection noise;

[0038] The ground point cloud was segmented using the RANSAC algorithm (1000 iterations and an inlier threshold of 3 mm).

[0039] Posture compensation algorithm:

[0040] Based on OpenPose's 17 key point detection (confidence > 0.7), a skeleton coordinate system is constructed.

[0041] Design the pose transformation matrix:

[0042] T=[R|t]=argminΣ||(P_i-(R·Q_i+t))||2

[0043] Where P is the actual point cloud and Q is the standard standing posture model;

[0044] The optimal rigid body transformation is solved by the LM optimization algorithm (λ=1e-4).

[0045] Verification results (see Table 1):

[0046] Table 1

[0047]

[0048]

[0049] 3. Wool thickness assessment model training

[0050] Dataset construction:

[0051] Simultaneously collect 5,000 sets of visible light and thermal infrared image pairs (including labels for measured thickness of manually sheared hair);

[0052] Data augmentation: random rotation (±15°), brightness perturbation (±20%), and adding synthetic hair noise.

[0053] Model Architecture:

[0054]

[0055] Training configuration:

[0056] Optimizer: AdamW(lr=3e-4, weight_decay=1e-5);

[0057] Loss function: Focal Loss (γ=2, α=[0.1, 0.15, 0.2, 0.25, 0.2, 0.1]);

[0058] Training period: 150 epochs (early stopping patience = 20).

[0059] 4. Validation of the Multimodal Growth Prediction System

[0060] Data timing processing:

[0061] Input window: 60 days of historical data (sliding window step size = 1 day).

[0062] Feature Engineering:

[0063] Body size parameters: 7-day moving average smoothing;

[0064] Food intake: converted into metabolizable energy (ME = 1.2 × DMI ^ 0.75);

[0065] Movement trajectory: Calculation of daily activity entropy (H = -Σp(x)logp(x)).

[0066] Transformer hyperparameters:

[0067]

[0068] Comparison of prediction performance (see Table 2):

[0069] Table 2

[0070]

[0071] 5. Field application effect

[0072] Improved economic indicators:

[0073] The accuracy rate of identifying inferior sheep: control group 72.5% → experimental group 93.4% (p < 0.01);

[0074] Average daily weight gain: experimental group (326±21g / d) vs control group (289±35g / d);

[0075] Feed conversion rate: FCR of experimental group = 4.1 vs. FCR of control group = 5.3.

[0076] Abnormal detection case: Sheep No. 7 was found to have stagnant growth 14 days in advance (actually diagnosed with digestive tract parasite infection).

Claims

1. An intelligent prediction method for sheep growth status based on multimodal data fusion, characterized in that: The method comprises the following steps: 1) Calculate sheep body size parameters using an RGB-D camera and a 3D dynamic calibration algorithm; 2) Fusing visible light / thermal infrared images, behavioral activity, and metabolic data to construct a three-modal fusion model of wool thickness; 3) Predict future growth trends based on the multimodal Transformer model and output weight predictions.

2. The method for intelligently predicting sheep growth status based on multimodal data fusion according to claim 1 is characterized in that: The 3D dynamic calibration algorithm uses ICP point cloud alignment and OpenPose pose estimation, and the body measurement error is ≤1.5cm.

3. The method for intelligent prediction of sheep growth status based on multimodal data fusion according to claim 1 is characterized in that: The wool thickness three-modal fusion model combines existing long-term tracking data before and after shearing to predict the wool coverage level and shearing timing of sheep in different production cycles.

4. The method for intelligently predicting sheep growth status based on multimodal data fusion according to claim 3 is characterized in that: The wool thickness three-modal fusion model is a two-channel convolutional neural network, in which: the first channel extracts visible light image texture features through ResNet50, the second channel segments thermal infrared temperature distribution through U-Net, and the fusion layer combines activity time series data and temperature zone feature maps to output wool thickness levels of 0-5.

5. The method for intelligently predicting sheep growth status based on multimodal data fusion according to claim 1 is characterized in that: The multimodal Transformer model is as follows: a body size parameter encoder, a wool thickness encoder, and a growth curve encoder are set, a cross-modal attention mechanism is used to fuse spatiotemporal features, and the decoder includes a temporal convolution layer and an LSTM prediction unit, which outputs the predicted weight value and confidence interval for the next 30 days.

6. The method according to claim 5, characterized in that The growth curve encoder integrates 60 days of historical food intake, movement trajectory, and environmental temperature and humidity data, and constructs a time series embedding vector through time position encoding.

Images