Intelligent Nutritional Pattern Management Method and System Based on HAP Multimodal Data

Abstract

This invention relates to the interdisciplinary field of computer science and animal husbandry, and discloses an intelligent nutrition model management method and system based on HAP multimodal data. The intelligent nutrition model management method based on HAP multimodal data includes: acquiring multi-source heterogeneous HAP data of livestock and poultry; extracting and fusing features from the multi-source heterogeneous HAP data; generating enhanced HAP feature data and HAP feature data sequences simulating physiological processes using a controllable generative adversarial network; detecting early abnormal signals using an anomaly detection algorithm and performing multimodal attribution analysis; and continuously optimizing the nutritional requirement prediction model and feeding strategy optimization model. This invention improves the early detection capability of weak signals indicating physiological abnormalities or nutritional imbalances in livestock and poultry by effectively utilizing previously overlooked unstructured information and combining it with multimodal fusion analysis.

Description

Technical Field

[0001] This invention relates to the field of computer and livestock farming technology, and more specifically, to an intelligent nutrition model management method and system based on HAP multimodal data. Background Technology

[0002] In modern intensive livestock and poultry farming, intelligent nutrition management is crucial for improving production efficiency and ensuring animal welfare.

[0003] Current technologies primarily rely on structured sensor data (such as body temperature and activity levels) for analysis. However, high-quality data is scarce in key physiological scenarios (such as early disease stages and drastic environmental changes), leading to insufficient model robustness. Furthermore, existing technologies fail to fully integrate and utilize early abnormal signals contained in unstructured information such as text records and audio data generated during the farming process. In addition, current technologies lack precise simulation and model validation methods for HAP (High Growth Potential, Active Health, and Preventive Nutritional Analysis) data of individual livestock and poultry based on different growth potentials, changes in health status, and abnormal conditions. These shortcomings limit the ability to provide early and accurate warnings of livestock and poultry health risks, affecting the precision and foresight of intelligent nutrition management, and constitute a major technical problem urgently needing to be solved in the current field of intelligent livestock farming.

[0004] These shortcomings limit the ability to provide early and accurate warnings of livestock and poultry health risks, and affect the precision and foresight of intelligent nutrition management, constituting a major technical problem that urgently needs to be solved in the current field of intelligent livestock farming. Summary of the Invention

[0005] This invention provides an intelligent nutrition model management method and system based on HAP multimodal data, which solves the technical problems of weak early anomaly detection capability and poor robustness of nutrition models caused by data scarcity, insufficient utilization of unstructured information and lack of dynamic process simulation and verification in related technologies.

[0006] This invention provides an intelligent nutrition pattern management method based on HAP multimodal data, including:

[0007] Multi-source heterogeneous HAP data of livestock and poultry were acquired, and a pre-trained large model was used to extract features from the multi-source heterogeneous HAP data. The features were then fused using a multimodal attention fusion model to obtain fused HAP feature data.

[0008] Based on fused HAP feature data, a controllable generative adversarial network (GAN) is used to generate enhanced HAP feature data and HAP feature data sequences simulating physiological processes. Specifically, the use of the controllable GAN includes:

[0009] Using a generator model, based on the input random noise vector, preset scenario conditions, and the potential state vector corresponding to the preset physiological indicator change path, enhanced HAP feature data and simulated physiological process HAP feature data sequences are generated.

[0010] Using a discriminator model, we can distinguish between real HAP feature data and HAP feature data generated by a generator model, and optimize the generator model and discriminator model through an adversarial training process.

[0011] By utilizing fused HAP feature data and enhanced HAP feature data, anomaly detection algorithms are used to detect early anomalous signals and perform multimodal attribution analysis to obtain attribution results.

[0012] Based on early abnormal signals and attribution results, the nutritional requirement prediction model and feeding strategy optimization model are continuously optimized.

[0013] Furthermore, the acquisition of multi-source heterogeneous HAP data of livestock and poultry includes structured sensor data, structured text, audio data, image data, and video data.

[0014] Furthermore, the core component of the multimodal attention fusion model is an attention module, which dynamically allocates attention weights based on the importance of different modal features in representing the current individual state. The specific steps are as follows:

[0015] First, the feature vectors of each modality are... Projected onto the same dimension through their respective independent MLP layers. ;

[0016] Then, all projected modal features are concatenated or summed, and then input into another MLP to calculate the attention score for each modality. The attention weights are then obtained by normalization using the Softmax function. .

[0017] Furthermore, the anomaly detection algorithm is specifically an autoencoder model based on reconstruction error, and the detection steps include:

[0018] The input HAP feature data is reconstructed using an autoencoder model to obtain reconstructed HAP feature data.

[0019] Calculate the reconstruction error between the input HAP feature data and the reconstructed HAP feature data;

[0020] When the reconstruction error exceeds the preset error threshold, an early abnormal signal is detected.

[0021] Furthermore, the preset error threshold is determined by a dynamic threshold adjustment strategy based on sliding window statistics.

[0022] Furthermore, the over-anomaly detection algorithm for detecting early anomalous signals and performing multimodal attribution analysis includes:

[0023] When the multimodal attention fusion model adopts an explicit attention mechanism, the magnitude of the attention weights corresponding to each original modality feature vector in the fused HAP feature data that leads to early anomalous signals is analyzed to determine the modality that contributes the most to the early anomalous signals.

[0024] The gradient value of the output result of the anomaly detection algorithm with respect to the feature vector of each original mode in the fused HAP feature data is calculated. The mode with the larger gradient value is determined to be the mode that contributes more to the early anomaly signal.

[0025] Furthermore, the continuous optimization of the nutrient requirement prediction model and the feeding strategy optimization model includes: using a comprehensive dataset containing real HAP data, enhanced HAP data, and simulated process HAP data to optimize the nutrient requirement prediction model. and feeding strategy optimization model Perform periodic retraining and parameter fine-tuning.

[0026] Furthermore, the nutritional requirement prediction model is used to predict an individual's dynamic requirements for multiple nutrients based on the individual's current fused HAP feature data and other relevant individual information.

[0027] The feeding strategy optimization model is used to generate specific feed formulas and recommended feeding amounts based on dynamic demand predicted by the nutritional requirement prediction model, combined with feed ingredient information and preset breeding goals, through optimization algorithms.

[0028] This invention provides an intelligent nutrition pattern management system based on HAP multimodal data, used to execute the aforementioned intelligent nutrition pattern management method based on HAP multimodal data, including:

[0029] The data processing module is used to acquire multi-source heterogeneous HAP data of livestock and poultry. It uses a pre-trained large model to extract features from the multi-source heterogeneous HAP data and performs feature fusion through a multimodal attention fusion model to obtain fused HAP feature data.

[0030] The data generation module is used to generate enhanced HAP feature data or HAP feature data sequences that simulate physiological processes based on fused HAP feature data and using a controllable generative adversarial network.

[0031] The anomaly analysis module is used to detect early anomalous signals and perform multimodal attribution analysis by utilizing fused HAP feature data and enhanced HAP feature data, and obtain attribution results through anomaly detection algorithms.

[0032] The model optimization module is used to continuously optimize the nutrient requirement prediction model and the feeding strategy optimization model based on early abnormal signals and attribution results.

[0033] The beneficial effects of this invention are as follows: by effectively utilizing previously ignored unstructured information such as text and audio, and combining it with multimodal fusion analysis, the ability to detect weak signals indicating physiological abnormalities or nutritional imbalances in livestock and poultry is improved, thus gaining valuable time for preventive health management and timely nutritional intervention.

[0034] By generating high-quality, scenario-specific (including dynamic change processes) HAP simulation data, the problem of scarcity of real data is effectively solved, enabling the nutrition management model to maintain stable and accurate performance under a wider range of farming conditions. In particular, it enhances the robustness and scenario adaptability of the model when dealing with rare or extreme situations.

[0035] Intelligent abnormal signal detection and preliminary attribution analysis can provide breeders and veterinarians with clearer problem directions, assist them in making more accurate nutritional adjustments or health management decisions, and provide a more intuitive understanding of the decision-making basis, thereby improving the accuracy and interpretability of the decisions.

[0036] Through a more comprehensive and in-depth understanding of the physiological state of livestock and poultry, and through continuous iterative optimization of model capabilities, the intelligent nutrition management model has been promoted to develop in a more refined (individualized needs meeting) and more forward-looking (predictive intervention based on early signals) direction, which ultimately helps to improve breeding efficiency and animal health and welfare. Attached Figure Description

[0037] Figure 1 This is a flowchart of the intelligent nutrition model management method based on HAP multimodal data in this invention;

[0038] Figure 2 This is a flowchart of step 1 in this invention;

[0039] Figure 3 This is a flowchart of step 2 in this invention;

[0040] Figure 4 This is a flowchart of step 3 in this invention;

[0041] Figure 5 This is a flowchart of step 4 in this invention. Detailed Implementation

[0042] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0043] At least one embodiment of the present invention discloses an intelligent nutrition pattern management method based on HAP multimodal data, such as Figures 1 to 5 Shown, including:

[0044] Step 1: Obtain multi-source heterogeneous HAP data of livestock and poultry, use a pre-trained large model to extract features from the multi-source heterogeneous HAP data, and use a multi-modal attention fusion model to fuse the features to obtain fused HAP feature data.

[0045] Specifically include:

[0046] Step 1.1: Acquisition and time-series alignment of multi-source heterogeneous HAP data;

[0047] Multi-source heterogeneous HAP data related to livestock and poultry were comprehensively collected using various sensors and recording systems deployed in the farm. During this data collection process, the data content across the three dimensions of HAP was specifically manifested as follows:

[0048] High growth potential dimension: Individual growth potential indicators are collected through genetic background analysis and growth performance monitoring sensors, combined with variety information and growth performance descriptions in text records;

[0049] Active dimension of health: Body temperature is collected through wearable sensors. Heart rate Activity level Physiological indicators, combined with abnormal cries from images, video data, and audio data acquired by the machine vision system. ;

[0050] Sub-health nutrition prevention dimension: through zookeeper observation logs The veterinary diagnostic information records an individual's nutritional status, immune level, stress response, and other preventative health management indicators.

[0051] The data acquisition system includes: wearable sensors for collecting physiological indicators, machine vision systems for acquiring image or video information, and systems for capturing ambient sound. and individual voices The system includes a microphone array and an electronic recording system for recording text information. All raw data acquired is timestamped. The data is then subjected to unified timeline alignment and standardization preprocessing to form a time-series dataset containing various types of original HAP data records:

[0052] ;

[0053] in Indicates the first The body temperature of individual livestock and poultry in each record; Indicates the first Heart rate of individual livestock and poultry in each record; Indicates the first The activity level of individual livestock and poultry in each record can be a quantitative value of steps or activity intensity; Indicates the first Ambient sound data in the record, such as audio recordings of equipment noise; Indicates the first Individual sound data in a record, such as audio recordings of abnormal calls from livestock and poultry; Indicates the first Text data recorded by zookeepers in each record, such as observation logs; Indicates the first Image data in the record; Indicates the first Video data in the record; Indicates the first The timestamp of each record; The index number represents the data record and is used to uniquely identify each data record; This represents a time-series dataset containing all original HAP data records.

[0054] Step 1.2: Multimodal feature extraction based on a pre-trained large model;

[0055] For various types of HAP data after time-series alignment, corresponding pre-trained large models are used for deep feature extraction. The aim is to transform the raw data into high-dimensional feature vectors that are easier for machines to analyze and process. Specifically:

[0056] For text data The Bidirectional Encoder Representations from Transformers (BERT) model can be used; the BERT model is a Transformer-based bidirectional encoder that utilizes its text understanding capabilities to represent the text data. The data is fed into a pre-trained BERT model, and the output corresponding to the [CLS] markers of its final hidden layer is obtained as text data. eigenvectors .

[0057] For audio data , A PANN (Large-Scale Pretrained Audio Neural Networks) model can be used, which typically includes Convolutional Neural Networks (CNN) layers for extracting temporal spectral features. The audio data is input into the pretrained PANN model, and the output of a layer before the final fully connected layer is extracted as the audio feature vector. and .

[0058] For structured sensor time-series data , , A Transformer-based encoder model is constructed, comprising a multi-head self-attention layer and a feedforward neural network layer. Multivariate sensor sequence data within a predetermined time window is taken as input, converted into a vector representation through an embedding layer, and then input into the Transformer encoder. The average pooling result or the output of a specific label is extracted as the sensor data feature vector. ;

[0059] For image data The Residual Network (ResNet) is used to extract the output of the average pooling layer as the image feature vector after the image data is input. .

[0060] For video data Image features can be extracted frame by frame first, and then further aggregated into video feature vectors through temporal pooling or temporal convolutional networks. .

[0061] Step 1.3, deep fusion of multimodal features;

[0062] Construct a multi-modal attention fusion model (MAFM) to process the feature vectors of multiple modalities extracted in step 1.2 (i.e., Weighted fusion is performed.

[0063] The core component of a multimodal attention fusion model is an attention module, which dynamically assigns attention weights based on the importance of different modal features in representing the current individual state. The multimodal attention fusion model can be specifically implemented as a structure containing a multilayer perceptron (MLP).

[0064] First, the feature vectors of each modality are... Projected onto the same dimension through independent MLP layers (containing two fully connected layers, using the ReLU activation function). ;

[0065] Then, all projected modal features are concatenated or summed, and then input into another MLP to calculate the attention score for each modality. The attention weights are then obtained by normalization using the Softmax function. .

[0066] Therefore, for any data record Its fused feature vector This can be obtained through processing using the MAFM model:

[0067] ;

[0068] in Indicates from the first Feature vectors extracted from the text data of each record; Indicates from the first Feature vectors extracted from environmental sound data of records; Indicates from the first Feature vectors extracted from individual voice data of each record; Indicates from the first Feature vectors extracted from sensor data of each record; Indicates from the first Feature vectors extracted from image data of records; Indicates from the first Feature vectors extracted from video data of each record; A function representing a multimodal attention fusion model; Indicates the first The fused feature vector of each record;

[0069] The MAFM model contains one or more such attention layers; these attention layers are used to compute modal features. (here) Attention weights (representing different modalities) Then, sum the weighted features:

[0070] ;

[0071] in Indicates the first The fused feature vector of each record Represents the corresponding mode The learnable projection matrix (in the MLP implementation above, this projection is already included in the MLP layers of each modality and in the feature transformation before the final weighted summation) serves to map the features of different modalities to a unified feature dimension space. ; Indicates the first Modality in records The attention weights are calculated based on the input features of all modalities using an auxiliary neural network (the aforementioned MLP used to calculate the attention score); Indicates the first Modality in records eigenvectors; This represents the summation operation for each mode. First, its characteristics Through the corresponding projection matrix Mapped to a unified fusion space, then multiplied by modality. Attention weights Finally, the weighted features of all modalities are summed to obtain the final fused features. .

[0072] Through this step of processing, each data record is ultimately processed. Generate a more information-rich and comprehensive representation of... Unified Enhanced Data View of Time Status Its dimensions are .

[0073] For example, in one application scenario, when the text log data of an individual livestock or poultry... The data recorded information indicating a "significant decrease in feed intake," and the corresponding sensor data... When the activity level remains at a normal level, the MAFM model, through its attention module, can learn the features that should be assigned to the text modality in this situation. Higher attention weight This results in the final fusion features. It can better reflect potential digestive system abnormalities or early disease signals indicated by text information, even if other sensor modal data have not yet shown obvious synchronous abnormalities.

[0074] Step 2: Based on the fused HAP feature data, a controllable generative adversarial network is used to generate enhanced HAP feature data and HAP feature data sequences that simulate physiological processes;

[0075] According to some embodiments of this application, this step aims to apply a Controllable Generative Adversarial Network (CGAN) model to generate high-quality HAP-enhanced data or HAP data sequences simulating specific physiological processes, based on the specific needs of the intelligent nutrition management model. During data generation, the system performs refined classification of the HAP data according to dynamic characteristics such as individual growth potential, changes in health status, and the development of abnormalities, ensuring that the generated data accurately reflects the physiological characteristics and patterns of change in different types of individuals.

[0076] Specifically, based on differences in individual growth potential, HAP data are divided into:

[0077] High growth potential type (data characteristics reflecting excellent genetic background and rapid weight gain ability);

[0078] Medium growth potential (growth performance data at the average level of the population);

[0079] Low growth potential type (data characteristics of slow growth or genetic defects).

[0080] Based on changes in health status, it is divided into:

[0081] Healthy and stable type (physiological indicators fluctuate stably within the normal range);

[0082] Health fluctuation type (slight fluctuations but not reaching the disease threshold);

[0083] Health decline type (a gradual change in health status that leads to a gradual deterioration);

[0084] Health recovery type (data patterns of recovery from illness or stress).

[0085] Based on the development of abnormal situations, it is divided into:

[0086] Acute abnormal type (data characteristics of sudden health problems);

[0087] Chronic abnormal type (data characteristics of long-term sub-health state);

[0088] Periodic abnormality type (regular abnormalities in a specific physiological cycle);

[0089] Warning abnormality type (early weak signals during the incubation period of the disease).

[0090] It should be noted that the CGAN model typically includes a generator model. A discriminator model .

[0091] Specifically, the generator model and the discriminator model A network structure based on a multilayer perceptron can be used. Generator model. The input layer receives a random noise vector. and conditional information (such as scene tags) or interpolation latent vector ), followed by, for example, three more. Fully connected hidden layers, each containing, for example, 256 neurons, using ReLU or LeakyReLU as the activation function; the number of neurons in the output layer is fused with the target HAP feature data. The dimensions are consistent, and according to Choose an appropriate activation function based on the numerical range of the features (e.g., if...). If the values ​​have been normalized to between -1 and 1, then the Tanh activation function can be used. Accordingly, the discriminator model... The input layer receives HAP fused feature data (real or generated ) and condition information (If applicable) This can be followed by, for example, three fully connected hidden layers, each containing, for example, 256 neurons, using ReLU or LeakyReLU activation functions; the output layer is a single neuron, which outputs a scalar value between 0 and 1 through the Sigmoid activation function, representing the probability that the input data is real data.

[0092] When training the CGAN model, an adversarial loss function can be used, such as the standard min-maximum loss function:

[0093] ;

[0094] in Represents conditional information. This represents the generator model in a generative adversarial network. This represents a discriminator model in a generative adversarial network. This represents the random noise vector input to the generator; This represents the latent vector obtained through interpolation; This represents the true HAP fusion feature data; This represents the generated HAP fusion feature data; Represents the loss function of the CGAN model; This indicates that the generator G is being optimized by minimization. This indicates that the discriminator D is optimized to the maximum extent. This represents the expectation given the actual data distribution. This represents the expectation under the noise distribution; This represents the logarithmic probability of the discriminator on the true data under condition c; This represents the log probability that the discriminator classifies the generated data as false under condition c;

[0095] The optimization process can use, for example, the Adam optimizer, with a learning rate of 0.0002 and a batch size of 64 for iterative training until the model converges.

[0096] Step 2.1, Enhancement of HAP data in key scarce scenarios;

[0097] For certain physiological scenarios in animal husbandry that occur infrequently but are crucial for model training (e.g., early clinical manifestations of specific diseases, initial stress responses in animals to extreme environments), the sample size of real-world HAP data for these scenarios is often insufficient. To address this issue, this application provides a method for generating HAP data for these scarce scenarios using a CGAN model. Specifically, the scenario is labeled... (For example, The condition (which can be set to "early respiratory infection") can be input into the generator model. At the same time, a random noise vector is input. .

[0098] Generator Model This leads to the output of simulated HAP fusion feature data for this scenario:

[0099] ;

[0100] in This represents the generator model in a generative adversarial network. This represents a discriminator model in a generative adversarial network. This represents the random noise vector that is provided as input to the generator model; This represents a conditional label used to represent a specific scarce physiological scenario; This represents the generated HAP fusion feature data;

[0101] The feature distribution should strive to be as close as possible to the real-world scenario. The data is similar.

[0102] Discriminator Model This is used to distinguish between real and generated data, and to encourage the generator model to improve through adversarial training. It produces more realistic data.

[0103] This is how it was generated The data can be used to augment the original dataset to enhance the training performance of downstream nutrition management models.

[0104] Step 2.2, simulation of HAP data sequence of gradual change process of specific physiological indicators;

[0105] To achieve in-depth stress testing of the dynamic response capabilities of nutrition management models, it is necessary to simulate certain physiological indicators in livestock and poultry (e.g., body temperature). This application describes the performance of HAP data during continuous and smooth changes. It provides a method utilizing the control capabilities of a CGAN model to perform interpolation or attribute editing operations within the latent space of a pre-trained GAN model. For example, latent vectors corresponding to "health status" can be obtained first. and the potential vector corresponding to the "high thermal state" Then, through and Perform linear interpolation between them:

[0106] ;

[0107] in Indicates the interpolation coefficients; A vector representing health status in the latent space of a GAN model; A vector representing the high-temperature state in the latent space of a GAN model; The latent vector obtained through interpolation;

[0108] A series obtained in this way Vector input to generator model From this, a set of HAP data sequences that can simulate a gradual increase in body temperature from a normal state to a hyperthermic state can be obtained:

[0109] ;

[0110] in This represents a simulated HAP data sequence. , , These represent the generator's input to the interpolation vector. , , The output, This indicates the number of samples in the simulated data sequence.

[0111] This generated sequence This data can be used as input to downstream nutrition management models to evaluate their predictive stability and accuracy under dynamic physiological changes. A specific application example is in sow farming, where this method can simulate the gradual physiological changes typically occurring in sows' multimodal HAP data, such as body temperature and activity levels, during the week before farrowing. By defining latent space vectors corresponding to the initial state (e.g., 7 days before farrowing, normal body temperature, moderate activity) and the final state (e.g., near farrowing, potentially slight temperature fluctuations, reduced activity, or restlessness), and performing smooth interpolation, simulated HAP data sequences are generated. These HAP data sequences can then be used to test existing intelligent nutrition requirement prediction models. Can we accurately and dynamically adjust the recommended supplementation programs for key prepartum nutrients (such as energy, fiber, and specific vitamins) in sows based on these simulated gradual signals?

[0112] Step 3: Using fused HAP feature data and enhanced HAP feature data, anomaly detection algorithms are used to detect early anomalous signals and perform multimodal attribution analysis to obtain attribution results;

[0113] It should be understood that this step involves configuring and running anomaly detection algorithms to identify early, weak signals indicating an individual's physiological abnormalities or nutritional imbalances, and to perform attribution analysis on the detected anomalies.

[0114] Step 3.1, Early Abnormal Signal Detection;

[0115] An anomaly detection algorithm based on contrastive learning to reconstruct the error model is employed.

[0116] First, utilize large-scale normal Train an autoencoder model using data. The autoencoder model contains an encoder part. and a decoder section The autoencoder model employs a multilayer perceptron structure, with the encoder... It contains several fully connected layers, with the number of neurons decreasing layer by layer (e.g., from the input dimension). →128→64→32), ultimately outputting a low-dimensional latent representation; Decoder The structure is symmetrical to the encoder, and the number of neurons increases layer by layer (e.g., 32→64→128→). The final output is the reconstructed feature vector; all hidden layers can use the ReLU activation function.

[0117] When training an autoencoder model, the objective is to minimize the reconstruction loss, specifically the mean squared error loss:

[0118] ;

[0119] in This represents the mean squared error loss function; Indicates the first Fusion HAP feature data of training samples, Indicates the number of training samples; Indicates the sample index; This refers to the encoder portion in the self-encoder model; This refers to the decoder portion in the autoencoder model; This represents the square of the L2 norm, which is the square of the Euclidean distance. Indicates the summation symbol;

[0120] For a new input data The reconstructed data after passing through the autoencoder model is as follows:

[0121] ;

[0122] in This refers to the encoder portion in the self-encoder model; This refers to the decoder portion in the autoencoder model; This represents the fused HAP feature data of the new input; This represents the data after the new input data has been reconstructed by the autoencoder;

[0123] Subsequently, the reconstruction error between the fused HAP feature data of the new input and the data reconstructed by the autoencoder is calculated:

[0124] ;

[0125] in This represents the reconstruction error between the input data after reconstruction by the autoencoder and the original input data. This represents the fused HAP feature data of the new input; This represents the data after the new input data has been reconstructed by the autoencoder; This represents the square of the L2 norm, which is the square of the Euclidean distance.

[0126] A pre-defined reconstruction error threshold is used to determine whether a reconstruction is normal. (It should be noted that the reconstruction error threshold is determined using a dynamic threshold adjustment strategy based on sliding window statistics to accommodate potential slow changes or seasonal fluctuations in data distribution.) Therefore, when the calculated... Greater than the preset When, then determine As an abnormal signal, mark it as .

[0127] Step 3.2, Multimodal attribution analysis;

[0128] When an abnormal signal is detected The system then initiates a multimodal attribution analysis program, the purpose of which is to preliminarily determine which types of data contributed the most to triggering this anomaly warning.

[0129] One possible attribution approach is: if the multimodal attention fusion model (MAFM) described in step 1.3 uses an explicit attention mechanism, then the cause of the current anomalous signal can be directly analyzed. Correspondence In the data, each original modal feature The attention weights assigned Size, with high The mode of the value can be considered as the main contributing mode that causes the anomaly;

[0130] Another possible attribution approach is to compute the output of the anomaly detection model (e.g., the reconstruction error for an autoencoder). For the classifier, this is the probability value of classifying it as an anomalous category. (Regarding the feature vectors of each modality before fusion in step 1.3...) The gradient value, i.e., the calculation of the gradient value. or ,in This represents the eigenvector of the m-th mode; This represents the weight assigned to the m-th modality in the attention mechanism; Indicates reconstruction error; This represents the gradient of the reconstruction error with respect to the eigenvector of the m-th mode; This indicates the probability that a sample is judged as abnormal; This represents the gradient of the anomaly probability with respect to the eigenvector of the m-th mode.

[0131] Correspondingly, modes with larger gradient values, or specific feature dimensions within them, typically indicate a more significant impact on the current anomaly detection. Through this attribution analysis process, an explanatory report can be obtained regarding the contribution of each data mode to the currently detected anomalous signal. It can provide managers with initial clues for locating problems.

[0132] For example, in livestock farming practice, when the system detects an abnormal signal of "unexplained high fever" in an individual animal, In this case, the multimodal attribution analysis program can analyze the contribution of each data source. If the analysis results... Display, body temperature sensor data The contribution was the highest, and the audio data The contribution of cough sound features detected in text logs was also relatively high. The description of "lethargy" in the text has a moderate contribution, and the activity level data is relatively low. If no obvious abnormalities are found, the system can preliminarily determine that the "unexplained high fever" is more likely related to respiratory infection than to ordinary environmental stress, thus providing more valuable reference information for further diagnosis by veterinarians.

[0133] Step 4: Based on early abnormal signals and attribution results, continuously optimize the nutritional requirement prediction model and the feeding strategy optimization model.

[0134] The various types of data and analysis results generated in the preceding steps, such as the unified augmented data view generated in step 1, are used to... HAP augmentation data generated in step 2 and simulated physiological process data sequences and the early abnormal signals detected in step 3. and its corresponding attribution explanation report This provides comprehensive feedback and is applied to the core livestock and poultry nutrition requirement prediction model. and feeding strategy optimization model This allows for continuous iterative optimization of the model and a higher level of intelligent decision support. It should be noted that the nutritional requirement prediction model... This refers to a dataset trained using historical HAP data, individual information, and feeding data, capable of adapting to the current individual's... A computational model predicts the dynamic requirements of various nutrients (such as energy, protein, amino acids, vitamins, and minerals) based on other relevant information (such as age, breed, and physiological stage). The feeding strategy optimization model... This refers to being based on The predicted results, combined with existing databases of feed ingredient nutrients, cost information, and breeding objectives (such as fastest weight gain, lowest cost, and specific meat quality improvement), generate computational models for specific, executable daily or phase-specific feed formulations and feeding amounts using optimization algorithms such as linear programming and reinforcement learning. These two models are the core decision-making components of the intelligent nutrition management system.

[0135] Step 4.1, Model Feedback and Iterative Optimization;

[0136] Specifically, a comprehensive dataset containing real HAP data, enhanced HAP data, and simulated process HAP data will be used to evaluate the nutritional requirement prediction model. and feeding strategy optimization model Perform periodic retraining and parameter fine-tuning.

[0137] In particular, when using When data sequences are used to stress-test the model and significant deviations in predictions are found within certain dynamic ranges, the model structure is adjusted or its loss function is optimized to improve the model's adaptability to dynamic changes. Simultaneously, early abnormal signals are detected. and its attribution report These can serve as new training samples or important auxiliary information (for example, as additional features for model input or to guide the model to pay more attention to changes in specific HAP indicators), thereby helping the model learn and identify these early, subtle abnormal patterns, and thus improving its ability to predict and identify potential health risks.

[0138] Step 4.2, Intelligent Decision Support;

[0139] Nutritional demand prediction model after continuous optimization and feeding strategy optimization model It can provide strong decision support for individualized, precise and forward-looking nutritional management of livestock and poultry.

[0140] For example, when nutritional requirement prediction models Combined with the latest When data predicts that an individual may be at risk of developing a deficiency in a specific nutrient in the future, the system can issue an early warning and optimize the feeding strategy model accordingly. Recommend corresponding feed formulation adjustment plans. For example, when the system detects, through the analysis in step 3, early symptoms similar to those of a certain disease in the past... and combined When the attribution results point to an abnormality in a specific HAP indicator, in addition to recommending nutritional adjustments, the system can also prompt animal husbandry staff to closely monitor the individual or arrange specific veterinary examinations. This decision support aims to transform the traditional reactive management model into a more proactive preventative management model.

[0141] The intelligent nutrition pattern management system based on HAP multimodal data is used to execute the above-mentioned intelligent nutrition pattern management method based on HAP multimodal data, including:

[0142] The data processing module is used to acquire multi-source heterogeneous HAP data of livestock and poultry. It uses a pre-trained large model to extract features from the multi-source heterogeneous HAP data and performs feature fusion through a multimodal attention fusion model to obtain fused HAP feature data.

[0143] The data generation module is used to generate enhanced HAP feature data or HAP feature data sequences that simulate physiological processes based on fused HAP feature data and using a controllable generative adversarial network.

[0144] The anomaly analysis module is used to detect early anomalous signals and perform multimodal attribution analysis by utilizing fused HAP feature data and enhanced HAP feature data, and obtain attribution results through anomaly detection algorithms.

[0145] The model optimization module is used to continuously optimize the nutrient requirement prediction model and the feeding strategy optimization model based on early abnormal signals and attribution results.

[0146] Here, the present invention provides an implementation example:

[0147] This application example simulates a modern pigsty environment with 100 fattening pigs, set in winter, focusing on risk monitoring and intelligent nutritional management intervention for early respiratory diseases (such as influenza) in the pig herd. The pigsty is equipped with smart ear tags (to collect individual body temperature and activity levels), RFID feeding troughs (to record feed intake and feeding time), high-definition cameras (to monitor group behavior and individual status), and microphone arrays (to collect ambient sounds and pig coughs, etc.). Farmers record key observation information (such as mental state and fecal condition) daily via mobile devices.

[0148] Implementation process example:

[0149] The system collected the following representative HAP data:

[0150] Sensor data: Most pigs' body temperatures fluctuated between 38.5℃ and 39.2℃, and their activity levels were normal. However, the body temperatures of three pigs (numbered A, B, and C) showed a slow upward trend over the past 12 hours, reaching 39.5℃, 39.6℃, and 39.4℃ respectively, and their activity levels decreased by approximately 15% compared to the previous day's average. Feed intake data showed that these three pigs' daily feed intake was 20% to 25% lower than the group average. Audio data: The background noise in the pigsty was stable, but the microphone array captured short, dry coughing sounds that were about 30% more frequent than usual near pigs numbered B and C. Text log data: The farmer's electronic log for the day recorded: "Pig number A is slightly lethargic and not eating actively; pig number B coughs occasionally; the overall drinking behavior of the pigs is normal." Image / video data: Camera footage showed that pig number A spent a long time lying in a corner and interacted less with other pigs.

[0151] The system calls the corresponding pre-trained large models (such as BERT for text, PANNs for audio, Transformer encoders for sensor time series, and ResNet for images) to extract feature vectors from the above-mentioned data types. Subsequently, a multimodal attention fusion model fuses these feature vectors. In this specific case, because the text directly mentions "slightly poor mental state" and "not actively eating," the audio detects a clear "coughing sound," and the sensor data also shows an increase in body temperature and a decrease in feed intake, the MAFM model assigns relatively high attention weights to these more indicative modalities (text, audio, and specific sensor indicators) during fusion, generating fused HAP feature data for pigs A, B, and C. , , These feature vectors can comprehensively reflect its current potential abnormal state.

[0152] Data augmentation for scarce scenarios: Since real HAP data samples of early influenza symptoms in swine herds may not be sufficient to adequately train the anomaly detection model, the system calls the CGAN model, using "early swine influenza" as the scenario condition. Generate a batch of simulated enhanced HAP feature data. These data simulate a comprehensive HAP pattern in terms of feature distribution, characterized by a gradual increase in body temperature (e.g., from 39.0°C to 40.5°C), accompanied by coughing (a specific audio feature pattern), decreased food intake, and text descriptions containing keywords such as "lethargy" and "rapid breathing."

[0153] Physiological Process Simulation Generation: To test the sensitivity and effectiveness of the nutrition management model in responding to the progressive development of influenza symptoms, the system also needs to generate simulations of the dynamic development of the disease from its initial stage to obvious symptoms. First, the system determines the potential vector of health status. (Representing normal body temperature, activity level, and food intake) and potential vectors of disease states (Representing obvious influenza symptoms). Subsequently, a series of simulated HAP feature data sequences were generated through linear interpolation. This set of data sequences simulated the entire process of pigs gradually transitioning from a healthy state to obvious influenza symptoms, including the dynamic changes of key indicators such as a gradual increase in body temperature, a gradual decrease in activity level, and a gradual increase in cough frequency.

[0154] The system integrates the HAP feature data of pigs A, B, and C. The input is fed into a trained autoencoder model for anomaly detection:

[0155] Anomaly detection: The autoencoder model attempts to reconstruct the fused HAP feature data of the input, generating reconstructed data. And calculate the reconstruction error , , Compare these reconstruction errors with a preset threshold. (This threshold was determined based on the 99th percentile of the reconstruction error of historical normal pig HAP data.) The system found that the reconstruction errors of all three pigs exceeded the threshold, with pig B having the highest reconstruction error. Therefore, the system marked these three pigs with early abnormal signals. , , .

[0156] By analyzing the attention weights of each modality in the MAFM model, the system found that for pig A, the body temperature sensor data and the text description of "slightly poor mental state" contributed the most; for pigs B and C, the coughing pattern in the audio data and the body temperature sensor data contributed significantly.

[0157] The above results were further verified by calculating the gradient of the anomaly detection model output (reconstruction error) with respect to each modal feature, and it was found that the cough sound features of B and C made a particularly significant contribution to the anomaly determination.

[0158] The system generates attribution reports. Preliminary findings suggest that pig A may be in the early stages of the disease, mainly exhibiting decreased mental state and a slight increase in body temperature; pigs B and C have more obvious respiratory symptoms, especially coughing, which are consistent with typical early influenza symptoms.

[0159] Model feedback and iterative optimization: The system integrates the HAP feature data of pigs A, B, and C. The generated augmented data and data sequences simulating disease progression Along with the detected early abnormal signals and attribution results Input into the nutrition requirement prediction model Iterative optimization is performed. Specifically, the system utilizes the generated sequence of gradual change process data. Stress tests were conducted on the model, and it was found that the model's prediction results were biased in the body temperature range of 39.8℃-40.2℃. Therefore, the part of the model structure responsible for handling high body temperature was adjusted and the loss function was optimized to improve the model's adaptability to pathological physiological changes.

[0160] Intelligent decision support: based on an optimized nutritional requirement prediction model and feeding strategy optimization model The system proposed personalized nutritional intervention plans for pigs A, B, and C:

[0161] It is recommended to increase the electrolyte content in the drinking water of all three pigs to ensure fluid balance;

[0162] For pigs B and C, which have shown obvious respiratory symptoms, it is recommended to increase the content of vitamins C and E in their feed (to 150% of the standard content) to enhance their immunity;

[0163] For pig A, it is recommended to increase feed palatability and feed it in small amounts multiple times to prevent its feed intake from decreasing further.

[0164] The system also advised the farmers to closely observe the development of symptoms in these three pigs and to isolate and monitor pigs that had been in close contact with them to prevent the spread of the disease.

[0165] A week later, follow-up results showed that, thanks to timely nutritional adjustments and health management interventions, pig A fully recovered, and the symptoms of pigs B and C were effectively controlled, preventing them from developing into severe cases and the disease from spreading further within the herd. This case validates the effectiveness of the intelligent nutritional model management method based on HAP multimodal data in early disease warning and precise nutritional intervention, especially its ability to sensitively detect weak abnormal signals and its proactive intervention capabilities.

[0166] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. An intelligent nutrition model management method based on HAP multimodal data, characterized in that, include: Multi-source heterogeneous HAP data of livestock and poultry were acquired, and a pre-trained large model was used to extract features from the multi-source heterogeneous HAP data. The features were then fused using a multimodal attention fusion model to obtain fused HAP feature data. The acquisition of multi-source heterogeneous HAP data of livestock and poultry includes structured sensor data, structured text, audio data, image data, and video data; The core component of the multimodal attention fusion model is the attention module, which dynamically assigns attention weights based on the importance of different modal features in representing the current individual state. The specific steps are as follows: First, the feature vectors of each modality are... Projected onto the same dimension through their respective independent MLP layers. ; Then, all projected modal features are concatenated or summed, and then input into another MLP to calculate the attention score for each modality. The attention weights are then obtained by normalization using the Softmax function. ; Based on fused HAP feature data, a controllable generative adversarial network (GAN) is used to generate enhanced HAP feature data and HAP feature data sequences simulating physiological processes. Specifically, the use of the controllable GAN includes: Using a generator model, based on the input random noise vector, preset scenario conditions, and the potential state vector corresponding to the preset physiological indicator change path, enhanced HAP feature data and simulated physiological process HAP feature data sequences are generated. Using a discriminator model, we can distinguish between real HAP feature data and HAP feature data generated by a generator model, and optimize the generator model and discriminator model through an adversarial training process. The generator model Discriminator Model A network structure based on a multilayer perceptron is adopted; an adversarial loss function is used during training. ； in Represents conditional information. This represents the generator model in a generative adversarial network. This represents a discriminator model in a generative adversarial network. This represents the random noise vector input to the generator; Represents the loss function of the CGAN model; This indicates that the generator G is being optimized by minimization. This indicates that the discriminator D is optimized to the maximum extent. This represents the expectation under the true data distribution; This represents the expectation under the noise distribution; This represents the logarithmic probability of the discriminator on the true data under condition c; This represents the log probability that the discriminator classifies the generated data as false under condition c; By utilizing fused HAP feature data and enhanced HAP feature data, anomaly detection algorithms are used to detect early anomalous signals and perform multimodal attribution analysis to obtain attribution results. The testing steps include: The input HAP feature data is reconstructed using an autoencoder model to obtain reconstructed HAP feature data. Calculate the reconstruction error between the input HAP feature data and the reconstructed HAP feature data; When the reconstruction error exceeds a preset error threshold, it is determined that an early abnormal signal has been detected. The preset error threshold is determined by a dynamic threshold adjustment strategy based on sliding window statistics. An anomaly detection algorithm based on contrastive learning to reconstruct the error model is employed. First, an autoencoder model is trained using large-scale normal data. The autoencoder model contains an encoder part. and a decoder section ; For a new input data The reconstructed data after passing through the autoencoder model is as follows: ； in This refers to the encoder portion in the self-encoder model; This refers to the decoder portion in the autoencoder model; This represents the reconstructed HAP feature data after the new input data has been reconstructed by the autoencoder; Then, calculate the new input. The reconstruction error between the new input data and the reconstructed HAP feature data after being reconstructed by the autoencoder: ； in This represents the reconstruction error between the input data after reconstruction by the autoencoder and the original input data. This represents the reconstructed HAP feature data after the new input data has been reconstructed by the autoencoder; This represents the square of the L2 norm, which is the square of the Euclidean distance. When an abnormal signal is detected Next, multimodal attribution analysis is initiated, with the aim of initially determining which types of data contributed the most to triggering this anomaly alert; Multimodal attribution analysis includes: When the multimodal attention fusion model adopts an explicit attention mechanism, the magnitude of the attention weights corresponding to each original modality feature vector in the fused HAP feature data that leads to early anomalous signals is analyzed to determine the modality that contributes the most to the early anomalous signals. The gradient value of the output result of the anomaly detection algorithm with respect to the feature vector of each original mode in the fused HAP feature data is calculated. The mode with the larger gradient value is determined to be the mode that contributes more to the early anomaly signal. Based on early anomalous signals and attribution results, the nutritional requirement prediction model and feeding strategy optimization model are continuously optimized, including: using a comprehensive dataset containing real HAP data, enhanced HAP data, and simulated process HAP data for the nutritional requirement prediction model. and feeding strategy optimization model Perform periodic retraining and parameter fine-tuning; The nutritional requirement prediction model is used to predict the dynamic requirements of an individual for multiple nutrients based on the individual's fused HAP feature data and other relevant individual information. The feeding strategy optimization model is used to generate specific feed formulas and recommended feeding amounts based on dynamic demand predicted by the nutritional requirement prediction model, combined with feed ingredient information and preset breeding goals, through optimization algorithms.

2. An intelligent nutrition pattern management system based on HAP multimodal data, characterized in that, The method for implementing the intelligent nutrition pattern management method based on HAP multimodal data as described in claim 1 includes: The data processing module is used to acquire multi-source heterogeneous HAP data of livestock and poultry. It uses a pre-trained large model to extract features from the multi-source heterogeneous HAP data and performs feature fusion through a multimodal attention fusion model to obtain fused HAP feature data. The data generation module is used to generate enhanced HAP feature data or HAP feature data sequences that simulate physiological processes based on fused HAP feature data and using a controllable generative adversarial network. The anomaly analysis module is used to detect early anomalous signals and perform multimodal attribution analysis by utilizing fused HAP feature data and enhanced HAP feature data, and obtain attribution results through anomaly detection algorithms. The model optimization module is used to continuously optimize the nutrient requirement prediction model and the feeding strategy optimization model based on early abnormal signals and attribution results.

Images