Intelligent nutrition mode management method and system based on HAP multi-modal data

Abstract

The invention relates to the cross technical field of computers and livestock breeding, and discloses an intelligent nutrition mode management method and system based on HAP multi-modal data, and the method comprises the steps: obtaining the multi-source heterogeneous HAP data of livestock, and carrying out the feature extraction and feature fusion of the multi-source heterogeneous HAP data; utilizing a controllable generative adversarial network to generate enhanced HAP feature data and an HAP feature data sequence simulating a physiological process; early abnormal signals are detected through an abnormal detection algorithm, and multi-modal attribution analysis is carried out; continuously optimizing the nutrition demand prediction model and the feeding strategy optimization model; according to the method, by effectively utilizing the neglected unstructured information in the past and combining the multi-modal fusion analysis, the early-stage discovery capability of the weak signal for indicating the physiological abnormality or nutritional imbalance of the livestock and poultry is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intersection between computers and animal husbandry, and more specifically, to an intelligent nutrition model management method and system based on HAP multimodal data. Background Art

[0002] In modern intensive livestock and poultry farming, intelligent nutrition management is crucial to improving production efficiency and ensuring animal welfare.

[0003] Existing technologies primarily rely on structured sensor data (such as body temperature and activity level) for analysis. However, high-quality data is scarce in critical physiological scenarios (such as early disease stages and dramatic environmental changes), resulting in insufficient model robustness. Furthermore, existing technologies fail to fully integrate and utilize early abnormality signals contained in unstructured information such as text records and audio generated during the farming process. Furthermore, existing technologies lack the means to accurately simulate and validate HAP (High Potential, Active, and Preventive) data for individual livestock and poultry based on their varying growth potential, health status, and abnormal development. These deficiencies limit the ability to provide early and accurate warnings of livestock and poultry health risks, hinder the sophistication and foresight of intelligent nutritional management, and constitute a major technical challenge that urgently needs to be addressed in the field of intelligent livestock farming.

[0004] These shortcomings limit the ability to provide early and accurate warning of livestock and poultry health risks, affect the refinement and foresight of intelligent nutrition management, and constitute a major technical problem that urgently needs to be solved in the current field of intelligent animal husbandry. Summary of the Invention

[0005] The present invention provides an intelligent nutrition pattern management method and system based on HAP multimodal data, which solves the technical problems in related technologies such as weak early anomaly detection capabilities and poor robustness of nutrition models caused by data scarcity, insufficient utilization of unstructured information, and lack of dynamic process simulation and verification.

[0006] The present invention provides an intelligent nutrition model management method based on HAP multimodal data, comprising:

[0007] Acquire multi-source heterogeneous HAP data of livestock and poultry, use a pre-trained large model to extract features from the multi-source heterogeneous HAP data, and fuse the features through a multimodal attention fusion model to obtain fused HAP feature data;

[0008] Based on the fused HAP feature data, a controllable generative adversarial network is used to generate enhanced HAP feature data and a HAP feature data sequence that simulates the physiological process. The controllable generative adversarial network specifically includes:

[0009] Using the generator model, based on the input random noise vector, preset scene conditions, and the potential state vector corresponding to the preset physiological indicator change path, enhanced HAP feature data and HAP feature data sequences that simulate physiological processes are generated;

[0010] Using the discriminator model, we distinguish between real HAP feature data and HAP feature data generated by the generator model, and optimize the generator and discriminator models through an adversarial training process.

[0011] By using fused and enhanced HAP feature data, anomaly detection algorithms are used to detect early abnormal signals and perform multimodal attribution analysis to obtain attribution results.

[0012] Based on early abnormal signals and attribution results, the nutritional demand prediction model and feeding strategy optimization model are continuously optimized.

[0013] Furthermore, the multi-source heterogeneous HAP data of livestock and poultry are obtained, including structured sensor data, structured text, audio data, image data and video data.

[0014] Furthermore, the core component of the multimodal attention fusion model is the attention module, which dynamically allocates attention weights according to the importance of different modal features in characterizing the current individual state. The specific steps are as follows:

[0015] First, the eigenvectors of each mode Projected to the same dimension through independent MLP layers ;

[0016] Then all the projected modal features are concatenated or summed, and then input into another MLP to calculate the attention score of each modality, and the attention weight is obtained by normalization through the Softmax function. .

[0017] Furthermore, the anomaly detection algorithm is specifically an autoencoder model based on reconstruction error, and the detection steps include:

[0018] The input HAP feature data is reconstructed using the autoencoder model to obtain reconstructed HAP feature data;

[0019] Calculating a reconstruction error between the input HAP feature data and the reconstructed HAP feature data;

[0020] When the reconstruction error is greater than a preset error threshold, it is determined that an early abnormal signal is detected.

[0021] Furthermore, the preset error threshold is determined by a dynamic threshold adjustment strategy based on sliding window statistics.

[0022] Furthermore, the anomaly detection algorithm detects early abnormal signals and performs multimodal attribution analysis, including:

[0023] When the multimodal attention fusion model adopts an explicit attention mechanism, the attention weights corresponding to the feature vectors of each original modality in the fused HAP feature data that causes the early abnormal signal are analyzed to determine the modality that contributes most to the early abnormal signal;

[0024] The gradient value of the output result of the anomaly detection algorithm with respect to each original modal feature vector in the fused HAP feature data is calculated, and the mode with a larger gradient value is determined as the mode that contributes more to the early abnormal signal.

[0025] Furthermore, the continuous optimization of the nutrient requirement prediction model and the feeding strategy optimization model includes: using a comprehensive data set including real HAP data, enhanced HAP data and simulated process HAP data to optimize the nutrient requirement prediction model. and feeding strategy optimization models Perform periodic retraining and parameter fine-tuning.

[0026] Furthermore, the nutritional demand prediction model is used to predict the dynamic demand for multiple nutrients of the current individual based on the fused HAP feature data and other relevant individual information.

[0027] The feeding strategy optimization model is used to generate specific feed formula and feeding amount recommendations through an optimization algorithm based on the dynamic demand predicted by the nutritional demand prediction model, combined with feed raw material information and preset breeding goals.

[0028] The present invention provides an intelligent nutrition pattern management system based on HAP multimodal data, which is used to implement the above-mentioned intelligent nutrition pattern management method based on HAP multimodal data, including:

[0029] The data processing module is used to obtain multi-source heterogeneous HAP data of livestock and poultry, extract features from the multi-source heterogeneous HAP data using a pre-trained large model, and fuse the features using a multimodal attention fusion model to obtain fused HAP feature data;

[0030] A data generation module is used to generate enhanced HAP feature data or HAP feature data sequences that simulate physiological processes based on the fused HAP feature data using a controllable generative adversarial network;

[0031] The anomaly analysis module is used to use the fused HAP feature data and enhanced HAP feature data to detect early abnormal signals through an anomaly detection algorithm and perform multimodal attribution analysis to obtain attribution results;

[0032] The model optimization module is used to continuously optimize the nutritional demand prediction model and feeding strategy optimization model based on early abnormal signals and attribution results.

[0033] The beneficial effects of this invention are: by effectively utilizing previously ignored unstructured information such as text and audio, and combining it with multimodal fusion analysis, it improves the ability to early detect weak signals indicating physiological abnormalities or nutritional imbalances in livestock and poultry, buying valuable time for preventive health management and timely nutritional intervention;

[0034] By generating high-quality, scenario-specific (including dynamically changing processes) HAP simulation data, the scarcity of real data is effectively addressed, enabling the nutrient management model to maintain stable and accurate performance under a wider range of farming conditions. This enhances the model's robustness and scenario adaptability, especially when dealing with rare or extreme situations.

[0035] Intelligent abnormal signal detection and preliminary attribution analysis can provide managers and veterinarians with clearer problem identification, helping them make more accurate nutritional adjustments or health management decisions. This also provides a more intuitive understanding of the basis for decision-making, thereby improving the accuracy and explainability of decisions.

[0036] Through a more comprehensive and in-depth understanding of the physiological status of livestock and poultry, and the continuous iterative optimization of model capabilities, the intelligent nutrition management model has been promoted to develop in a more refined (individualized demand satisfaction) and more forward-looking (predictive intervention based on early signals) direction, which will ultimately help improve breeding efficiency and animal health and welfare. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the intelligent nutrition pattern management method based on HAP multimodal data in the present invention;

[0038] Figure 2 is a flow chart of step 1 in the present invention;

[0039] Figure 3 It is a flow chart of step 2 in the present invention;

[0040] Figure 4 It is a flow chart of step 3 in the present invention;

[0041] Figure 5 It is a flow chart of step 4 in the present invention. DETAILED DESCRIPTION

[0042] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed solely to enable those skilled in the art to better understand and implement the subject matter described herein, and that the functions and arrangements of the elements discussed may be varied without departing from the scope of this specification. Various examples may omit, substitute, or add various processes or components as needed. Furthermore, features described in some examples may be combined in other examples.

[0043] At least one embodiment of the present invention discloses an intelligent nutrition pattern management method based on HAP multimodal data, such as Figures 1 to 5 As shown, including:

[0044] Step 1: Obtain multi-source heterogeneous HAP data of livestock and poultry, use a pre-trained large model to extract features from the multi-source heterogeneous HAP data, and fuse the features through a multimodal attention fusion model to obtain fused HAP feature data;

[0045] Specifically include:

[0046] Step 1.1: Multi-source heterogeneous HAP data acquisition and time series alignment;

[0047] Through the various sensors and recording systems deployed in the farms, we comprehensively collect multi-source heterogeneous HAP data related to livestock and poultry. During this data collection process, the data content of the three dimensions of HAP is specifically reflected as follows:

[0048] High growth potential dimension (High): Individual growth potential indicators are collected through genetic background analysis and growth performance monitoring sensors, combined with variety information and growth performance descriptions in text records;

[0049] Active health dimension: collecting body temperature through wearable sensors , heart rate , activity level Physiological indicators such as abnormal sounds in images, video data and audio data obtained by the machine vision system ;

[0050] Sub-health nutrition prevention dimension (Preventive): Through the breeder's observation log and veterinary diagnostic information to record individual nutritional status, immunity level, stress response and other preventive health management indicators.

[0051] The data acquisition system includes: wearable sensors for collecting physiological indicators, machine vision systems for acquiring image or video information, and sensors for capturing environmental sounds. and individual voices A microphone array and an electronic recording system for recording text messages. All collected raw data are time-stamped. , and perform unified time axis alignment and standardization preprocessing to form a time series dataset containing various types of original HAP data records:

[0052] ;

[0053] in Indicates the The body temperature of individual livestock and poultry in the records; Indicates the Heart rate of individual livestock and poultry in the records; Indicates the The activity amount of individual livestock and poultry in each record can be the quantitative value of the number of steps or activity intensity; Indicates the Ambient sound data in records, such as audio recordings of equipment noise; Indicates the Individual sound data in records, such as audio records of abnormal calls of livestock and poultry; Indicates the Text data recorded by the breeder in each record, such as observation logs; Indicates the Image data in records; Indicates the Video data in records; Indicates the The timestamp of the record; The index number of the data record, which is used to uniquely identify each data record; Represents a time series dataset containing all raw HAP data records.

[0054] Step 1.2, multimodal feature extraction based on pre-trained large model;

[0055] For various HAP data after time series alignment, the corresponding pre-trained large model is called to perform deep feature extraction, the purpose of which is to convert the raw data into high-dimensional feature vectors that are easier for machine analysis and processing. Specifically:

[0056] For text data , a bidirectional encoder representation method (Bidirectional Encoder Representations from Transformers, BERT) model can be used; the BERT model is a bidirectional encoder based on Transformer, which uses its text understanding ability to transform the text data Input into a pre-trained BERT model and obtain the output corresponding to the [CLS] tag of its final hidden layer as text data The eigenvector of .

[0057] For audio data 、 , a PANNs (Large-Scale Pretrained Audio Neural Networks) model can be used, which usually includes a Convolutional Neural Networks (CNN) layer for extracting time-frequency spectrum features; the audio data is input into the pre-trained PANNs model, and the output of a layer before the final fully connected layer is extracted as the audio feature vector and .

[0058] For structured sensor time series data 、 、 , build a Transformer-based encoder model, including a multi-head self-attention layer and a feedforward neural network layer; take the multivariate sensor sequence data within a predetermined time window as input, convert it into a vector representation through the embedding layer, and input it into the Transformer encoder, extracting the average pooling result of its output sequence or the output of a specific tag as the sensor data feature vector ;

[0059] For image data , using the Residual Network (ResNet), after inputting the image data, extract the output of its average pooling layer as the image feature vector .

[0060] For video data , we can first extract image features frame by frame, and then further aggregate them into video feature vectors through time pooling or time convolution network .

[0061] Step 1.3, deep fusion of multimodal features;

[0062] Construct a multi-modal attention fusion model (MAFM) to extract the feature vectors of multiple modalities (i.e. ) for weighted fusion.

[0063] The core component of the multimodal attention fusion model is an attention module, which can dynamically assign attention weights based on the importance of different modal features in representing the current individual state. The multimodal attention fusion model can be specifically implemented as a structure containing a multilayer perceptron (MLP):

[0064] First, the eigenvectors of each mode are Projected to the same dimension through independent MLP layers (including two fully connected layers, using ReLU activation function) ;

[0065] Then, all the projected modal features are concatenated or summed, and then input into another MLP to calculate the attention score of each modality, and the attention weight is obtained by normalizing it through the Softmax function. .

[0066] Therefore, for any data record , whose fusion feature vector The following can be obtained by processing the MAFM model:

[0067] ;

[0068] in Indicates that from The feature vectors extracted from the text data of the records; Indicates that from Feature vectors extracted from the environmental sound data of the records; Indicates that from Feature vectors extracted from individual sound data of records; Indicates that from Feature vectors extracted from the sensor data of records; Indicates that from Feature vectors extracted from the image data of the records; Indicates that from Feature vectors extracted from the video data of records; A function representing a multimodal attention fusion model; Indicates the The fused feature vector of the records;

[0069] The MAFM model contains one or more such attention layers; these attention layers are used to calculate the features of each modality (Here represents the attention weights of different modalities) , and then sum the weighted features:

[0070] ;

[0071] in Indicates the The fused feature vector of the records, Represents the corresponding mode The learnable projection matrix (in the above MLP implementation, this projection is included in the feature transformation before the MLP layer of each modality and the final weighted summation) is used to map the features of different modalities to a unified feature dimension space. ; Indicates the Modal in record The attention weights are calculated based on the input features of all modalities through an auxiliary neural network (the MLP used to calculate the attention score mentioned above); Indicates the Modal in record The eigenvector of Represents the summation operation, for each mode , first its characteristics Through the corresponding projection matrix Mapped to a unified fusion space, then multiplied by the modality The attention weight Finally, the weighted features of all modalities are summed to obtain the final fusion feature .

[0072] Through this step of processing, each data record is finally Generate a more information-rich, comprehensive characterization of Unified enhanced data view of the moment status , whose dimensions are .

[0073] For example, in one application scenario, when the text log data of a livestock individual The information of "significant decrease in feed intake" was recorded in the data, and the corresponding sensor data When the activity level is still maintained at a normal level, the MAFM model can learn the features that should be given to the text modality in this case through its attention module. Higher attention weight , so that the final fusion feature It can better reflect potential digestive system abnormalities or early disease signals indicated by text information, even if other sensor modality data have not yet shown synchronous obvious abnormalities.

[0074] Step 2: Based on the fused HAP feature data, a controllable generative adversarial network is used to generate enhanced HAP feature data and HAP feature data sequences that simulate physiological processes;

[0075] According to some embodiments of the present application, this step aims to apply a Controllable Generative Adversarial Network (CGAN) model to generate high-quality HAP-enhanced data or HAP data sequences that simulate specific physiological processes, based on the specific needs of intelligent nutrition management models. During the data generation process, the system performs refined classification of HAP data based on dynamic characteristics such as individual growth potential, changes in health status, and the development of abnormal conditions, ensuring that the generated data accurately reflects the physiological characteristics and changing patterns of different individual types.

[0076] Specifically, based on the differences in individual growth potential, HAP data are divided into:

[0077] High growth potential type (data characteristics reflecting excellent genetic background and ability to gain weight quickly);

[0078] Medium growth potential type (growth performance data at the average level of the group);

[0079] Low growth potential type (data characteristics of slow growth or genetic defects).

[0080] Based on changes in health status, it is divided into:

[0081] Healthy and stable type (physiological indicators fluctuate stably within the normal range);

[0082] Healthy fluctuating type (mild fluctuations but not reaching the disease threshold);

[0083] Declining health (a gradual deterioration in health);

[0084] Healthy recovery type (data pattern of recovery from illness or stress).

[0085] Based on the development of abnormal situations, it is divided into:

[0086] Acute abnormal type (data characteristics of sudden health problems);

[0087] Chronic abnormal type (data characteristics of long-term sub-health status);

[0088] Periodic abnormality (abnormal regularity of a specific physiological cycle);

[0089] Warning abnormality type (early weak signal during the incubation period of the disease).

[0090] It should be noted that the CGAN model usually includes a generator model and a discriminator model .

[0091] Specifically, the generator model and the discriminator model A network structure based on a multi-layer perceptron can be used. Generator model The input layer receives a random noise vector and conditional information (such as scene tags or interpolated latent vector ), which may be followed by, for example, three, Fully connected hidden layer, each layer can contain, for example, 256 neurons, and use ReLU or LeakyReLU as the activation function; the number of neurons in the output layer is the same as the target HAP fusion feature data The dimensions are consistent and based on Select an appropriate activation function based on the numerical range of the feature (for example, if has been normalized to -1 to 1, then the Tanh activation function can be used). Accordingly, the discriminator model The input layer receives HAP fusion feature data (real or generated ) and conditional information (If applicable), it can also be followed by, for example, three fully connected hidden layers, each layer can contain, for example, 256 neurons, and also use ReLU or LeakyReLU activation function; the output layer is one neuron, and it outputs a scalar value between 0 and 1 through the Sigmoid activation function, indicating the probability that the input data is true data.

[0092] When training the CGAN model, an adversarial loss function can be used, such as the standard minimum-maximum loss function:

[0093] ;

[0094] in Represents condition information, Represents the generator model in the generative adversarial network; Represents the discriminator model in the generative adversarial network; Represents the random noise vector input to the generator; represents the latent vector obtained by interpolation; represents the real HAP fusion feature data; Represents the generated HAP fusion feature data; Represents the loss function of the CGAN model; Indicates minimization optimization of generator G; Indicates maximization optimization of the discriminator D; Represents the expectation under the real data distribution; represents the expectation under the noise distribution; Represents the logarithmic probability of the discriminator for the real data under condition c; represents the logarithmic probability that the discriminator judges the generated data to be false under condition c;

[0095] The optimization process can use, for example, the Adam optimizer, set the learning rate to 0.0002, and the batch size to 64 for iterative training until the model converges.

[0096] Step 2.1: HAP data enhancement for key scarce scenarios;

[0097] For some physiological scenarios in the breeding process that occur less frequently but are of great significance to model training (for example, early clinical manifestations of specific diseases, early stress responses of animals to extreme environments), the amount of real HAP data samples in these scenarios is often insufficient. To address this problem, this application provides a method for generating HAP data in these scarce scenarios using the CGAN model. Specifically, the scene labels (For example, Can be set to "early respiratory infection") as a condition input to the generator model , and input a random noise vector .

[0098] Generator Model Then output the simulated HAP fusion feature data in this scenario:

[0099] ;

[0100] in Represents the generator model in the generative adversarial network; Represents the discriminator model in the generative adversarial network; represents the random noise vector provided as input to the generator model; represents a conditional label used to represent a specific scarce physiological scenario; Represents the generated HAP fusion feature data;

[0101] The feature distribution should be consistent with the actual situation in the scene. The data is similar.

[0102] Discriminator model It is used to distinguish real data from generated data and to promote the generator model through the adversarial training process. Producing more realistic data.

[0103] Generated in this way The data can be used to augment the original dataset to enhance the training of downstream nutrient management models.

[0104] Step 2.2, simulation of HAP data series of the gradual change process of specific physiological indicators;

[0105] In order to achieve a deep stress test of the dynamic response capability of the nutrition management model, it is necessary to simulate the physiological indicators of livestock and poultry (e.g., body temperature, ) HAP data representation during continuous and smooth changes. This application provides a method to utilize the control capability of the CGAN model to perform interpolation or attribute editing operations in the latent space of the trained GAN model. For example, the latent vectors corresponding to the "health state" can be obtained first. and the latent vector corresponding to the “high heat state” Then, by and Perform linear interpolation between:

[0106] ;

[0107] in represents the interpolation coefficient; A representation vector representing the health status in the latent space of the GAN model; A representation vector representing the hyperthermia state in the GAN model latent space; The latent vector obtained by interpolation;

[0108] A series of Vector input to the generator model In this way, we can obtain a set of HAP data sequences that can simulate the gradual increase of body temperature from normal state to hyperthermia state:

[0109] ;

[0110] in represents the simulated HAP data series, 、 、 Respectively represent the generator interpolation vector 、 、 The output, Represents the number of samples in the simulated data series.

[0111] This generated sequence It can be used as input data for downstream nutritional management models to evaluate their predictive stability and accuracy under dynamic physiological change conditions. A specific application example is that in sow farming scenarios, this method can be used to simulate the gradual physiological changes that usually occur in multimodal HAP data such as body temperature and activity level of sows during the week before delivery. By setting the latent space vectors corresponding to the starting state (such as 7 days before delivery, normal body temperature and moderate activity level) and the ending state (such as before delivery, body temperature may fluctuate slightly, activity level may decrease, or irritability), and performing smooth interpolation, a simulated HAP data sequence is generated. The HAP data sequence can then be used to test existing intelligent nutritional demand prediction models. Is it possible to accurately and dynamically adjust the recommended supplementation plan for key nutrients (such as energy, fiber, and specific vitamins) for sows before parturition based on these simulated gradual signals?

[0112] Step 3: Using the fused HAP feature data and the enhanced HAP feature data, an anomaly detection algorithm is used to detect early abnormal signals and perform multimodal attribution analysis to obtain attribution results.

[0113] It should be understood that this step is based on the unified enhanced data view generated in step 1 (This includes real HAP data and enhanced HAP data generated in step 2). By configuring and running anomaly detection algorithms, we can identify early, weak signals indicating individual physiological abnormalities or nutritional imbalances, and perform attribution analysis on the detected abnormalities.

[0114] Step 3.1, early abnormal signal detection;

[0115] Adopt an anomaly detection algorithm based on contrastive learning reconstruction error model:

[0116] First, using large-scale normal The data trains an autoencoder model, which contains an encoder part. and a decoder section ; The autoencoder model adopts a multi-layer perceptron structure, the encoder Contains several fully connected layers with decreasing number of neurons (e.g., from input dimension →128→64→32), and finally outputs a low-dimensional potential representation; the decoder The structure is symmetrical to the encoder, and the number of neurons increases layer by layer (for example, 32→64→128→ ), and finally output the reconstructed feature vector; all hidden layers can use the ReLU activation function;

[0117] When training the autoencoder model, the goal is to minimize the reconstruction loss, the mean square error loss:

[0118] ;

[0119] in represents the mean square error loss function; Indicates the The fused HAP feature data of training samples, Indicates the number of training samples; Indicates the sample index; represents the encoder part in the autoencoder model; represents the decoder part of the autoencoder model; Represents the square of the L2 norm, that is, the square of the Euclidean distance; represents the summation symbol;

[0120] For a new input data , and the reconstructed data after passing through the autoencoder model is:

[0121] ;

[0122] in represents the encoder part in the autoencoder model; represents the decoder part of the autoencoder model; Represents the newly input fused HAP feature data; Represents the data after the new input data is reconstructed by the autoencoder;

[0123] Then, the reconstruction error between the newly input fused HAP feature data and the data reconstructed by the autoencoder is calculated:

[0124] ;

[0125] in Represents the reconstruction error between the input data after reconstruction by the autoencoder and the original input data; Represents the newly input fused HAP feature data; Represents the data after the new input data is reconstructed by the autoencoder; It represents the square of L2 norm, that is, the square of Euclidean distance.

[0126] Pre-set a reconstruction error threshold to determine whether it is normal (It should be noted that the reconstruction error threshold is determined by a dynamic threshold adjustment strategy based on sliding window statistics to adapt to possible slow changes or seasonal fluctuations in data distribution.) Therefore, when the calculated Greater than the preset When is an abnormal signal and marked as .

[0127] Step 3.2, multimodal attribution analysis;

[0128] When an abnormal signal is detected After that, the system starts the multimodal attribution analysis program, whose purpose is to preliminarily determine which types of data contribute most to triggering this abnormal warning;

[0129] An optional attribution method is: if the multimodal attention fusion model MAFM described in step 1.3 adopts an explicit attention mechanism, then the cause of the current abnormal signal can be directly analyzed. Correspondence In the data, each original modal feature The assigned attention weight size, with a higher The mode of values can be considered as the main contributing mode that causes the anomaly;

[0130] Another alternative attribution is to compute the output of the anomaly detection model (e.g., for an autoencoder, the reconstruction error ; For the classifier, it is the probability value of being judged as an abnormal category) About each modal feature vector before fusion in step 1.3 The gradient value of or ,in represents the eigenvector of the mth mode; Represents the weight assigned to the mth modality in the attention mechanism; represents the reconstruction error; represents the gradient of the reconstruction error with respect to the mth modal eigenvector; Indicates the probability that a sample is judged as abnormal; represents the gradient of the abnormal probability with respect to the mth modal eigenvector.

[0131] Correspondingly, the mode with a larger gradient value or a specific feature dimension therein usually indicates that it has a more obvious influence on the judgment of the current anomaly. Through this attribution analysis process, an explanatory report on the contribution of each data mode to the currently detected abnormal signal can be obtained. , which can provide managers with preliminary clues to locate problems.

[0132] For example, in farming practice, when the system detects an abnormal signal of "unexplained high fever" in an individual livestock or poultry When the multimodal attribution parser is used, it can analyze the contribution of each data source. Display, body temperature sensor data The contribution is the highest, while audio data The contribution of cough sound features detected in the text log is also high. The contribution of the description of "mental depression" is average, and the activity data If there are no obvious abnormalities, the system can preliminarily determine that the "unexplained high fever" is more likely to be related to respiratory infection rather than ordinary environmental stress, thereby providing more valuable reference information for further diagnosis by the veterinarian.

[0133] Step 4: Continuously optimize the nutrient demand prediction model and feeding strategy optimization model based on early abnormal signals and attribution results;

[0134] Combine the various data and analysis results generated in the previous steps, such as the unified enhanced data view generated in step 1 , HAP enhanced data generated in step 2 and simulated physiological process data series , and the early abnormal signals detected in step 3 and its corresponding attribution explanation report , comprehensive feedback and application to the core livestock and poultry nutritional demand prediction model and feeding strategy optimization models In order to achieve continuous iterative optimization of the model and higher level of intelligent decision support. It should be noted that the nutritional demand prediction model It is obtained by training with historical HAP data, individual information and feeding data, and can be used according to the current individual A computational model that predicts the dynamic requirements of the pig for various nutrients (such as energy, protein, amino acids, vitamins, minerals, etc.) based on other relevant information (such as age, breed, and physiological stage). It means based on The system combines the predicted results with the existing feed ingredient nutrient database, cost information, and farming objectives (such as fastest weight gain, lowest cost, and specific meat quality improvement). Through optimization algorithms such as linear programming and reinforcement learning, it generates a computational model that recommends specific, actionable daily or phased feed formulas and feeding amounts. These two models are the core decision-making components of the intelligent nutrition management system.

[0135] Step 4.1, model feedback and iterative optimization;

[0136] Specifically, a comprehensive dataset including real HAP data, enhanced HAP data and simulated HAP data was used to evaluate the nutrient demand prediction model. and feeding strategy optimization models Perform periodic retraining and parameter fine-tuning.

[0137] In particular, when using When the model is stress-tested using data sequences and a large deviation in the prediction results of the model is found in certain dynamic change intervals, the model structure will be adjusted or its loss function will be optimized to improve the model's adaptability to dynamic changes. At the same time, the early abnormal signals detected and its attribution report , can serve as new training samples or important auxiliary information (for example, it can be used as additional features of model input, or used to guide the model to pay more attention to changes in specific HAP indicators), thereby helping the model learn and identify these early, weak abnormal patterns, thereby improving its ability to predict and identify potential health risks.

[0138] Step 4.2, intelligent decision support;

[0139] Nutritional demand prediction model after continuous optimization and feeding strategy optimization models , which can provide strong decision-making support for the individualized, precise and forward-looking nutritional management of livestock and poultry.

[0140] For example, when the nutrient requirement prediction model Combined with the latest When the data predicts that an individual may be at risk of a specific nutrient deficiency in the future, the system can issue an early warning and optimize the feeding strategy model. Recommend corresponding feed formula adjustment plan. For example, when the system detects similar early symptoms of a disease in history through the analysis of step 3, , and combined with When attribution results point to abnormalities in specific HAP indicators, in addition to recommending nutritional adjustments, the system can also prompt animal husbandry managers to conduct focused observations or schedule specific veterinary examinations for that individual. This type of decision support aims to transform the traditional reactive management model into a more proactive, preventative one.

[0141] The intelligent nutrition pattern management system based on HAP multimodal data is used to implement the above-mentioned intelligent nutrition pattern management method based on HAP multimodal data, including:

[0142] The data processing module is used to obtain multi-source heterogeneous HAP data of livestock and poultry, extract features from the multi-source heterogeneous HAP data using a pre-trained large model, and fuse the features using a multimodal attention fusion model to obtain fused HAP feature data;

[0143] A data generation module is used to generate enhanced HAP feature data or HAP feature data sequences that simulate physiological processes based on the fused HAP feature data using a controllable generative adversarial network;

[0144] The anomaly analysis module is used to use the fused HAP feature data and enhanced HAP feature data to detect early abnormal signals through an anomaly detection algorithm and perform multimodal attribution analysis to obtain attribution results;

[0145] The model optimization module is used to continuously optimize the nutritional demand prediction model and feeding strategy optimization model based on early abnormal signals and attribution results.

[0146] Here, the present invention provides an implementation example:

[0147] This application example simulates a modern piggery housing 100 fattening pigs during winter, focusing on risk monitoring for early respiratory diseases (such as influenza) and intelligent nutritional management interventions. The piggery is equipped with smart ear tags (collecting individual body temperature and activity levels), RFID-based feed troughs (recording feed intake and feeding duration), high-definition cameras (monitoring group behavior and individual status), and microphone arrays (collecting ambient sounds and pig coughing). Keepers record key observations (such as mental state and fecal status) daily using mobile devices.

[0148] Implementation process example:

[0149] The system collected the following representative HAP data:

[0150] Sensor data: Most pigs' body temperatures fluctuated between 38.5°C and 39.2°C, and their activity levels were normal. However, the body temperatures of three pigs (A, B, and C) showed a slow upward trend over the past 12 hours, reaching 39.5°C, 39.6°C, and 39.4°C, respectively. Their activity levels decreased by approximately 15% compared to the previous day's average. Feed intake data showed that these three pigs' daily feed intake was 20% to 25% lower than the herd average. Audio data: While the background noise level in the piggery was stable, the microphone array detected short, dry coughs around pigs B and C, which increased approximately 30% compared to the usual frequency. Text data: The breeder's electronic logbook for that day noted: "Pig A appears to be in a slightly poor state of mind and is not actively feeding; Pig B coughs occasionally; drinking behavior in the herd is generally normal." Image / video data: Camera footage shows Pig A spending extended periods of time crouching in a corner, interacting less with other pigs.

[0151] The system calls the corresponding pre-trained large models (such as BERT to process text, PANNs to process audio, Transformer encoder to process sensor timing, and ResNet to process images) to extract the feature vectors of the above-mentioned types of data. Subsequently, the multimodal attention fusion model fuses these feature vectors. In this particular case, since the text directly mentions "slightly poor mental state" and "lack of active feeding", a clear "coughing sound" is detected in the audio, and the sensor data also shows an increase in body temperature and a decrease in feed intake, the MAFM model will give these highly indicative modalities (text, audio, specific sensor indicators) relatively high attention weights during fusion, generating fused HAP feature data for pigs A, B, and C. 、 、 ,These feature vectors can comprehensively reflect its current potential abnormal state.

[0152] Scarce scenario data enhancement: Since real HAP data samples of early influenza symptoms in pigs may not be sufficient to fully train the anomaly detection model, the system calls the CGAN model with "early swine influenza" as the scenario condition , generate a batch of simulated enhanced HAP feature data The data simulated a comprehensive HAP pattern in terms of feature distribution, including a gradual increase in body temperature (e.g., from 39.0°C to 40.5°C), accompanied by coughing (specific audio feature patterns), decreased feed intake, and text descriptions containing keywords such as "lethargy" and "shortness of breath."

[0153] Physiological process simulation generation: In order to test the sensitivity and effective response of the nutrition management model to the gradual development of influenza symptoms, the system also needs to generate a dynamic process of simulating the disease from the initial stage to obvious symptoms. First, the system determines the health state potential vector (representing normal body temperature, activity level, and food intake) and disease state latent vectors (representing overt flu symptoms). Subsequently, a series of simulated HAP characteristic data sequences were generated through linear interpolation. This data sequence simulated the entire process of a pig's transition from a healthy state to overt flu symptoms, including the dynamic changes in key indicators such as the gradual increase in body temperature, the gradual decrease in activity level, and the gradual increase in coughing frequency.

[0154] The system integrates the HAP characteristic data of pigs A, B, and C Input to the trained autoencoder model for anomaly detection:

[0155] Abnormal signal detection: The autoencoder model attempts to reconstruct the input fused HAP feature data and generate reconstructed data , and calculate the reconstruction error 、 、 Compare these reconstruction errors with the preset threshold (This threshold is determined based on the 99th percentile of the reconstruction error of historical normal pig HAP data). The system found that the reconstruction errors of the three pigs exceeded the threshold, among which pig B had the highest reconstruction error ( ), so the system flagged early abnormal signals for these three pigs 、 、 .

[0156] By analyzing the attention weights of each modality in the MAFM model, the system found that for pig A, the body temperature sensor data and the description of "slightly poor mental state" in the text contributed the most; for pigs B and C, the coughing pattern in the audio data and the body temperature sensor data contributed significantly.

[0157] The above results were further verified by calculating the gradient of the anomaly detection model output (reconstruction error) with respect to each modal feature. It was found that the cough sound features of B and C made a particularly significant contribution to anomaly determination.

[0158] System-generated attribution reports , preliminary inference: Pig A may be in the early stage of the disease, mainly manifested by a decline in mental state and a slight increase in body temperature; Pigs B and C have more obvious respiratory symptoms, especially coughing, which are consistent with typical early influenza symptoms.

[0159] Model feedback and iterative optimization: The system integrates the HAP characteristic data of pigs A, B, and C , generated augmented data and data sequences that simulate disease progression , together with the detected early abnormal signals and attribution results , input into the nutritional demand prediction model In particular, the system uses the generated gradual process data sequence A stress test on the model revealed deviations in its prediction results when the body temperature was between 39.8°C and 40.2°C. Therefore, the part of the model structure responsible for handling high body temperature was adjusted specifically, and the loss function was optimized to improve the model's adaptability to pathological physiological changes.

[0160] Intelligent decision support: based on optimized nutritional demand prediction model and feeding strategy optimization models The system proposes personalized nutritional intervention plans for pigs A, B, and C:

[0161] For all three pigs, it was recommended to increase the electrolyte content in drinking water to ensure fluid balance;

[0162] For pigs B and C, since they have shown obvious respiratory symptoms, it is recommended to increase the content of vitamin C and E in their feed (increase to 150% of the standard content) to enhance immunity;

[0163] For pig A, it is recommended to increase the palatability of the feed and feed it in small amounts at multiple times to prevent its feed intake from decreasing further;

[0164] At the same time, the system also recommends that breeders closely monitor the development of symptoms in these three pigs and isolate and monitor pigs that have close contact with them to prevent the spread of the disease.

[0165] Follow-up results one week later showed that, after timely nutritional adjustments and health management interventions, Pig A had fully recovered. Pigs B and C also had their symptoms effectively controlled, preventing them from developing severe symptoms, and the disease had not spread further within the herd. This example demonstrates the effectiveness of intelligent nutrition model management based on HAP multimodal data in early disease warning and targeted nutritional intervention, particularly its ability to sensitively capture subtle abnormal signals and proactively intervene.

[0166] The above describes an embodiment of the present invention, but this embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Ordinary technicians in this field can also make more forms of equivalent embodiments based on the inspiration of this embodiment, all of which are protected by this embodiment.

Claims

1. An intelligent nutrition model management method based on HAP multimodal data, characterized by: include: Acquire multi-source heterogeneous HAP data of livestock and poultry, use a pre-trained large model to extract features from the multi-source heterogeneous HAP data, and fuse the features through a multimodal attention fusion model to obtain fused HAP feature data; Based on the fused HAP feature data, a controllable generative adversarial network is used to generate enhanced HAP feature data and a HAP feature data sequence that simulates the physiological process. The controllable generative adversarial network specifically includes: Using the generator model, based on the input random noise vector, preset scene conditions, and the potential state vector corresponding to the preset physiological indicator change path, enhanced HAP feature data and HAP feature data sequences that simulate physiological processes are generated; Using the discriminator model, we distinguish between real HAP feature data and HAP feature data generated by the generator model, and optimize the generator and discriminator models through an adversarial training process. By using fused and enhanced HAP feature data, anomaly detection algorithms are used to detect early abnormal signals and perform multimodal attribution analysis to obtain attribution results. Based on early abnormal signals and attribution results, the nutritional demand prediction model and feeding strategy optimization model are continuously optimized.

2. The intelligent nutrition pattern management method based on HAP multimodal data according to claim 1, characterized in that: The multi-source heterogeneous HAP data of livestock and poultry are obtained, including structured sensor data, structured text, audio data, image data and video data.

3. The intelligent nutrition pattern management method based on HAP multimodal data according to claim 1, characterized in that: The core component of the multimodal attention fusion model is the attention module, which dynamically allocates attention weights based on the importance of different modal features in representing the current individual state. The specific steps are as follows: First, the eigenvectors of each mode Projected to the same dimension through independent MLP layers ; Then all the projected modal features are concatenated or summed, and then input into another MLP to calculate the attention score of each modality, and the attention weight is obtained by normalization through the Softmax function. .

4. The intelligent nutrition pattern management method based on HAP multimodal data according to claim 1, characterized in that: The anomaly detection algorithm is specifically an autoencoder model based on reconstruction error, and the detection steps include: The input HAP feature data is reconstructed using the autoencoder model to obtain reconstructed HAP feature data; Calculating a reconstruction error between the input HAP feature data and the reconstructed HAP feature data; When the reconstruction error is greater than a preset error threshold, it is determined that an early abnormal signal is detected.

5. The intelligent nutrition pattern management method based on HAP multimodal data according to claim 4, characterized in that: The preset error threshold is determined by a dynamic threshold adjustment strategy based on sliding window statistics.

6. The intelligent nutrition pattern management method based on HAP multimodal data according to claim 1, characterized in that: The anomaly detection algorithm detects early abnormal signals and performs multimodal attribution analysis, including: When the multimodal attention fusion model adopts an explicit attention mechanism, the attention weights corresponding to the feature vectors of each original modality in the fused HAP feature data that causes the early abnormal signal are analyzed to determine the modality that contributes most to the early abnormal signal; The gradient value of the output result of the anomaly detection algorithm with respect to each original modal feature vector in the fused HAP feature data is calculated, and the mode with a larger gradient value is determined as the mode that contributes more to the early abnormal signal.

7. The intelligent nutrition pattern management method based on HAP multimodal data according to claim 1, characterized in that: The continuous optimization of the nutrient requirement prediction model and feeding strategy optimization model includes: using a comprehensive data set containing real HAP data, enhanced HAP data and simulated process HAP data to optimize the nutrient requirement prediction model. and feeding strategy optimization models Perform periodic retraining and parameter fine-tuning.

8. The intelligent nutrition pattern management method based on HAP multimodal data according to claim 1, characterized in that: The nutritional demand prediction model is used to predict the dynamic demand for multiple nutrients of the current individual based on the fused HAP feature data and other relevant individual information; The feeding strategy optimization model is used to generate specific feed formula and feeding amount recommendations through an optimization algorithm based on the dynamic demand predicted by the nutritional demand prediction model, combined with feed raw material information and preset breeding goals.

9. Intelligent nutrition pattern management system based on HAP multimodal data, characterized by: The method for intelligent nutrition pattern management based on HAP multimodal data according to any one of claims 1 to 8 comprises: The data processing module is used to obtain multi-source heterogeneous HAP data of livestock and poultry, extract features from the multi-source heterogeneous HAP data using a pre-trained large model, and fuse the features using a multimodal attention fusion model to obtain fused HAP feature data; A data generation module is used to generate enhanced HAP feature data or HAP feature data sequences that simulate physiological processes based on the fused HAP feature data using a controllable generative adversarial network; The anomaly analysis module is used to use the fused HAP feature data and enhanced HAP feature data to detect early abnormal signals through anomaly detection algorithms and perform multimodal attribution analysis to obtain attribution results; The model optimization module is used to continuously optimize the nutritional demand prediction model and feeding strategy optimization model based on early abnormal signals and attribution results.

Images