Intelligent feed recommendation and precise feeding system based on multi-mode intelligent perception of live pigs

Abstract

The invention provides an intelligent feed recommendation and precise feeding system based on multi-mode intelligent perception of live pigs. The system comprises a sensing module, a data processing module, a decision module and an execution module. The sensing module collects visual, sound, behavior and physiological data of live pigs, the data processing module cleans, standardizes and fuses the data, the decision module generates a feed feeding decision through a reinforcement learning algorithm, and the execution module automatically controls feeding equipment according to the decision. Data cleaning adopts different strategies to remove noise, standardization ensures compatibility of different modal data, and data fusion extracts features through a convolutional neural network and LSTM and uses a self-attention mechanism to perform weighted fusion. A reinforcement learning algorithm optimizes decisions, balances the health state, the feed utilization rate and the cost, and dynamically adjusts the feeding strategy. Through automatic whole-process management, the system can significantly improve the feed utilization rate, reduce the feed cost, improve the health condition of live pigs, and maximize the economic benefits of a farm.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and breeding, and specifically to an intelligent feed recommendation and precise feeding system based on multi-modal intelligent perception of live pigs. Background Art

[0002] With the growth of the global population and the increasing demand for meat, traditional breeding models face problems such as low efficiency, resource waste, and difficult health management. Modern breeding industries have started to rely on the Internet of Things (IoT), big data, artificial intelligence (AI), and sensor technologies to promote intelligent development in order to achieve automated and precise management. However, despite the great potential of intelligent technologies in the breeding industry, there are still some technical problems in practical applications.

[0003] First, the problem of multi-modal data fusion. The data collected in live pig breeding usually comes from multiple different types of sensors, including visual images, sounds, behaviors, and physiological data, etc. These data have different characteristics and structures. How to effectively perform data cleaning, standardization, and fusion to achieve data interoperability and accurate analysis is a major challenge in current technologies.

[0004] Second, the training difficulty of reinforcement learning algorithms. Reinforcement learning requires a large amount of data and computing resources to be effectively trained in complex environments. Due to the dynamic changes in the breeding environment, there are uncertainties in the health status, feed requirements, and environmental conditions of live pigs, resulting in insufficient adaptability and convergence speed of traditional reinforcement learning algorithms in the actual environment. In addition, the design of the reward function also needs to take into account multiple objectives, such as pig health, feed utilization rate, and cost, etc., making the optimization of the reinforcement learning model more complex. Summary of the Invention

[0005] The purpose of the present invention is to solve the above problems. For this reason, the technical solution adopted by the present invention is as follows: An intelligent feed recommendation and precise feeding system based on multi-modal intelligent perception of live pigs, including a perception module, a data processing module, a decision-making module, and an execution module.

[0006] The perception module is responsible for collecting live pig visual image data, sound data, physiological data, and behavior data.

[0007] The data processing module is responsible for sequentially performing data cleaning, data standardization, and data fusion on the data collected by the perception module; different cleaning strategies are adopted for different modal data; different standardization strategies are adopted for different modal data; data fusion operations generate fusion features; the fusion features are used for live pig health analysis and the health status code is output.

[0008] The decision-making module utilizes the fused features and the health status encoding, and combines with the policy network in the reinforcement learning algorithm to generate the feed feeding decision; the parameters of the policy network are optimized and updated depending on the reward function; the influencing factors of the reward function include the health status encoding, the feed utilization rate, and the feed cost.

[0009] The execution module automatically controls the precise feeding device according to the feed feeding decision to perform feed distribution and feedback monitoring.

[0010] The acquisition devices of the perception module include a visual sensor, a sound sensor, a behavior detection device, and a biosensor; the visual sensor is used to collect visual image data of the external appearance and body shape of live pigs in real time; the sound sensor is used to collect sound data of live pigs including the call frequency and pitch in real time; the behavior detection device is used to collect behavior data of live pigs including the movement speed and acceleration in real time; the biosensor is used to collect physiological data including the body temperature, blood glucose level, and changes in internal hormones of live pigs in real time.

[0011] Furthermore, different cleaning strategies need to be adopted for different modality data in the data cleaning, specifically as follows:

[0012] For the visual image data, the Gaussian filtering algorithm is used to remove the noise in the visual image, and the Sobel operator is used to detect whether the visual image data is blurred. If the detection result is lower than the set first threshold, the blind deblurring algorithm is used to repair the visual image data; for the sound data, a band-pass filter is designed to remove the low-frequency and high-frequency noises and retain the frequency band of pig calls; for the behavior data, a low-pass filter is used to remove the noise data caused by device vibration and signal interference; for the physiological data, threshold detection based on physiological common sense is used, that is, the physiological data values outside the set physiological data range are removed.

[0013] Furthermore, different standardization strategies are adopted for different modality data in the data standardization, specifically as follows:

[0014] For the visual image data X1, all pixel values are made to fall within the range [0, 1]; the form is expressed as where X g represents the standardized visual image data;

[0015] For the sound data X2, the behavior data X3, and the physiological data X4, Z-score standardization is used, and the forms are respectively and where μ2, μ3, and μ4 respectively represent the means of the sound data X2, the behavior data X3, and the physiological data X4, and σ2, σ3, and σ4 respectively represent the variances of the sound data X2, the behavior data X3, and the physiological data X4, X v 、X sand X p respectively represent the standardized voice data, behavior data, and physiological data.

[0016] Furthermore, the data fusion is to integrate the visual image data X g , voice data X v , behavior data X s , and physiological data X p The specific steps are as follows:

[0017] For the visual image data, a convolutional neural network is used to extract visual features, and the visual features include body size and the form of live pigs; the visual features are represented as F g ;

[0018] For the voice data, spectral analysis is used to extract voice features, and from the voice features, it can be analyzed whether the live pigs are in a state of hunger or anxiety. The voice features are represented as a feature vector F v ;

[0019] For the behavior data, a first LSTM neural network is used to extract behavior features, and the behavior features are represented as a feature vector F s ;

[0020] For the physiological data, a second LSTM neural network is used to extract physiological features X p , and the structure of the second LSTM is different from that of the first LSTM. The physiological features are represented as a feature vector F p ;

[0021] After obtaining the feature vectors F g , F v , F s and F p , feature fusion is performed on all the feature vectors. Before feature fusion, dimension alignment operations are performed on F g , F v , F s and F p .

[0022] Furthermore, the dimension alignment operation linearly transforms all the feature vectors and maps them to the same dimension d. The linear transformation formula for each modal feature vector is as follows:

[0023] F‘ g = W g F g + b g

[0024] F‘ v = W v F v + b v

[0025] F‘ s = W s F s + b s

[0026] F‘ p = W p F p + b p

[0027] Wherein, W g 、W v 、W s and W p are weight matrices, b g 、b v 、b s and b p are bias term parameters, F‘ g 、F‘ v 、F‘ s and F‘ p are the eigenvectors after linear transformation of F g 、F v 、F s and F p respectively.

[0028] Furthermore, the feature fusion uses a self-attention mechanism, and a dynamic modality weighting mechanism is added to the self-attention mechanism; the dynamic modality weighting mechanism is expressed as follows:

[0029] ω i = σ(W w F‘ i + b w ), i ∈ {g, v, s, p}

[0030] Wherein, ω i represents the dynamic weight of the corresponding modality of F′ i ,W w is the weight matrix, b w is the bias term parameter;

[0031] After generating the dynamic weights, the eigenvectors F‘ i after linear transformation of all modalities are weighted to generate and concatenated into an overall eigenvector F all , and the formula is as follows:

[0032]

[0033] is the eigenvector weighted by F‘ i respectively.

[0034] Further, the self-attention mechanism is implemented as follows:

[0035] First, calculate the query Q, key K, and value vector V. The formulas are as follows:

[0036] Q = F all W Q , K = F all W K , V = F all W V

[0037] where W Q , W K and W V are weight matrices;

[0038] Next, calculate the attention score A:

[0039]

[0040] Finally, calculate the fused feature F:

[0041] F = AV

[0042] The fused feature F contains all visual image features, behavioral features, sound features, and physiological features, and is used to analyze the health of live pigs and input to the decision-making module;

[0043] The health analysis of live pigs is to input the fused feature F into a Softmax classification network to predict the health status of live pigs. The health status includes normal and specific disease types;

[0044] Encode the health status. The rule is: Encode the disease types according to the preset severity level. The more severe the disease type, the smaller the corresponding encoded value. The lowest encoded value is 0, and the encoded value for normal is the largest.

[0045] Further, the decision-making module is constructed based on the reinforcement learning algorithm. The construction steps are as follows:

[0046] Define the state space: The state space is denoted as S, and s t is the fused feature at the t-th moment in the state space S. The s t is the fused feature F output by the data processing module;

[0047] Define the state space: The state space is denoted as S, and s t is the fused feature at the t-th moment in the state space S. The a t is the fused feature F output by the data processing module;

[0048] Define the action space: The action space is denoted as A, and a tis the decision-making action at the t-th moment in the action space A, and the decision-making action includes feed type selection, feeding amount, and feeding timing;

[0049] Reward function design: The reward function needs to balance the health status encoding R h , feed utilization rate R e and feed cost R c for three objectives,

[0050] The feed cost is the sum of the products of all feed types and their corresponding unit prices,

[0051] The feed utilization rate is the ratio of the weight gain of live pigs to the feed consumption, and is defined as follows:

[0052]

[0053] The comprehensive representation of the reward function is as follows, r t represents the reward at the t-th moment in the reward function R:

[0054]

[0055] where α is a fixed weight; β t and γ t are dynamic weights, C max is the maximum feed budget cost;

[0056] The dynamic update formula for β t is: β t = 1 - e -λt , where λ is the exponential weight;

[0057] The dynamic update formula for γ t is

[0058] The reinforcement learning algorithm also includes a policy network; the input of the policy network is the fused feature F, and the output is the corresponding decision-making action;

[0059] The decision-making process of the reinforcement learning algorithm is: input the current state s t into the policy network to select a decision-making action a t , execute the action a t and interact with the environment through the output to the execution module to obtain the next state s t+1 , and calculate the reward r t+1 , and use r t+1 to update the parameters of the policy network.

[0060] Further, the execution module automatically performs precise feeding according to the decision actions generated by the decision module, and interacts with the environment to feedback the execution effect of the decision actions. The environmental interaction is to control the perception module and the data processing module to generate new fusion features, and the new fusion features are the next state a in the decision module. t+1 。

[0061] Compared with the prior art, the advantages of the present invention are as follows:

[0062] (1) Through the multi-modal intelligent perception technology and in combination with the reinforcement learning algorithm to optimize the feed feeding decision in real time, the present invention can dynamically adjust the feed type, feeding amount and feeding time, so as to precisely meet the physiological needs of live pigs and avoid feed waste. At the same time, by balancing the feed cost and the health status through the reward function, the overall feed cost input is reduced, and the economic benefit is maximized.

[0063] (2) Through the multi-modal fusion of visual, sound, behavior and physiological data, the present invention comprehensively analyzes the health status of live pigs and predicts possible health problems in advance, and precisely recommends functional feeds, which can effectively reduce the disease incidence rate and improve the overall health level of live pigs in the farm.

[0064] (3) The present invention realizes the full-automatic operation from perception, live pig health analysis, decision-making to feeding, significantly reducing the need for manual intervention and lowering the labor cost. At the same time, the precise control and dynamic optimization of the feeding equipment are realized by using the reinforcement learning algorithm, which can perform feeding management more efficiently and greatly improve the operation efficiency of the farm. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0066] Figure 1 is the system framework diagram of the present invention;

[0067] Figure 2 is the schematic flow diagram of the data processing module of the present invention;

[0068] Figure 3 is the schematic flow diagram of the decision module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0069] The following further illustrates the present invention in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it, but the embodiments given are not intended to limit the present invention.

[0070] This embodiment provides a feed intelligent recommendation and precise feeding system based on multi-modal intelligent perception of live pigs. This embodiment is used in a large-scale live pig farm, which adopts the "one household, one meter" system, and equips each live pig with an independent perception module to collect live pig data. The goal of this embodiment is to improve feed utilization rate, optimize feeding management, enhance the health level of live pigs, and reduce labor costs through intelligent means.

[0071] This embodiment includes a perception module, a data processing module, a decision-making module, and an execution module. Their main functions are as follows: The perception module is responsible for collecting physiological data, behavioral data, and environmental data of live pigs; the data processing module is responsible for data cleaning, data standardization, and data fusion of the data collected by the perception module; the decision-making module generates feed feeding decisions using reinforcement learning algorithms based on the results of the data processing module; the execution module automatically controls the precise feeding equipment according to the feed feeding decisions to perform feed distribution and feedback monitoring.

[0072] The acquisition devices of the perception module at least include visual sensors, sound sensors, behavior detection devices, and biosensors; the visual sensors are used to collect visual image data of the appearance and body shape of live pigs in real time; the sound sensors are used to collect sound data of live pigs including call frequency and pitch in real time; the behavior detection devices are used to collect behavioral data including the movement speed and acceleration of live pigs in real time; the biosensors are used to collect physiological data including the body temperature, blood sugar level, and changes in internal hormones of live pigs.

[0073] The goal of data cleaning is to remove noise, outliers, and duplicate data in the data, ensure data quality, and avoid affecting the analysis and decision-making of the system. For example, during data transmission, problems such as duplicate sampling or multiple uploads of the same data may occur; another example is that the acquisition device fails, resulting in missing or exceeding normal thresholds in the collected data; yet another example is that the visual image data collected by the visual sensor is unclear.

[0074] Since there are differences in the types of multi-modal data collected by the perception module, different cleaning strategies need to be adopted for different modal data. Specifically as follows:

[0075] For visual image data, use the Gaussian filtering algorithm to remove noise in the visual image, and use the Sobel operator to detect whether the visual image data is blurred. If the detection result is lower than the set first threshold, use the blind deblurring algorithm to repair the visual image data;

[0076] For sound data, design a band-pass filter to remove low-frequency and high-frequency noise and retain the frequency band of pig calls; for example, the pig call frequency is from 100 Hz to 1 kHz, and noise below 100 Hz and above 1 kHz can be removed;

[0077] For behavioral data, a low-pass filter is used to remove the noise data caused by device vibration and signal interference;

[0078] For physiological data, threshold detection based on physiological common sense is used, that is, the physiological data values outside the set physiological data range are excluded; for example, the normal body temperature of live pigs should be between 38°C and 40°C. When the body temperature data exceeds or is lower than this range, it is considered abnormal data and excluded.

[0079] In this embodiment, the goal of data standardization is to convert multi-modal data from different sources into the same scale, eliminate data biases caused by differences in sensor ranges, data units, etc., so as to ensure the compatibility of different types of data and facilitate subsequent fusion, analysis, and modeling.

[0080] Since multi-modal data has its own characteristics, different standardization strategies are adopted for different modal data. The specific data standardization is as follows:

[0081] For visual image data X1, since image data usually contains a large number of pixel values, and the range of each pixel value is from 0 to 255, the pixel values are divided by 255 so that all pixel values fall within the range of [0, 1]; it is expressed in the form of where X g represents the standardized visual image data;

[0082] For sound data X2, behavioral data X3, and physiological data X4, Z-score standardization is used, and the forms are respectively and where μ2, μ3, and μ4 represent the means of sound data X2, behavioral data X3, and physiological data X4 respectively, and σ2, σ3, and σ4 represent the variances of sound data X2, behavioral data X3, and physiological data X4 respectively. X v 、X s and X p represent the standardized sound data, behavioral data, and physiological data respectively. Z-score standardization is adopted because sound data, behavioral data, and physiological data usually fluctuate within a certain range.

[0083] The data fusion is to integrate the data features included in visual image data X g 、sound data X v 、behavioral data X s 、physiological data X p to analyze the health status and feed requirements of live pigs. The specific steps are as follows:

[0084] For visual image data, a convolutional neural network is used to extract visual features, which include body size and the morphology of live pigs; if the body is too thin or the morphology is abnormal, there may be health problems; the visual features are represented as F g ;

[0085] For sound data, spectral analysis is used to extract sound features, which include frequency distribution and pitch fluctuation; from the sound features, it can be analyzed whether the live pigs are in a state of hunger or anxiety; the sound features are represented as a feature vector F v ;

[0086] For behavior data, a first LSTM neural network is used to extract behavior features, from which the movement patterns and activity levels of live pigs can be analyzed; the reason for using LSTM to extract behavior features is that behavior data is a time series, and the LSTM neural network has been proven to be particularly effective in extracting time series features. The behavior features are represented as a feature vector F s ;

[0087] For physiological data, a second LSTM neural network is used to extract physiological features X p , from which it can be analyzed whether the physiological indicators of live pigs are normal; physiological data is also a time series, but the dimension is different from that of behavior data, so the structure of the second LSTM is different from that of the first LSTM; the physiological features are represented as a feature vector F p .

[0088] After obtaining the feature vectors F g , F v , F s and F p , feature fusion is performed on all the feature vectors. Before feature fusion, dimension alignment operations are performed on F g , F v , F s and F p ;

[0089] The dimension alignment operation is to linearly transform and map all the feature vectors to the same target dimension d. The linear transformation formula for each modal feature vector is as follows:

[0090] F‘ g =W g F g +b g

[0091] F‘ v =W v F v +b v

[0092] F‘s = W s F s + b s

[0093] F' p = W p F p + b p

[0094] Wherein, W g , W v , W s and W p are weight matrices, b g , b v , b s and b p are bias term parameters, F' g , F' v , F' s and F' p are the eigenvectors after linear transformation of F g , F v , F s and F p respectively.

[0095] The feature fusion uses the self-attention mechanism and adds a dynamic modality weighting mechanism to the self-attention mechanism; the dynamic modality weighting mechanism is expressed as follows:

[0096] ω i = σ(W w F' i + b w ), i ∈ {g, v, s, p}

[0097] Wherein, ω i represents the dynamic weight of the corresponding modality of F', W i is the weight matrix, b w is the bias term parameter. w

[0098] After generating the dynamic weights, the eigenvectors F' i after linear transformation of all modalities are weighted to generate and concatenated into an overall eigenvector F all , and the formula is as follows:

[0099]

[0100] is the eigenvector weighted by F' i .

[0101] The implementation of the self-attention mechanism is as follows,

[0102] First, calculate the query Q, key K, and value vector V using the following formulas:

[0103] Q = F all W Q , K = F all W K , V = F all W V

[0104] where W Q , W K and W V are weight matrices;

[0105] Next, calculate the attention score A:

[0106]

[0107] Finally, calculate the fused feature F:

[0108] F = AV

[0109] The fused feature F includes all visual image features, behavioral features, sound features, and physiological features, and is used to analyze the health of live pigs and generate feed feeding decisions for the decision-making module.

[0110] The analysis of the health of live pigs is to input the fused feature F into a Softmax classification network to predict the health status of live pigs, and the health status includes normal and specific disease types;

[0111] Encode the health status with the following rule: Encode the disease types according to a preset severity level, where the more severe the disease type, the smaller the corresponding encoded value, and the lowest encoded value is 0, while the encoded value for normal is the largest. This encoding method makes the rewards for feed feeding decisions that improve the health status of live pigs greater, while the rewards for feed feeding decisions that do not improve the health status of live pigs are smaller.

[0112] In this embodiment, the decision-making module generates feed feeding decisions using the results of the analysis of the health of live pigs and the fused feature. The decision-making module is constructed based on the reinforcement learning algorithm, and comprehensively considers information on the health status of live pigs, feed cost, and feed utilization rate. The construction steps are as follows:

[0113] Define the state space: The state space is denoted as S, and s t is the fused feature at the t-th moment in the state space S. The s t is the fused feature F output by the data processing module.

[0114] Define the action space: The action space is denoted as A, and a tis the decision-making action at the t-th moment in the action space A. The decision-making action includes feed type selection, feeding amount, and feeding timing.

[0115] The feed types include: protein feed, green feed, silage feed, energy feed, roughage, mineral feed, and health feed, etc. The health feed is used to treat corresponding diseases.

[0116] Reward function design: The reward function needs to balance the health state R h , feed utilization rate R e and feed cost R c for three objectives.

[0117] The feed cost is the sum of the products of all feed types and their corresponding unit prices.

[0118] The feed utilization rate is defined as follows:

[0119]

[0120] The comprehensive representation of the reward function R is as follows, r t represents the reward at the t-th moment in the reward function R:

[0121]

[0122] where α is a fixed weight; β t and γ t are dynamic weights, C max is the maximum budget cost;

[0123] β t The dynamic update formula for β is: β t = 1 - e -λt ,

[0124] γ t The dynamic update formula for γ is

[0125] The reinforcement learning algorithm outputs feed feeding decisions through a policy network. In this embodiment, the type of the policy network is not limited and can be the policy network in Deep Deterministic Policy Gradient (DDPG) or the policy network in Actor-Critic;

[0126] The decision-making process of the reinforcement learning algorithm is: Input the current state s t into the policy network to select a decision-making action a t , execute the action a t and interact with the execution module environment to obtain the next state s t+1 , and calculate the reward r t+1 , and use r t+1 to update the parameters of the policy network.

[0127] When the reward function value r t+1 is larger, the policy network will consider that the current action a t is beneficial to the health of the live pigs, and the live pigs gain more weight, the feed cost and feed utilization rate are reasonable, and the live pigs are relatively healthy. Therefore, the policy network will gradually consider that the current action a t for the current state s t of the live pigs is reasonable. When the reward function value r t+1 is smaller, the opposite is true.

[0128] By guiding the optimization of the policy network parameters through the reward function, the policy network can make more reasonable decisions.

[0129] In this embodiment, the execution module automatically performs precise feeding according to the decision-making action generated by the decision-making module, and interacts with the environment to feedback the execution effect of the decision-making action. The environmental interaction is to control the perception module and the data processing module to generate new fusion features, and the new fusion features are the next state s t+1 in the decision-making module.

[0130] Through the application of multi-modal intelligent perception technology and the real-time optimization of feed feeding decisions by reinforcement learning algorithms, the system can significantly improve the feed utilization rate in the farm and effectively reduce the feeding cost. The multi-modal perception module collects the physiological, behavioral and environmental data of the live pigs, provides comprehensive information support for the system, and combines with the reinforcement learning algorithm to dynamically adjust the feed type, feeding amount and feeding time, so that the feed feeding can accurately meet the physiological needs of the live pigs and avoid feed waste caused by overfeeding or unreasonable feeding. In addition, through the scientific design of the reward function, the system can balance the feed cost and the health status of the live pigs, dynamically optimize the feed type and dosage on the premise of ensuring the health of the live pigs, so as to significantly reduce the overall feed cost input and finally maximize the economic benefits of the farm.

[0131] The multi-modal data fusion function of the system significantly improves the health level of the live pigs. By using visual sensors to collect body shape and appearance data, sound sensors to collect call frequency and pitch data, behavior detection devices to collect motion behavior data, and biosensors to collect physiological data such as body temperature and blood sugar, the system can comprehensively analyze the health status of the live pigs and identify potential health problems. The fused multi-modal features are input into the health analysis module, which can accurately judge whether the live pigs are in a healthy state. Based on the health analysis results, the system can intelligently recommend supplementary feed or medicated feed, accurately meet the nutritional needs of the live pigs, timely intervene in health problems, and reduce the incidence of diseases. This intelligent health management method can not only improve the growth status of individual live pigs, but also improve the health level of the entire farm, thus achieving higher breeding efficiency.

[0132] In addition, the system realizes the full-process automation from data perception to intelligent decision-making and then to precise feeding, greatly reducing the need for manual intervention and significantly reducing labor costs. The tasks of feed feeding that used to rely on manual labor, such as monitoring the health status of live pigs and adjusting feed feeding strategies, are all automatically completed by the system. At the same time, the application of reinforcement learning algorithms in the feeding equipment can dynamically optimize the feeding strategy to ensure the accuracy and flexibility of feed distribution.

[0133] The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A feed intelligent recommendation and precision feeding system based on multimodal intelligent perception of pigs, comprising a perception module, a data processing module, a decision module and an execution module, characterized in that: The perception module is responsible for collecting visual image data, sound data, physiological data, and behavioral data of pigs; The data processing module is responsible for data cleaning, data standardization and data fusion of the data collected by the perception module in sequence; different cleaning strategies are used for different modal data; different standardization strategies are used for different modal data; data fusion operation generates fusion features; fusion features are used to analyze the health of pigs and output health status codes; The decision module uses fusion features and health status coding, combined with the policy network in the reinforcement learning algorithm to generate feed feeding decisions; the policy network parameters are optimized and updated based on the reward function; the influencing factors of the reward function include health status coding, feed utilization rate and feed cost; The execution module automatically controls the precision feeding equipment according to the feed feeding decision to perform feed distribution and feedback monitoring.

2. According to claim 1, a feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs is characterized in that: The acquisition equipment of the perception module includes visual sensors, sound sensors, behavior detection equipment and biological sensors; the visual sensors are used to collect visual image data of the appearance and body shape of live pigs in real time; the sound sensors are used to collect sound data of live pigs including call frequency and pitch in real time; the behavior detection equipment is used to collect behavior data including movement speed and acceleration of live pigs in real time; the biological sensors are used to collect physiological data including body temperature, blood sugar level and hormone changes of live pigs in real time.

3. According to the feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs according to claim 1, the cleaning strategy is as follows: For visual image data, a Gaussian filtering algorithm is used to remove noise in the visual image, and a Sober operator is used to detect whether the visual image data is blurred. If the detection result is lower than a set first threshold, a blind deblurring algorithm is used to repair the visual image data; For the sound data, a bandpass filter is designed to remove low-frequency and high-frequency noise and retain the frequency of the pig's call. part; For behavioral data, a low-pass filter is used to remove noise data caused by device vibration and signal interference; For physiological data, a threshold detection based on common physiological knowledge is used, that is, physiological data values ​​outside the set physiological data range are eliminated.

4. The feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs according to claim 1 is characterized in that: The data standardization adopts different standardization strategies for different modal data, as follows: For the visual image data X1, all pixel values ​​fall within the range of [0,1]; the form is expressed as Where X g Represents standardized visual image data; The sound data X2, behavior data X3, and physiological data X4 are standardized using Z-score in the following forms: and Where μ2, μ3 and μ4 represent the means of sound data X2, behavior data X3 and physiological data X4, respectively; σ2, σ3 and σ4 represent the variances of sound data X2, behavior data X3 and physiological data X4, respectively; v , X s and X p Represent the standardized sound data, behavioral data, and physiological data respectively.

5. According to the feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs in claim 1, the data fusion is to integrate visual image data X g , sound dataX v , behavioral dataX s , Physiological dataX p The data features included are as follows: For visual image data, a convolutional neural network is used to extract visual features, including body size and pig morphology; the visual features are represented by F g ; For the sound data, spectrum analysis is used to extract sound features, from which it is possible to analyze whether the piglets are hungry or anxious. The sound features are represented by feature vectors F v ; For the behavior data, the first LSTM neural network is used to extract the behavior features, which are represented as feature vectors F s ; For physiological data, the second LSTM neural network is used to extract physiological features X p , the structure of the second LSTM is different from that of the first LSTM, and the physiological feature is represented by a feature vector F p ; After getting the eigenvector F g 、F v 、F s and F p After that, all the feature vectors are subjected to feature fusion. Before feature fusion, F g 、F v 、F s and F p Perform dimension alignment operations.

6. According to the feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs according to claim 5, the dimension alignment operation linearly transforms all the feature vectors and maps them to the same dimension d. The linear transformation formula for each modal feature vector is as follows: F‘ g =W g F g +b g F‘ v =W v F v +b v F‘ s =W s F s +b s F‘ p =W p F p +b p in, W g , W v , W s and W p is the weight matrix, b g 、b v 、b s and b p is the bias parameter, F' g 、F' v 、F' s and F' p F g 、F v 、F s and F p The eigenvector after linear transformation.

7. According to the feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs according to claim 6, the feature fusion uses a self-attention mechanism, and a dynamic modal weighting mechanism is added to the self-attention mechanism; the dynamic modal weighting mechanism is expressed as follows: oh i =σ(W w F' i +b w ),i∈{g,v,s,p} in, ω i Indicates F' i The dynamic weight of the corresponding mode, W w is the weight matrix, b w is the bias parameter; After generating dynamic weights, the eigenvector F' after linear transformation of all modes i Weighted to generate And concatenate into an overall feature vector F all , the formula is as follows: It's F' i The weighted feature vector.

8. According to the feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs according to claim 7, the self-attention mechanism is implemented as follows: First, we calculate the query Q, key K, and value vector V. The formula is as follows: Q=F all W Q ,K=F all W K ,V=F all W V in, W Q , W K and W V is the weight matrix; Next, calculate the attention score A: Finally, calculate the fusion feature F: F =AV The fusion feature F includes all visual image features, behavioral features, sound features and physiological features, which are used to analyze pig health and input into the decision-making module; The pig health analysis is to input the fusion feature F into the Softmax classification network to predict the health status of the pig, and the health status includes normal and specific disease types; The health status is encoded according to the following rule: the disease type is encoded according to the preset severity level, the more serious the disease type, the smaller the corresponding coding value, the lowest coding value is 0, and the normal coding value is the largest.

9. According to the feed intelligent recommendation and precise feeding system based on multimodal intelligent perception of pigs according to claim 1, the decision module is constructed according to the reinforcement learning algorithm, and the construction steps are as follows: Define the state space: The state space is denoted as S, s t is the fusion feature at the tth moment in the state space S, where s t That is, the fusion feature F output by the data processing module; Define action space: The action space is represented by A, a t is the decision action at the tth moment in the action space A, and the decision action includes the selection of feed type, feeding amount, and feeding time; Reward function design: The reward function needs to balance the health state encoding R h , feed utilization rate R e and feed cost R c Three goals, The feed cost is the sum of the products of all feed types and the corresponding unit prices. The feed utilization rate is the ratio of pig weight gain to feed consumption, and is defined as follows: The comprehensive representation of the reward function is as follows, r t Expressed as the reward at the tth moment in the reward function R: in, α is a fixed weight; β t and γ t is the dynamic weight, C max is the maximum feed budget cost; β t The dynamic update formula is: t =1-e -λt , λ is the exponential weight; γ t The dynamic update formula is The input of the strategy network is the fusion feature F, and the output is the corresponding decision action; The decision-making process of the reinforcement learning algorithm is: the current state s t Input into the policy network to select a decision action a t , execute decision action a t And output to the execution module for environmental interaction to obtain the next state s t+1 , and calculate the reward r t+1 , using r t+1 The policy network parameters are updated.

10. According to claim 1 or 9, a feed intelligent recommendation and precision feeding system based on multimodal intelligent perception of pigs, the execution module automatically performs precision feeding according to the decision action generated by the decision module, and interacts with the environment to feedback the execution effect of the decision action, the environmental interaction is to control the perception module and the data processing module to generate a new fusion feature, and the new fusion feature is the next state a in the decision module t+1 .

Images