Method and system for predicting feed intake of periparturient dairy cows in pasture breeding scenarios

Abstract

The present application discloses a method and system for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario, and relates to the field of dairy cow breeding technology. The method includes: obtaining first monitoring data collected by a smart collar, second monitoring data collected by a visible light camera, and third monitoring data collected by an infrared camera, respectively extracting features from the first monitoring data, the second monitoring data, and the third monitoring data to obtain first features, second features, and third features, performing feature fusion on the first features, the second features, and the third features to obtain fused features, using the fused features as input, and using a trained feed intake prediction model to determine the feed intake of periparturient dairy cows. The present application can be combined with smart collars, visible light cameras, and infrared cameras to complete feeding behavior monitoring and feed intake prediction of periparturient dairy cows, and achieve real-time monitoring and precise analysis.

Description

Technical Field

[0001] The present application relates to the field of dairy cattle breeding technology, and in particular to a method and system for predicting feed intake of periparturient dairy cows in a pasture breeding scenario. Background Art

[0002] The peripartum period of dairy cows usually includes the pre-peripartum period from 21 days before delivery to calving and the post-peripartum period from calving to 21 days after delivery. The peripartum period is a special physiological period for dairy cows. After pregnancy, calving, and lactation, the cows' feed intake and feeding environment change dramatically, which can easily cause negative energy balance in dairy cows. It is also a high-incidence period for various metabolic diseases such as ketosis, displaced abomasum, and retained placenta, which will have a great impact on subsequent production and reproductive performance. During the peripartum period, the feeding behavior of dairy cows plays a vital role in their own production and health. Accurate monitoring and analysis of these feeding behaviors are of great significance for optimizing feeding management, improving production efficiency, and ensuring the health of dairy cows.

[0003] However, during the critical stage of periparturient dairy cow farming, traditional feeding behavior monitoring and analysis methods are difficult to achieve real-time monitoring and accurate analysis because they are highly dependent on manual observation to obtain monitoring data and highly dependent on manual experience to analyze monitoring data. Summary of the invention

[0004] The purpose of this application is to provide a method and system for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario, which can be combined with smart collars, visible light cameras and infrared cameras to complete the feeding behavior monitoring and feed intake prediction of periparturient dairy cows, and realize real-time monitoring and accurate analysis.

[0005] To achieve the above objectives, this application provides the following solutions.

[0006] In a first aspect, the present application provides a method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario, and the method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario comprises:

[0007] Obtaining first monitoring data collected by a smart collar, second monitoring data collected by a visible light camera, and third monitoring data collected by an infrared camera; the smart collar is worn on the neck of a periparturient dairy cow, and the first monitoring data includes the feeding time, dry matter intake, rumination time, rest time, activity time, and activity amount of the periparturient dairy cow; the visible light camera and the infrared camera are both installed in the dairy cow feeding area of ​​the pasture, the second monitoring data is a visible light video obtained by shooting the feeding process of the periparturient dairy cow, and the third monitoring data is an infrared video obtained by shooting the feeding process of the periparturient dairy cow;

[0008] Feature extraction is performed on the first monitoring data, the second monitoring data and the third monitoring data respectively to obtain a first feature, a second feature and a third feature; the first feature includes feeding time, feeding frequency, rumination time and activity change value obtained by feature extraction of the first monitoring data, and the activity change value includes the mean and standard deviation of the activity; the second feature includes the head posture and feeding rate obtained by feature extraction of the second monitoring data; the third feature includes the feed pile shape parameters obtained by feature extraction of the third monitoring data;

[0009] Performing feature fusion on the first feature, the second feature, and the third feature to obtain a fused feature;

[0010] The fused features are used as input and the trained feed intake prediction model is used to determine the feed intake of periparturient dairy cows.

[0011] In a second aspect, the present application provides a system for predicting feed intake of periparturient dairy cows in a pasture breeding scenario, and the system for predicting feed intake of periparturient dairy cows in a pasture breeding scenario includes:

[0012] A smart collar is worn on the neck of a periparturient dairy cow to collect first monitoring data; the first monitoring data includes feeding time, dry matter intake, rumination time, rest time, activity time and activity amount of the periparturient dairy cow;

[0013] A visible light camera is installed in the feeding area of ​​dairy cows in the pasture, and is used to collect second monitoring data; the second monitoring data is a visible light video obtained by shooting the feeding process of periparturient dairy cows;

[0014] An infrared camera is installed in the feeding area of ​​dairy cows in the pasture, and is used to collect third monitoring data; the third monitoring data is an infrared video obtained by shooting the feeding process of periparturient dairy cows;

[0015] The processor is respectively connected to the smart collar, the visible light camera and the infrared camera for executing the above-mentioned method for predicting the feed intake of periparturient dairy cows in the pasture breeding scenario.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0017] The present application provides a method and system for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario, obtaining the first monitoring data collected by the smart collar, the second monitoring data collected by the visible light camera, and the third monitoring data collected by the infrared camera, so that the feeding behavior monitoring of periparturient dairy cows can be completed in combination with the smart collar, the visible light camera, and the infrared camera, and then the first monitoring data, the second monitoring data, and the third monitoring data are respectively extracted to obtain the first feature, the second feature, and the third feature, and the first feature, the second feature, and the third feature are feature fused to obtain the fused feature, and the fused feature is used as input, and the feed intake of periparturient dairy cows is determined using a trained feed intake prediction model, so that the feed intake prediction of periparturient dairy cows can be completed in combination with the smart collar, the visible light camera, and the infrared camera. The present application obtains the monitoring data of feeding behavior by setting a smart collar, a visible light camera, and an infrared camera, and further analyzes the monitoring data of feeding behavior through feature extraction, feature fusion, and a trained feed intake prediction model, so as to realize real-time monitoring and accurate analysis of feeding behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is an application environment diagram of a method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario provided in Example 1 of the present application.

[0020] Figure 2 A flow chart of a method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario provided in Example 1 of the present application.

[0021] Figure 3 Schematic diagram of the F1 (H-mean value)-confidence curve during Yolov7 model training provided in Example 1 of the present application.

[0022] Figure 4 Schematic diagram of the accuracy-confidence curve during Yolov7 model training provided in Example 1 of the present application.

[0023] Figure 5 Schematic diagram of the precision-recall curve during Yolov7 model training provided in Example 1 of the present application.

[0024] Figure 6 A schematic diagram of the recall rate-confidence curve during Yolov7 model training provided in Example 1 of the present application.

[0025] Figure 7 A schematic diagram of the binarization process provided in Example 1 of the present application.

[0026] Figure 8 A schematic diagram of the structure of a computer device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] Example 1.

[0029] The method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal communicates with the server through the network. The data storage system can store the data that the server needs to process. The data storage system can be set up separately, integrated on the server, or placed on the cloud or other servers. The terminal can send the first monitoring data, the second monitoring data, and the third monitoring data to be processed to the server. After the server receives the first monitoring data, the second monitoring data, and the third monitoring data to be processed, the server extracts features of the first monitoring data, the second monitoring data, and the third monitoring data to be processed, respectively, to obtain the first feature, the second feature, and the third feature; the first feature, the second feature, and the third feature are feature fused to obtain the fused feature; the fused feature is used as input, and the feed intake of periparturient dairy cows is determined using the trained feed intake prediction model. The server can feed back the obtained feed intake to the terminal.

[0030] In addition, in some embodiments, the method for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario can also be implemented independently by a server or a terminal. For example, the terminal can directly process the first monitoring data, the second monitoring data, and the third monitoring data to be processed, or the server can obtain the first monitoring data, the second monitoring data, and the third monitoring data to be processed from the data storage system, and process the first monitoring data, the second monitoring data, and the third monitoring data to be processed.

[0031] The terminals may be, but are not limited to, various desktop computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers, or a cloud server.

[0032] like Figure 2 As shown, a method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 Taking the server in as an example, the method for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario includes the following steps.

[0033] Step S1, obtaining first monitoring data collected by a smart collar, second monitoring data collected by a visible light camera, and third monitoring data collected by an infrared camera; the smart collar is worn on the neck of a periparturient dairy cow, and the first monitoring data includes the feeding time, dry matter intake, rumination time, rest time, activity time, and activity amount of the periparturient dairy cow; the visible light camera and the infrared camera are both installed in the dairy cow feeding area in the pasture, the second monitoring data is a visible light video obtained by shooting the feeding process of the periparturient dairy cow, and the third monitoring data is an infrared video obtained by shooting the feeding process of the periparturient dairy cow.

[0034] Step S2, respectively extracting features from the first monitoring data, the second monitoring data and the third monitoring data to obtain first features, second features and third features; the first features include feeding time, feeding frequency, rumination time and activity change value obtained by extracting features from the first monitoring data, and the activity change value includes the mean and standard deviation of the activity; the second features include the head posture and feeding rate obtained by extracting features from the second monitoring data; the third features include the feed pile shape parameters obtained by extracting features from the third monitoring data.

[0035] Step S3, performing feature fusion on the first feature, the second feature and the third feature to obtain a fused feature.

[0036] Step S4, using the fused features as input, and using the trained feed intake prediction model to determine the feed intake of periparturient dairy cows.

[0037] By implementing the above-mentioned steps S1 to S4, this embodiment can combine the smart collar, the visible light camera and the infrared camera to monitor the feeding behavior of the periparturient dairy cows, obtain the first monitoring data, the second monitoring data and the third monitoring data, improve the efficiency and accuracy of obtaining the feeding behavior monitoring data, further perform feature extraction and feature fusion on the first monitoring data, the second monitoring data and the third monitoring data, and combine the trained feeding intake prediction model to predict the feeding intake of the periparturient dairy cows, complete the analysis process of the feeding behavior monitoring data, and improve the efficiency and accuracy of analyzing the feeding behavior monitoring data.

[0038] The development of smart collar technology has facilitated the collection of cow behavior data. It can obtain data such as cows' feeding time and activity in real time, laying the foundation for subsequent analysis. However, a single smart collar monitoring data can only reflect part of the cow's behavioral information and cannot fully understand its feeding behavior. The emergence of infrared-visible light video fusion technology provides a new idea for solving this problem. In the identification of cow feeding behavior, head posture and feeding rate recognition is one of the key links. By analyzing visible light video, it is possible to identify cows' movements such as raising their heads, lowering their heads, and chewing during feeding, and then understand their feeding preferences and health status. In addition, the analysis of feed pile shape parameters is also of great significance for evaluating the feeding situation of dairy cows. By processing infrared videos, the shape parameters of feed piles can be obtained to judge feeding efficiency and feed utilization.

[0039] Although fusing multimodal data to identify cow feeding behavior can solve the problems existing in single modal data, the applicant found that there are still some problems in fusing multimodal data. First, data fusion faces data compatibility challenges. The formats and standards of smart collar monitoring data and video image data may be different, and effective fusion and conversion are required. Secondly, feature extraction is difficult, and it is necessary to comprehensively consider multiple factors such as head posture, feeding movements, and feed pile shape. Accurate extraction of these features is the key. Finally, optimization of model training is also an important aspect. The current model has deficiencies in accuracy and adaptability, and needs further improvement.

[0040] Based on this, the research purpose of this embodiment is to develop a method for predicting the feed intake of periparturient dairy cows in a large-scale ranch farming scenario. Through smart collars and infrared-visible light video fusion technology, the feeding behavior of dairy cows can be monitored and the feed intake can be predicted, and accurate monitoring and analysis can be achieved. It can accurately identify the feeding behavior of dairy cows, including feeding time, feeding frequency, head posture, rumination time, etc., and predict the feed intake of dairy cows based on this information. Unlike general environmental recognition methods, this method only needs to use videos in natural farming scenarios to predict the feed intake of periparturient dairy cows. This not only reduces the cost and difficulty of data collection, but also improves practicality and operability. Through in-depth analysis and processing of multimodal data, it provides a scientific basis for dairy farming management and helps to achieve precise farming decisions.

[0041] Hereinafter, this embodiment introduces the method of this embodiment from four aspects: installation and deployment of sensor equipment, feature extraction, feature fusion and model construction.

[0042] 1. Installation and deployment of sensor equipment.

[0043] The sensing equipment of this embodiment includes a smart collar, a visible light camera and an infrared camera. The sensing equipment is selected first: according to the actual situation of the ranch, appropriate smart collars, infrared cameras and visible light cameras are selected to ensure that the smart collar has the ability to accurately collect basic data such as feeding time, activity time, rumination time, etc., to ensure that the infrared camera has high resolution and high sensitivity and can produce clear images under low light conditions, to ensure that the installation angle and field of view of the visible light camera can complement the infrared camera to achieve all-round and multi-environment image acquisition.

[0044] Then proceed to install the sensor equipment, as follows.

[0045] For smart collars: Fit a smart collar for each peripartum cow, ensure that the smart collar is worn securely and does not affect the cow's normal activities, and check whether the sensors of the smart collar, such as the acceleration sensor and temperature sensor, are working properly. The smart collar is used to collect basic data such as the cow's feeding time, dry matter intake, rumination time, rest time, activity time and activity level.

[0046] For infrared cameras and visible light cameras: Install infrared cameras and visible light cameras at key locations in the cows' feeding area, such as around the feed trough and in the corners of the feeding area, to ensure that the infrared cameras and visible light cameras can cover the cows' feeding area and avoid being blocked by other objects. At the same time, adjust the angles of the infrared cameras and visible light cameras according to the cows' feeding habits and behavioral characteristics so that the captured video can clearly show information such as the cows' head movements and changes in the feed pile. After the infrared cameras and visible light cameras are installed, test them to check indicators such as image quality and transmission stability. Based on the test results, fine-tune the positions and angles of the infrared cameras and visible light cameras to ensure the best shooting effect.

[0047] Specifically, for infrared cameras: install infrared cameras at key locations in the cow's feeding area to ensure that the cow's feeding area is covered; use infrared cameras to continuously capture video of the feed pile while the cow is feeding to ensure the quality and integrity of the video data; and simultaneously record the cow's movement and changes in the volume of the feed pile. The infrared camera should have high resolution and sensitivity, be able to produce clear images in low light conditions, and be used to capture the cow's overall behavior, contour information, and feed pile parameter information.

[0048] For visible light cameras: install a visible light camera in a position complementary to the infrared camera, and use the visible light camera to continuously shoot videos of the periparturient dairy cows’ feeding process to ensure the integrity and continuity of the video data, and obtain detailed information about the cows’ feeding process, such as head posture, feeding movements, changes in feed pile volume, etc. The installation angle and field of view of the visible light camera should be coordinated with the infrared camera to achieve all-round, multi-environment image acquisition.

[0049] During the image acquisition process of the visible light camera, adjust the parameters of the visible light camera, such as resolution, frame rate, exposure time, etc., according to the lighting conditions and environmental characteristics of the cow's feeding area to ensure that clear and stable video images are captured. You can use lighting equipment or adjust the window position to control the light to ensure that the light at the shooting site is sufficient and uniform, and avoid excessive or weak light affecting the video quality. Choose a simple and non-interfering background to avoid the background from interfering with the recognition of the cow's head posture and feeding movements. You can also control the acquisition process and use a visible light camera to continuously shoot videos of the peripartum cow's feeding process to ensure the integrity and continuity of the video data. The shooting time should cover the main feeding time period of the cow to obtain sufficient sample data. During the shooting process, monitor the quality of the video in real time, such as image clarity and stability. If the video quality is found to be deteriorating, adjust the camera parameters or change the shooting position in time.

[0050] In the data collection process of the sensor device, this embodiment introduces data synchronization technology. Data synchronization means: using synchronous collection equipment to ensure that the monitoring data collected by the smart collar, visible light camera and infrared camera are strictly synchronized in time. Specifically, the Network Time Protocol (NTP) and other technologies can be used to achieve data synchronization to ensure that the time bases of the smart collar, visible light camera and infrared camera are consistent. More specifically, a time synchronization module is set in the smart collar, visible light camera and infrared camera. The time synchronization module is connected to the synchronous collection device through a wireless or wired network. The synchronous collection device sends a time signal at a fixed time interval. After receiving the time signal, the smart collar, visible light camera and infrared camera make synchronous adjustments and add time tags to the collected monitoring data to ensure the time sequence and consistency of the monitoring data.

[0051] After the sensor device completes the data collection process, this embodiment introduces data cleaning technology to perform data preprocessing. Data cleaning refers to cleaning the collected monitoring data to remove abnormal values ​​and noise data. For the monitoring data of the smart collar, it is necessary to check the integrity and accuracy of the monitoring data; for the monitoring data of the visible light camera and infrared camera, it is necessary to deal with the noise, blur and other problems in the image.

[0052] Data integrity check refers to checking whether the monitoring data collected by the smart collar is complete, including whether key data such as feeding time and activity level are missing. Data accuracy check refers to verifying the accuracy of the collected monitoring data, such as checking whether the feeding time is reasonable and whether the activity level is within the normal range. Abnormal data, such as data mutations and incorrect sensor readings, are marked or deleted.

[0053] Among them, noise removal refers to: using image filtering and denoising algorithms, such as median filtering and Gaussian filtering, to remove noise in the image and improve image quality. Blur processing and repair refers to: for blurred images, using image enhancement algorithms, such as histogram equalization and contrast enhancement, to improve the clarity of the image. Occlusion area processing can also be introduced: for areas in the image that are occluded by the cow's body or other objects, they are repaired or ignored based on the information of adjacent images.

[0054] In this embodiment, a smart collar, a visible light camera and an infrared camera are simultaneously provided to obtain first monitoring data collected by the smart collar, second monitoring data collected by the visible light camera and third monitoring data collected by the infrared camera. The first monitoring data includes the feeding time, dry matter intake, rumination time, rest time, activity time and activity amount of periparturient dairy cows. The second monitoring data is a visible light video obtained by shooting the feeding process of periparturient dairy cows. The third monitoring data is an infrared video obtained by shooting the feeding process of periparturient dairy cows.

[0055] 2. Feature extraction

[0056] (1) Smart collar feature extraction: Extract features related to feeding behavior from the monitoring data of smart collars. Specifically, obtain the feeding time, rumination time and activity data of dairy cows from smart collars. Record the start time and end time of feeding of dairy cows every day. Calculate the time interval of feeding time, duration of feeding and other time features to determine the feeding time and feeding frequency. Feeding frequency refers to the number of times feeding in a unit time. Record the start time and end time of rumination of dairy cows every day to determine the rumination duration. Record the daily activity of dairy cows, analyze the changing trend of activity, extract the mean and standard deviation of activity, determine the change value of activity, and finally extract the first feature including feeding time, feeding frequency, rumination duration and change value of activity.

[0057] (2) Target detection: Use the target detection algorithm to detect the cow's head using visible light video, that is, detect the position and posture of the cow's head in the visible light video, and use the infrared video to detect the feed pile, that is, detect the position of the feed pile in the infrared video. Specifically, you can use a target detection model based on deep learning, select a target detection model suitable for visible light video, such as the Yolov7 model, select a target detection model suitable for infrared video, such as the Yolov7 model, use a large amount of labeled data to train the Yolov7 model, and improve the Yolov7 model's detection accuracy and robustness for the cow's head and feed pile. You can also adjust the parameters of the Yolov7 model, and adjust the parameters according to the characteristics of the cow's head or the feed pile, such as the convolution kernel size, step size, anchor box size, etc., to improve the accuracy of target detection.

[0058] The training process of the Yolov7 model includes: model construction, verification and performance testing. The model consists of input, backbone network, neck network and output (Prediction). The model performance is evaluated by precision, recall and mean average precision to improve the generalization ability of the model. After the training is completed, the collected visible light video is input into the trained head detection model. The model analyzes and identifies the cow's head and marks the position of the cow's head with a bounding box. The collected infrared video is input into the trained feed pile detection model. The model analyzes and identifies the feed pile and marks the position of the feed pile with a bounding box.

[0059] Multiple cow targets appearing in the video can be effectively detected and identified, ensuring that each cow's head posture, feeding movements and corresponding feed pile can be accurately recorded.

[0060] (3) Head posture feature extraction: Based on the changes in the cow's head posture, different posture categories are defined, such as front head lowering to feed, side torted neck to feed, and head raising. The detected cow head contour is analyzed to determine the head posture features, such as front head lowering to feed, side torted neck to feed, head raising, etc., and classify and mark them. This feature can be obtained through image analysis and processing, and can reflect the cow's attention and interest concentration during the feeding process.

[0061] A dataset of periparturient dairy cow feeding posture images was established. Specifically, segmented images were extracted from visible light videos to remove low-quality images caused by cows blocking each other, dim light, and standing angles. Video format conversion, video segmentation, frame extraction and other operations were performed to improve the effect of subsequent processing. The images were labeled using the labelimg tool in the python environment to mark the head posture of periparturient dairy cows when feeding. The image format was saved in txt format. The definition of label classification is shown in Table 1.

[0062] The Yolov7 algorithm is used to build a recognition model for the head posture of dairy cows during feeding, which includes model training, model verification and model testing. First, the labeled data is divided into training set, verification set and test set in a ratio of 7:2:1. The relevant parameters of the input, backbone network, neck network and output (Prediction) of the model are set. During the training process, the learning rate is adjusted to 0.001, the number of iterations is 500, and the stochastic gradient descent optimization algorithm is used to improve the generalization ability of the model. The training results are as follows: Figure 3 , Figure 4 , Figure 5 and Figure 6 shown.

[0063] Table 1 Typical head postures

[0064]

[0065] (4) Feeding rate characteristics: By analyzing the feeding actions of dairy cows in visible light videos, we can identify feeding actions such as chewing and swallowing, and specifically extract the action characteristics of dairy cows during feeding, such as the frequency, amplitude, and speed of chewing actions. These characteristics can be obtained by analyzing the movement of the cow's head and mouth. Based on the extracted action characteristics, we use machine learning algorithms or deep learning algorithms to classify and identify feeding actions to determine whether the cow is chewing, swallowing, or performing other actions. We count the frequency and duration of feeding actions to reflect the feeding efficiency and rhythm of dairy cows. Based on the frequency and duration of feeding actions, we calculate the feeding rate of dairy cows, that is, the amount of food eaten per unit time. Feeding rate is an important indicator for evaluating the feeding efficiency of dairy cows, and can reflect the feeding efficiency and digestion capacity of dairy cows. Specifically, we use machine learning models or deep learning models to analyze and learn the frequency and duration of feeding actions, establish a relationship model between feeding rate and the frequency and duration of feeding actions, and calculate the feeding rate through the model.

[0066] Obtain videos of the feeding process of periparturient dairy cows: select different angles to capture videos of dairy cows feeding on the pile, fix the camera position, and transfer the captured videos to a mobile hard disk for storage at a fixed time. After the experiment, replay the videos to analyze the feeding behavior posture and pile parameters.

[0067] (5) Shape parameter characteristics of feed piles: Use edge detection algorithms, such as Canny edge detection algorithm, Sobel algorithm, contour tracking algorithm, etc., to enhance the edge information of the feed pile and highlight the shape contour of the feed pile. Use the contour tracking algorithm to process the infrared video, extract the contour of the feed pile, and separate the feed pile from the background, providing a basis for the extraction of shape parameter characteristics of the feed pile. Use image recognition technology to analyze the extracted shape of the feed pile and calculate the shape parameters of the feed pile, such as area, perimeter, diameter, shape, curvature, etc. These shape parameters can reflect the size, shape and distribution of the feed pile. Changes in the shape parameter characteristics of the feed pile can reflect the feeding situation and preference of dairy cows for feed. Area calculation: Calculate the area of ​​the feed pile by calculating the number of pixels within the contour of the feed pile or using a mathematical model. Perimeter calculation: Calculate the perimeter of the contour of the feed pile to reflect the size and shape characteristics of the feed pile. Diameter calculation: Calculate the diameter of the feed pile, such as the major axis diameter and the minor axis diameter. Curvature calculation: Use labelimg in Python to label the curvature of the feed pile, or use image j to label the curvature of the feed pile.

[0068] The cow's head feeding data and feed pile curvature data were synchronized to ensure the correspondence of the data. Statistical methods were used to analyze the correlation between the feed pile curvature and the cow's feed intake using the Pearson correlation coefficient, so as to understand the impact of the feed pile shape characteristics on the cow's feeding behavior, which facilitated the subsequent comprehensive analysis of the cow's real-time feed intake.

[0069] Based on this, in this embodiment, feature extraction is performed on the first monitoring data, the second monitoring data, and the third monitoring data respectively to obtain the first feature, the second feature, and the third feature, which specifically includes the following steps.

[0070] (1) Perform feature extraction on the first monitoring data to obtain a first feature.

[0071] (2) Using the second monitoring data as input, the trained head detection model is used to detect the head of the periparturient dairy cow to obtain head detection data.

[0072] (3) The head posture of periparturient dairy cows was analyzed based on the head detection data to obtain the head posture, which included frontal head lowering to feed, sideways torted neck to feed, and head raising.

[0073] (4) The feeding actions of periparturient dairy cows were analyzed based on the head detection data to obtain the frequency and duration of the feeding actions. The feeding rate of periparturient dairy cows was determined based on the frequency and duration of the feeding actions. The feeding actions included chewing and swallowing.

[0074] Among them, the feeding actions of periparturient dairy cows are analyzed based on head detection data to obtain the frequency and duration of the feeding actions, and the feeding rate of periparturient dairy cows is determined based on the frequency and duration of the feeding actions, specifically including: performing motion analysis on the mouth movements of periparturient dairy cows based on head detection data to obtain motion characteristics; classifying and identifying the motion characteristics to determine the feeding actions, and recording the frequency and duration of the feeding actions; taking the frequency and duration of the feeding actions as input, and using the trained feeding rate calculation model to determine the feeding rate.

[0075] (5) The head posture and the feeding rate are combined to form a second feature obtained by extracting features from the second monitoring data.

[0076] (6) Using the third monitoring data as input, the feed pile is detected using the trained feed pile detection model to obtain feed pile detection data.

[0077] (7) Extracting the outline of the feed pile from the feed pile detection data, and determining the shape parameters of the feed pile based on the outline of the feed pile, wherein the shape parameters of the feed pile include the area, perimeter, diameter, curvature and shape of the feed pile.

[0078] (8) The feed pile shape parameter is used as the third feature obtained by extracting the third monitoring data.

[0079] When introducing data cleaning technology, in this embodiment, feature extraction is performed on the first monitoring data, the second monitoring data and the third monitoring data respectively to obtain the first feature, the second feature and the third feature, specifically including: data cleaning is performed on the first monitoring data, the second monitoring data and the third monitoring data respectively to obtain the first cleaned data, the second cleaned data and the third cleaned data, and the data cleaning includes removing outliers, removing noise and deblurring; feature extraction is performed on the first cleaned data, the second cleaned data and the third cleaned data respectively to obtain the first feature, the second feature and the third feature.

[0080] This embodiment also introduces image enhancement technology to perform image enhancement processing on the collected visible light video to improve the quality and contrast of the image so as to better perform feature extraction. Image enhancement includes: grayscale conversion: converting the color video image into a grayscale image to reduce the amount of data and improve the speed and efficiency of subsequent processing; denoising: using filtering algorithms such as median filtering and Gaussian filtering to remove noise in the image, such as salt and pepper noise, Gaussian noise, etc.; contrast enhancement: by adjusting the contrast of the image, the contours of the cow's head and body are made clearer, which is convenient for subsequent target detection and feature extraction; edge enhancement: using edge detection algorithms such as Sobel operator and Canny operator to enhance the edge information of the image and highlight the boundaries of the cow's head posture and feeding action. Based on this, in this embodiment, the second monitoring data is used as input, and the head of the peripartum cow is detected using the trained head detection model to obtain head detection data, which specifically includes: performing a first image processing on the second monitoring data to obtain first enhanced data, and the first image processing includes graying, denoising, contrast enhancement and edge enhancement; taking the first enhanced data as input, and detecting the head of the peripartum cow using the trained head detection model to obtain head detection data.

[0081] This embodiment also introduces image enhancement technology to perform image enhancement processing on the collected infrared video to improve the clarity and contrast of the image in order to better extract features. Image enhancement includes: grayscale conversion: converting the infrared video image into a grayscale image for subsequent processing and analysis; enhancing contrast, adjusting brightness, etc. to improve image clarity and readability; binarization: performing binarization processing based on the grayscale difference between the feed pile and the background to separate the feed pile from the background. Based on this, in this embodiment, the third monitoring data is used as input, and the feed pile is detected using the trained feed pile detection model to obtain feed pile detection data, which specifically includes: performing a second image processing on the third monitoring data to obtain second enhanced data, and the second image processing includes grayscale, contrast enhancement, brightness adjustment and binarization; using the second enhanced data as input, and using the trained feed pile detection model to detect the feed pile to obtain feed pile detection data.

[0082] 3. Feature fusion

[0083] The first feature, the second feature, and the third feature are concatenated. At this time, if the dimension of the feature vector of the first feature is n, and the dimension of the feature vector of the second and third features is m, then the dimension of the feature vector of the multimodal fusion feature obtained after concatenation is n+m, ensuring that the dimensions of the feature vectors match for subsequent model training.

[0084] This embodiment adopts different feature combination strategies, such as weighted combination, feature selection, etc., to highlight the contribution of important features. For example, according to the importance score of the feature, a corresponding weight is assigned to each feature, thereby achieving weighted combination.

[0085] Based on this, in this embodiment, the first feature, the second feature and the third feature are fused to obtain fused features, specifically including: performing feature selection on the first feature, the second feature and the third feature to obtain selected features; and performing weighted combination on all selected features to obtain fused features.

[0086] (IV) Model construction.

[0087] A prediction model for the feed intake of periparturient dairy cows in a large-scale ranch breeding scenario is constructed. The detailed steps are as follows.

[0088] (1) Data preparation.

[0089] Several peripartum dairy cows were selected, and each cow was fitted with a smart collar. At the same time, infrared cameras and visible light cameras were installed in the cows' feeding areas. The smart collars collected basic data such as the cows' feeding time and activity in real time; the infrared camera photographed the feed piles during the cows' feeding process to analyze the shape parameters of the feed piles; the visible light camera photographed the cows' head posture and feeding movements during the feeding process. The monitoring data of the smart collars, the visible light cameras, and the infrared cameras were integrated to ensure the integrity, consistency, and accuracy of the data.

[0090] Clean and pre-process the collected monitoring data to remove outliers and noise data to improve data quality. For the monitoring data of smart collars, check the integrity and accuracy of the data; for the monitoring data of visible light cameras and infrared cameras, deal with problems such as blur and occlusion in the image. Specifically, duplicate data processing: check whether there is duplicate data in the data, such as duplicate feeding records, duplicate video frames, etc., and delete or merge them. Abnormal data processing: process outliers in the data, such as missing data, data errors, etc., which can be processed by interpolation, filling, deletion and other methods.

[0091] Select appropriate data annotation tools to annotate data and determine feed intake.

[0092] (2) Feature extraction.

[0093] Extract features related to feeding behavior from the monitoring data of the smart collar, such as feeding time, rumination time, activity change value, etc. These features directly reflect the nutritional status and digestive capacity of dairy cows. Use target detection algorithms, such as Yolov7, to extract features such as cow head posture and feeding actions from visible light videos, and calculate the frequency and duration of feeding actions, and further calculate the feeding rate. Extract the outline of the feed pile from the infrared video, and further calculate the shape parameters of the feed pile, such as the area, height, shape and other features of the feed pile. After obtaining the features, further data normalization is performed: the various extracted features are normalized and mapped to the [0, 1] interval to accelerate the model training process and improve the convergence speed of the model.

[0094] (3) Feature fusion.

[0095] The feature vector of the smart collar is concatenated with the feature vectors of the infrared camera and the visible light camera to form a new multimodal feature vector. For example, if the dimension of the feature vector of the smart collar is n, and the dimension of the feature vector of the infrared camera and the visible light camera is m, then the dimension of the concatenated multimodal feature vector is n+m. As an example, if the dimension of the feature vector of the smart collar is 10, and the dimension of the feature vector of the infrared camera and the visible light camera is 20, then the dimension of the concatenated multimodal feature vector is 30.

[0096] Different feature combination strategies, such as feature combination and feature selection, can be used to highlight the contribution of important features. Feature selection means: according to the importance and relevance of features, select features that contribute significantly to the analysis of feeding behavior characteristics, reduce feature dimensions, and reduce model complexity. Specifically, a feature importance evaluation method based on random forest can be used to select the most important features from multimodal features for fusion. Through feature selection, some redundant features are removed, which improves the training efficiency and accuracy of the model. Feature combination means: combining the selected features to construct a multimodal feature vector to make full use of the information of multimodal data. For example, according to the importance score of the feature, a corresponding weight is assigned to each feature, and then the weighted features are combined to achieve weighted combination. Through experiments and analysis, it was determined that the weights of features such as feeding time and rumination time in the smart collar are relatively high, and the weights of features such as head posture and feed pile shape parameters are also relatively high.

[0097] In the process of feature fusion, the feature selection method is combined to select the most important features for fusion to improve the performance and generalization ability of the model.

[0098] (4) Model construction.

[0099] According to the characteristics of multimodal data and the requirements of the analysis task, select a suitable model, such as random forest, support vector machine, deep learning model, etc. For the selected model, adjust the model parameters, such as the number of decision trees, kernel function parameters, number of layers and nodes of the neural network, etc., to optimize the performance of the model.

[0100] As an example, this embodiment specifically selects a model based on a random forest algorithm to construct a feed intake prediction model. Random forest is an integrated learning method with good classification and regression performance. The parameters of the random forest are set, such as the number of decision trees, the depth of the tree, etc., and the optimal parameter combination can be selected by methods such as cross-validation.

[0101] In this embodiment, the Yolov7 model can also be selected to construct a feed intake prediction model to adapt to the characteristics of multimodal data, and the Yolov7 model is trained using the fused multimodal feature vector, and the weights and parameters of the model are adjusted to improve the model's ability to recognize feeding behavior characteristics.

[0102] (5) Model training.

[0103] The labeled data set is divided into training set, validation set and test set. The ratio can be set according to the actual situation, such as 7:2:1. The training set is used for model training, the validation set is used to adjust the model parameters, and the test set is used to evaluate the performance of the model.

[0104] Use the training set to train the constructed model, and update the model parameters iteratively so that the model can learn the characteristics and rules in multimodal data. During the training process, monitor the performance indicators of the model, such as accuracy, recall, F1 value, etc., to evaluate the training effect of the model. You can also monitor the loss function value of the model and adjust the training strategy in time to ensure the training effect of the model. Use regularization, cross-validation and other technologies to optimize the model, prevent overfitting, and improve the generalization ability of the model.

[0105] During the training process, select appropriate optimization algorithms, such as Stochastic Gradient Descent (SGD), AdaGrad (adaptive gradient algorithm), AdaDelta (adaptive learning rate adjustment algorithm), etc., to update the model parameters and improve training efficiency and performance. Adjustments are made based on the performance indicators of the validation set to prevent overfitting.

[0106] (6) Model validation.

[0107] Use the validation set to validate the trained model and evaluate the model's performance indicators, such as accuracy, recall, F1 value, etc., to evaluate the model's classification performance and measure the model's recognition accuracy for different categories of data. The mean square error is also introduced to evaluate the model's regression performance and measure the model's prediction accuracy for continuous values. At the same time, observe the consistency between the model's prediction results on the validation set and the actual results to evaluate the model's generalization ability. According to the validation results, tune the model, such as adjusting the model's parameters, optimizing the feature extraction method, etc., to improve the model's performance.

[0108] Through the prediction of the test set data, it was found that the model can accurately predict the feed intake of periparturient dairy cows. The prediction accuracy of the model reached a high level for different dairy cows and different feeding scenarios, with an average accuracy of more than 85%.

[0109] (7) Model fusion.

[0110] This embodiment may consider using multiple models for fusion to improve the performance and robustness of the model. For example, the prediction results of the random forest model and the Yolov7 model may be fused by weighted averaging, voting, etc.

[0111] This embodiment explores different fusion strategies and selects the best fusion method to improve the performance of the model. For example, different weights can be assigned to each model according to the performance of the model, so as to achieve weighted fusion. Specifically, a corresponding weight is assigned to each model according to the performance of each model, and then the prediction results of the model are weighted averaged to obtain the final prediction result.

[0112] The fusion strategy of this embodiment is selected as follows.

[0113] Performance-based fusion strategy: According to the performance of the model, select the model with the best performance or multiple models with good performance for fusion to improve the overall performance of the model.

[0114] Feature-based fusion strategy: According to the characteristics of multimodal data and the importance of features, different features are selected for fusion to make full use of the information of multimodal data.

[0115] Fusion strategy based on monitoring data: feature fusion method can be used to splice or combine the feature vector of the smart collar with the feature vector of the visible light camera and the infrared camera to form a new multimodal feature vector and a multimodal data set. Data fusion technology, such as principal component analysis and independent component analysis, can be used to reduce the dimensionality of the multimodal data set, reduce the dimension of the data, and improve the training efficiency of the model.

[0116] Through the above steps, a periparturient dairy cow feed intake prediction model can be constructed in a large-scale ranch breeding scenario. This model can comprehensively utilize smart collar data, visible light camera video data, and infrared camera video data to accurately analyze the feeding behavior characteristics of periparturient dairy cows.

[0117] This example analyzes the importance of the features of the model and finds that the features such as feeding time and rumination time in the smart collar and the features such as head posture and feed pile shape parameters in the video make important contributions to the prediction of feed intake, which shows that the fusion of multimodal data can make full use of the information of different modal data and improve the accuracy of feed intake prediction. The feed intake prediction method based on multimodal fusion is compared with the traditional single modality prediction method, and it is found that the multimodal fusion method can better reflect the feeding behavior characteristics of dairy cows and improve the accuracy and reliability of feed intake prediction.

[0118] This embodiment continuously collects image data of peripartum cows during feeding, including infrared images and visible light images, with a frequency of one frame per second to ensure the continuity and integrity of the image. Input the head images of cows in different postures during feeding, use the multi-level threshold mean filtering method based on domain similarity to calculate the eight-directional gradient, and then use the Canny contour extraction technology to identify the contour. The contour area is screened by the contour function to accurately highlight the head target. Finally, the image is binarized to complete the extraction of the binary image of the cow's head. For the specific process diagram, see Figure 7 . The curvature of the pile was marked using the labelimg tool, and a mathematical model was established to calculate the volume of the pile (for example, the shape of the pile was approximated as a cone or a cuboid, and the volume was calculated by measuring the relevant dimensions). The extracted head posture, feeding rate, and feed pile shape parameters were associated with the characteristic data of the corresponding smart collar (such as feeding time, rumination time, etc.), and the data from different sources were matched by the cow number and sorted into a data set. The sorted data set was divided into a training set, a validation set, and a test set in a ratio of 7:2:1. The data set contains rich feature information, covering multiple key parameters of the cow's head posture and the pile. Through manual inspection of some samples in the data set, it was found that the data was highly accurate and could better reflect the feeding behavior characteristics of dairy cows. The correlation analysis of the head posture features and pile parameters in the data set was performed, and the Pearson correlation coefficient was used to calculate that some parameters in the head posture features (such as raising the head and lowering the head) were correlated with the changes in the pile, which provided a valuable basis for subsequent model training.

[0119] Based on this, in this embodiment, before using the fused features as input and using the trained feed intake prediction model to determine the feed intake of periparturient dairy cows, the method for predicting feed intake of periparturient dairy cows in the pasture breeding scenario also includes the following steps.

[0120] (1) Construct a data set, which includes multiple samples and labels corresponding to each sample. The sample is the feature after sample fusion, and the label is the sample feed intake.

[0121] (2) Construct multiple initial feed intake prediction models.

[0122] (3) For each initial feed intake prediction model, the data set is used to train the initial feed intake prediction model to obtain a trained feed intake prediction model.

[0123] (4) All trained feed intake prediction models are combined into a trained feed intake prediction model.

[0124] Based on this, in this embodiment, the fused features are used as input, and the trained feed intake prediction model is used to determine the feed intake of the periparturient dairy cows, specifically including: for each trained feed intake prediction model, the fused features are used as input, and the trained feed intake prediction model is used to determine the initial feed intake of the periparturient dairy cows; all initial feed intakes are weighted summed to obtain the feed intake of the periparturient dairy cows.

[0125] The goal of this embodiment is to comprehensively, accurately and intelligently analyze the feeding characteristics of peripartum dairy cows based on smart collar data, visible light video data and infrared video data, so as to provide a scientific basis for dairy farming management. The smart collar provides text data including feeding time and feeding frequency, the visible light camera provides video data of the head posture and behavior of peripartum dairy cows during feeding, and the infrared camera provides video data of the shape of the feed pile. The data of different dimensions are integrated in time and space to reveal the response mechanism to the changes in feeding posture and behavior of peripartum dairy cows, so as to accurately predict the feed intake of dairy cows.

[0126] This embodiment also performs a change rule analysis, which is as follows.

[0127] (1) Smart collar.

[0128] Time series analysis: Conduct time series analysis on feeding duration and feeding frequency data to observe the trends and periodic changes of the data. Specifically, methods such as moving average and exponential smoothing can be used to smooth the data, remove noise interference, and highlight the overall trend of the data.

[0129] Specifically, the data such as the cow’s feeding time and activity level collected by the smart collar is transmitted to the data server or storage device in real time through wireless communication technology. According to research needs, the feeding data within a specific time period is filtered out, such as daily, weekly or stage feeding data. If it is necessary to analyze the feeding behavior of different individual cows, the data is individually screened to extract the feeding data of each cow. Select an appropriate moving average window size, perform moving average processing on the feeding duration and feeding frequency data, remove noise interference in the data, and highlight the overall trend of the data. The moving average calculation formula is: ,in, for The moving average of time, is the window size of the moving average, for The original data at the moment.

[0130] Trend line drawing: Draw a trend chart of feeding duration and feeding frequency data, observe the overall trend of the data, and determine whether the data is rising, falling, or remaining stable.

[0131] Trend feature extraction: Analyze the characteristics of the trend graph, such as the slope of the trend, turning points, etc., to understand the changing trends and patterns of feeding behavior.

[0132] Statistical analysis: Calculate the mean, variance, standard deviation and other statistical indicators of feeding time and feeding frequency data to understand the average level and dispersion of dairy cows' feeding behavior. At the same time, conduct a significance test to analyze whether the differences in feeding time and feeding frequency between different time periods and different dairy cows are statistically significant.

[0133] Specifically, mean calculation: calculate the mean of feeding time and feeding frequency to reflect the average level of feeding behavior of dairy cows. Variance and standard deviation calculation: calculate the variance and standard deviation of feeding time and feeding frequency to reflect the degree of dispersion of data, and the standard deviation is the square root of the variance. Hypothesis test method selection: select appropriate significance test methods according to data characteristics and research needs, such as t-test, variance analysis, etc.

[0134] Feeding pattern analysis: Based on the data analysis results, the feeding patterns of dairy cows are summarized, such as the daily peak feeding time, the trough feeding time, the changing patterns of feeding duration and feeding frequency, etc. These patterns can provide a reference for the feeding management of peripartum dairy cows and reasonably arrange the feeding time and feed supply.

[0135] Analysis of influencing factors: Explore the factors that affect the feeding time and frequency of dairy cows, such as the parity of dairy cows, feeding environment, feed quality, health status, etc. Through correlation analysis and regression analysis, a mathematical model between feeding time and frequency and these influencing factors is established to provide a basis for predicting the feed intake of dairy cows.

[0136] Specifically, based on the results of correlation analysis, evaluate the degree of influence of each influencing factor on feeding time and feeding frequency, and screen out important influencing factors. Select a suitable regression model, such as a linear regression model, a multivariate linear regression model, etc., to establish a regression equation between feeding time and feeding frequency and important influencing factors. Use the least squares method and other methods to estimate the parameters of the regression model and obtain the coefficients of the regression model. By calculating indicators such as the determination coefficient, test the goodness of fit of the regression model to determine whether the model can well describe the relationship between feeding time and feeding frequency and influencing factors. Perform a significance test on the coefficients of the regression equation to determine whether each influencing factor has a significant effect on feeding time and feeding frequency.

[0137] (2) Visible light camera.

[0138] Head posture change analysis: Analyze the changing patterns of cows’ head posture, such as the changes in head posture when eating different types of feed and in different environmental conditions. Changes in head posture can be used as an important basis for judging cows’ feeding interest and feeding status.

[0139] Specifically, the posture parameters such as the tilt angle and rotation angle of the head are calculated to describe the change of the head posture. The posture estimation algorithm can be used, such as the key point detection method, to analyze the detected cow head posture and calculate the posture parameters such as the tilt angle and rotation angle of the head. Among them, the key feature points of the cow's head are selected as eyes, ears, nose, etc., which are used to describe the characteristics of the head posture. The posture parameters such as the tilt angle and rotation angle of the cow's head are calculated by calculating the geometric relationship between the feature points, such as the vector angle, Euclidean distance, etc. Observe the change trend of the head posture parameters over time, and analyze the change law of the cow's head posture during the feeding process, such as whether the change of the head posture is related to the feeding action.

[0140] (3) Infrared camera.

[0141] Shape change analysis: Analyze the changes in feed pile shape parameters during the cow's feeding process, such as the change trend of the area and height of the feed pile over time, the change law of the shape, etc. The change of feed pile shape parameters can reflect the cow's feeding situation and preference for feed, and reveal the shape characteristics and change law of the feed pile.

[0142] Specifically, the change trend of the feed pile shape parameters during the cow feeding process is analyzed, such as the change of area, height, top of the feed pile and other parameters over time. The change trend of the shape parameters is observed to determine whether the feed pile is gradually decreasing or increasing, as well as the speed and magnitude of the change, and the periodic change law of the shape parameters is analyzed, such as whether there is a feeding cycle, a rest cycle, etc.

[0143] Based on this, in this embodiment, after obtaining the first feature, the second feature and the third feature, the method for predicting the feed intake of periparturient dairy cows in the pasture breeding scenario also includes the following steps.

[0144] (1) Counting the feeding time and feeding frequency in the first feature to obtain time series data of feeding time and time series data of feeding frequency; calculating a first statistical index of the time series data of feeding time and a second statistical index of the time series data of feeding frequency; determining the feeding pattern of periparturient dairy cows based on the time series data of feeding time, the time series data of feeding frequency, the first statistical index and the second statistical index, wherein the first statistical index and the second statistical index both include a change trend, a mean, a variance and a standard deviation, and the feeding pattern includes a peak feeding time, a trough feeding time and a change pattern of feeding time and feeding frequency.

[0145] (2) Calculate the posture parameters of the head posture in the second feature, and obtain statistical time series data of the posture parameters; determine the change pattern of the head posture based on the time series data of the posture parameters.

[0146] (3) Counting the feed pile shape parameters in the third feature to obtain time series data of the feed pile shape parameters; and determining the change trend and change law of the feed pile shape parameters based on the time series data of the feed pile shape parameters.

[0147] In the management of peripartum dairy cows, the health status and production performance of cows and calves are closely related to feeding behavior, and traditional monitoring methods are difficult to meet the needs of refined management due to their limitations. To this end, this embodiment proposes a multi-dimensional monitoring method closely centered on peripartum dairy cows and their calves, that is, a peripartum dairy cow feed intake prediction method in a large-scale pasture farming scenario. The method integrates smart collars, visible light cameras, and infrared cameras. The core is to comprehensively and accurately monitor and analyze the feeding behavior indicators of peripartum dairy cows and their impact on the health status and production performance of cows and calves through multi-dimensional information. As the front end of data collection, the smart collar can record key behavioral indicators such as feeding time, dry matter intake, rumination time, rest time, and activity of dairy cows in real time and accurately. These indicators not only directly reflect the nutritional status and digestive ability of dairy cows, but also are closely related to their postpartum health status, lactation performance, and the birth weight, daily weight gain and health status of calves. Visible light cameras enable real-time monitoring and visual analysis of cow behavior, and can capture the dynamic behavior of cows, including eating, ruminating, resting, and activity, making it possible to detect abnormal behavior in a timely manner. The addition of infrared cameras further enhances the monitoring capability of this method under low-light conditions. Infrared cameras can penetrate obstacles such as smoke and haze to produce clear images, providing reliable guarantees for monitoring the behavior of cows and calves at night or in bad weather conditions. The introduction of this technology makes this method more comprehensive and has no blind spots, and can better adapt to complex and changing breeding environments. In addition, this method also draws on advanced algorithms in the fields of multimodal video target tracking and infrared video behavior detection to improve the accuracy and efficiency of data processing. By analyzing massive amounts of video data through machine learning algorithms, it can automatically identify and track the behavioral trajectories of cows and calves, providing strong support for refined breeding management.

[0148] Large-scale ranch farming has complex production scenarios, multiple business processes, and common obstruction of targets. In view of this, this embodiment establishes a peripartum dairy cow feed intake prediction method in large-scale ranch farming scenarios based on computer vision technology, accurately identifies and analyzes the entire process of peripartum dairy cow feeding behavior, and integrates smart collars, visible light cameras, and infrared camera technologies to achieve comprehensive and accurate monitoring of dairy cow behavior and effective prediction of health status, providing strong support for improving the management level of peripartum dairy cow farming and ensuring the health and production performance of cows and calves.

[0149] Example 2.

[0150] The present embodiment provides a system for predicting feed intake of periparturient dairy cows in a pasture breeding scenario. The system for predicting feed intake of periparturient dairy cows in a pasture breeding scenario includes the following components.

[0151] The smart collar is worn on the neck of a periparturient dairy cow and is used to collect first monitoring data, which includes the feeding time, dry matter intake, rumination time, rest time, activity time and activity amount of the periparturient dairy cow.

[0152] The visible light camera is installed in the feeding area of ​​the dairy cows in the pasture, and is used to collect the second monitoring data. The second monitoring data is the visible light video obtained by shooting the feeding process of the periparturient dairy cows.

[0153] The infrared camera is installed in the feeding area of ​​the dairy cows in the pasture, and is used to collect the third monitoring data. The third monitoring data is the infrared video obtained by shooting the feeding process of the periparturient dairy cows.

[0154] Visible light cameras and infrared cameras are used to capture video data of cows in dairy barns, focusing on the cows’ head behavior and changes in feed piles in the feeding area.

[0155] The processor is respectively connected to the smart collar, the visible light camera and the infrared camera for executing the method for predicting the feed intake of periparturient dairy cows in the pasture breeding scenario described in Example 1.

[0156] This embodiment also provides a memory for storing computer programs and monitoring data.

[0157] The processor is connected to the memory and is used to retrieve and execute a computer program to implement the method for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario described in Example 1.

[0158] This embodiment can also perform a feeding behavior analysis report, as follows.

[0159] Report generation: Based on the analysis results of the model, a feeding behavior analysis report for peripartum dairy cows is generated. The report includes information such as the cows’ feeding time, feeding frequency, feeding amount, feeding behavior characteristics, as well as the cows’ health status assessment and recommendations.

[0160] Report push: The generated report will be pushed to the farmers to help them understand the feeding situation of dairy cows in a timely manner, adjust feeding management strategies, and improve the production performance and health level of dairy cows.

[0161] The system of this embodiment makes judgments based on data such as individual head movement posture and pile volume changes, and can provide personalized feeding assessments for each dairy cow. Farmers can take different management measures for different dairy cows based on these personalized assessment results, achieve precise farming decisions, and improve farming efficiency.

[0162] This embodiment constructs a periparturient dairy cow feeding behavior monitoring system that integrates multimodal data, and the system includes devices such as smart collars, infrared cameras, visible light cameras, data acquisition servers, and analysis terminals. Data acquisition software has been developed to collect smart collar data, video image data from infrared cameras, and visible light cameras in real time, and transmit the data to the data acquisition server. Data processing and analysis software has been developed, and the data processing and analysis software is run on the data acquisition server to clean, preprocess, and extract features from the collected data, and the extracted multimodal features are input into the feed intake prediction model for analysis and prediction. Result display software has been developed, and the result display software is run on the analysis terminal to display the prediction results of the feed intake prediction model to the user in the form of charts, images, etc., to help users intuitively understand the feeding behavior characteristics and feed intake of dairy cows.

[0163] Specifically, the real-time monitoring and evaluation process is as follows: the smart collar collects basic data such as the cow's feeding time and activity level in real time, and transmits it to the data collection server through wireless communication technology; the infrared camera and visible light camera take video images of the cow's feeding process at regular intervals, and transmit the video data to the data collection server.

[0164] Data processing and analysis are as follows: after receiving the data, the data acquisition server immediately cleans, preprocesses and extracts features of the data, and inputs the extracted multimodal features into the feed intake prediction model in real time for analysis and prediction.

[0165] The real-time result display is as follows: after the analysis terminal receives the prediction results of the feed intake prediction model, it immediately displays the results to the user in the form of charts, images, etc., including information such as the feed intake change trend and feeding behavior characteristics of the dairy cows.

[0166] The results and applications of feed intake assessment are as follows: Through the evaluation of real-time monitoring data, it is found that the model can accurately predict the feed intake of periparturient dairy cows in real time, and the prediction error is within an acceptable range, which provides an important basis for dairy farming managers to adjust feeding management strategies in a timely manner. According to the feed intake prediction results, farming managers can adjust the feed supply and feeding time of dairy cows in a timely manner to avoid feed waste and malnutrition of dairy cows. By analyzing the characteristics of dairy cows' feeding behavior, farming managers can promptly discover health problems of dairy cows, such as loss of appetite and digestion problems, and take corresponding measures for treatment and prevention to ensure the health and production performance of dairy cows.

[0167] This embodiment performs fusion analysis on the multimodal data collected over a long period of time, explores the potential rules and features in the data, and analyzes the feed intake data of different cows in different time periods, and finds that the feed intake of cows has certain seasonal and periodic changes. Therefore, this embodiment further designs model updates: according to the changes in long-term monitoring data, the feed intake prediction model is regularly updated and optimized to adapt to the changes in the feeding behavior of cows.

[0168] In this embodiment, the application and feedback of the feed intake prediction results are as follows.

[0169] Farming efficiency evaluation: Through long-term monitoring and prediction of the feed intake of dairy cows, the production efficiency of dairy cows can be calculated, including indicators such as milk production, milk quality, and feed cost. Based on the prediction results, farming managers can adjust farming strategies in a timely manner to improve farming efficiency.

[0170] Farming environment optimization: By analyzing the relationship between the feeding behavior of dairy cows and the farming environment, it is found that environmental factors have a certain impact on the feeding behavior and feed intake of dairy cows. According to the prediction results, farming managers can optimize the farming environment, provide more comfortable living and production conditions for dairy cows, and promote the healthy growth of dairy cows and improve their production performance.

[0171] Multimodal fusion can make full use of the multi-dimensional information of periparturient dairy cows’ feeding behavior and improve the accuracy and reliability of feed intake assessment. The fusion of multimodal data provides a more comprehensive and in-depth analysis method for dairy farming management, which helps to optimize feeding management strategies and ensure the health and production performance of dairy cows.

[0172] Example 3.

[0173] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario is implemented.

[0174] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0175] In an exemplary embodiment, a computer device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario as described in Example 1.

[0176] Example 4.

[0177] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario described in Example 1 is implemented.

[0178] Example 5.

[0179] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario described in Example 1.

[0180] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario, characterized in that: The method for predicting feed intake of periparturient dairy cows in the pasture breeding scenario includes: Obtaining first monitoring data collected by a smart collar, second monitoring data collected by a visible light camera, and third monitoring data collected by an infrared camera; the smart collar is worn on the neck of a periparturient dairy cow, and the first monitoring data includes the feeding time, dry matter intake, rumination time, rest time, activity time, and activity amount of the periparturient dairy cow; the visible light camera and the infrared camera are both installed in the dairy cow feeding area of ​​the pasture, the second monitoring data is a visible light video obtained by shooting the feeding process of the periparturient dairy cow, and the third monitoring data is an infrared video obtained by shooting the feeding process of the periparturient dairy cow; Feature extraction is performed on the first monitoring data, the second monitoring data and the third monitoring data respectively to obtain a first feature, a second feature and a third feature; the first feature includes feeding time, feeding frequency, rumination time and activity change value obtained by feature extraction of the first monitoring data, and the activity change value includes the mean and standard deviation of the activity; the second feature includes the head posture and feeding rate obtained by feature extraction of the second monitoring data; the third feature includes the feed pile shape parameters obtained by feature extraction of the third monitoring data; Performing feature fusion on the first feature, the second feature, and the third feature to obtain a fused feature; Taking the fused features as input, and using the trained feed intake prediction model to determine the feed intake of periparturient dairy cows; Before using the fused features as input and using the trained feed intake prediction model to determine the feed intake of periparturient dairy cows, the feed intake prediction method for periparturient dairy cows in the pasture breeding scenario further includes: Constructing a data set; the data set includes multiple samples and labels corresponding to each of the samples; the samples are features after sample fusion; the labels are sample feed intake; Construct multiple initial feed intake prediction models; For each of the initial feed intake prediction models, the initial feed intake prediction model is trained using the data set to obtain a trained feed intake prediction model; Combining all the trained feed intake prediction models into a trained feed intake prediction model; The fused features are used as input and the feed intake prediction model trained is used to determine the feed intake of periparturient dairy cows, specifically including: For each of the trained feed intake prediction models, the fused features are used as input, and the trained feed intake prediction model is used to determine the initial feed intake of the periparturient dairy cow; The feed intake of the periparturient dairy cows is obtained by taking a weighted sum of all the initial feed intakes.

2. The method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario according to claim 1, characterized in that: Feature extraction is performed on the first monitoring data, the second monitoring data, and the third monitoring data to obtain a first feature, a second feature, and a third feature, specifically including: Performing data cleaning on the first monitoring data, the second monitoring data and the third monitoring data respectively to obtain first cleaned data, second cleaned data and third cleaned data; the data cleaning includes removing outliers, removing noise and deblurring; Feature extraction is performed on the first cleaned data, the second cleaned data, and the third cleaned data respectively to obtain a first feature, a second feature, and a third feature.

3. The method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario according to claim 1, characterized in that: Feature extraction is performed on the first monitoring data, the second monitoring data, and the third monitoring data to obtain a first feature, a second feature, and a third feature, specifically including: Performing feature extraction on the first monitoring data to obtain a first feature; Taking the second monitoring data as input, using the trained head detection model to detect the head of the periparturient dairy cow to obtain head detection data; Analyze the head posture of the periparturient dairy cow based on the head detection data to obtain the head posture; the head posture includes frontal head lowering to feed, side torticollis to feed and head raising; Analyzing the feeding action of the peripartum dairy cow based on the head detection data to obtain the frequency and duration of the feeding action, and determining the feeding rate of the peripartum dairy cow based on the frequency and duration of the feeding action; the feeding action includes chewing action and swallowing action; The head posture and the feeding rate are combined to form a second feature obtained by extracting features from the second monitoring data; Taking the third monitoring data as input, detecting the feed pile using the trained feed pile detection model to obtain feed pile detection data; Extracting the outline of the feed pile in the feed pile detection data, and determining the feed pile shape parameters based on the outline of the feed pile; the feed pile shape parameters include the area, perimeter, diameter, curvature and shape of the feed pile; The feed pile shape parameter is used as the third feature obtained by extracting features from the third monitoring data.

4. The method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario according to claim 3, characterized in that: The second monitoring data is used as input, and the head of the peripartum dairy cow is detected using the trained head detection model to obtain head detection data, which specifically includes: Performing a first image processing on the second monitoring data to obtain first enhanced data; the first image processing includes graying, denoising, contrast enhancement and edge enhancement; Taking the first enhanced data as input, using the trained head detection model to detect the head of the periparturient dairy cow to obtain head detection data; The third monitoring data is used as input, and the feed pile is detected using the trained feed pile detection model to obtain feed pile detection data, which specifically includes: Performing second image processing on the third monitoring data to obtain second enhanced data; the second image processing includes grayscale conversion, contrast enhancement, brightness adjustment and binarization; The second enhanced data is used as input, and the trained feed pile detection model is used to detect the feed pile to obtain feed pile detection data.

5. The method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario according to claim 3, characterized in that: The feeding action of the peripartum dairy cow is analyzed based on the head detection data to obtain the frequency and duration of the feeding action, and the feeding rate of the peripartum dairy cow is determined based on the frequency and duration of the feeding action, specifically including: Performing motion analysis on the mouth movements of periparturient dairy cows based on the head detection data to obtain motion features; Classifying and identifying the action features, determining the feeding action, and recording the frequency and duration of the feeding action; The frequency and duration of feeding actions are used as input, and the feeding rate is determined using the trained feeding rate calculation model.

6. The method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario according to claim 1, characterized in that: The first feature, the second feature and the third feature are subjected to feature fusion to obtain a fused feature, specifically comprising: Performing feature selection on the first feature, the second feature, and the third feature to obtain selected features; All the selected features are weighted combined to obtain fused features.

7. The method for predicting feed intake of periparturient dairy cows in a pasture breeding scenario according to claim 1, characterized in that: After obtaining the first feature, the second feature and the third feature, the method for predicting feed intake of periparturient dairy cows in the pasture breeding scenario further includes: The feeding time and feeding frequency in the first feature are counted to obtain time series data of the feeding time and time series data of the feeding frequency; a first statistical index of the time series data of the feeding time and a second statistical index of the time series data of the feeding frequency are calculated; the feeding pattern of the periparturient dairy cow is determined based on the time series data of the feeding time, the time series data of the feeding frequency, the first statistical index and the second statistical index; the first statistical index and the second statistical index both include a change trend, a mean, a variance and a standard deviation; the feeding pattern includes a peak feeding time, a trough feeding time and a change rule of the feeding time and the feeding frequency; Calculating the posture parameters of the head posture in the second feature, and obtaining statistical time series data of the posture parameters; determining the change pattern of the head posture based on the time series data of the posture parameters; The feed pile shape parameters in the third feature are counted to obtain time series data of the feed pile shape parameters; and the change trend and change law of the feed pile shape parameters are determined based on the time series data of the feed pile shape parameters.

8. A feed intake prediction system for periparturient dairy cows in a pasture breeding scenario, characterized in that: The feed intake prediction system for periparturient dairy cows in the pasture breeding scenario includes: A smart collar is worn on the neck of a periparturient dairy cow to collect first monitoring data; the first monitoring data includes feeding time, dry matter intake, rumination time, rest time, activity time and activity amount of the periparturient dairy cow; A visible light camera is installed in the feeding area of ​​dairy cows in the pasture, and is used to collect second monitoring data; the second monitoring data is a visible light video obtained by shooting the feeding process of periparturient dairy cows; An infrared camera is installed in the feeding area of ​​dairy cows in the pasture, and is used to collect third monitoring data; the third monitoring data is an infrared video obtained by shooting the feeding process of periparturient dairy cows; A processor is respectively connected to the smart collar, the visible light camera and the infrared camera for executing the method for predicting the feed intake of periparturient dairy cows in a pasture breeding scenario as described in any one of claims 1 to 7.

Images