An AI-based real-time diagnostic system and method for peripartum dairy cow health status

Abstract

This invention provides an artificial intelligence-based real-time health status diagnosis system and method for peripartum dairy cows, belonging to the field of smart livestock technology. The method includes collecting image information and sensor data from the cowshed; preprocessing the collected image and sensor data; constructing image feature extraction models and sensor data feature extraction models; extracting features from the preprocessed image and sensor data based on these models; fusing the image and sensor data features to obtain fused features; constructing a health diagnosis model; and diagnosing the fused features based on the health diagnosis model to obtain the health status information of the dairy cows. This invention overcomes the limitations of single-modal diagnosis, establishes a comprehensive health profile of peripartum dairy cows, and provides reliable decision support for smart farms.

Description

Technical Field

[0001] This invention relates to the field of smart animal husbandry technology, and in particular to an artificial intelligence-based real-time diagnostic system and method for the health status of peripartum dairy cows. Background Technology

[0002] Current peripartum dairy cow health monitoring mainly relies on manual observation or single sensor technology, which has significant drawbacks: traditional manual inspections cannot achieve 24-hour continuous monitoring, and relying on experience makes it easy to miss early symptoms; wearable sensor-based solutions are difficult to capture visual features such as udder swelling and hoof lesions, and ear tag temperature sensors are easily affected by environmental interference; pure visual diagnostic models have a high false alarm rate in foggy weather, at night, or in scenarios where feces obscure the view, and cannot correlate with key behavioral indicators such as sudden drop in feed intake and reduced rumination.

[0003] Therefore, it is essential to design an artificial intelligence-based real-time diagnostic system and method for the health status of peripartum dairy cows. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an artificial intelligence-based real-time diagnostic system and method for the health status of peripartum dairy cows.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] This invention provides an artificial intelligence-based real-time diagnostic system for the health status of peripartum dairy cows, comprising:

[0007] The data acquisition module is used to collect image information and sensor data information in the cattle shed;

[0008] The data preprocessing module is used to preprocess the acquired image information and sensor data.

[0009] The feature extraction module is used to extract features from the preprocessed image information and sensor data information;

[0010] The feature fusion module is used to fuse image features and sensor data features.

[0011] The health diagnosis module is used to build a health diagnosis model and perform diagnosis on the fused features based on the health diagnosis model.

[0012] This invention also provides an artificial intelligence-based method for real-time diagnosis of peripartum dairy cow health status, applied to the aforementioned artificial intelligence-based real-time diagnosis system for peripartum dairy cow health status, comprising:

[0013] Step 1: Collect image information and sensor data from the cattle shed;

[0014] Step 2: Preprocess the acquired image information and sensor data;

[0015] Step 3: Construct an image information feature extraction model and a sensor data information feature extraction model. Based on the image information feature extraction model and the sensor data information feature extraction model, extract features from the preprocessed image information and sensor data information to obtain image features and sensor data features.

[0016] Step 4: Perform feature fusion on image features and sensor data features to obtain fused features;

[0017] Step 5: Construct a health diagnosis model, and diagnose the fused features based on the health diagnosis model to obtain the health status information of the dairy cows.

[0018] Preferably, in step 1, the acquisition of image information and sensor data information in the cattle shed specifically includes:

[0019] The cowshed uses individualized partitions, with each partition containing one dairy cow, one feed trough, and one water trough. Weight sensors are installed in the feed trough, water trough, and inside the partition to collect the weight information of the feed, water, and dairy cow, respectively. Based on the weight information of the feed and water, the weight of a single feeding and the amount of water consumed at a single feeding are calculated to obtain the dairy cow's feeding information.

[0020] Ear tag-type temperature sensors are installed in the cows' ears to collect their body temperature information in real time.

[0021] Temperature and humidity sensors, light intensity sensors, and ammonia concentration sensors were evenly installed inside the cattle shed to collect environmental information.

[0022] Multiple cameras are evenly placed inside the cowshed to collect image information of the dairy cows.

[0023] Preferably, in step 2, the acquired image information is preprocessed, specifically as follows:

[0024] Image information of dairy cows is acquired and subjected to dual-domain filtering for image denoising, decomposing the cow images into low-frequency and high-frequency images. Wavelet thresholds for different high-frequency images are obtained based on Bayesian estimation, and wavelet denoising is performed using an improved Garrote threshold function. Gamma transform is then used to correct the denoised high-frequency images, achieving filtering and contrast adjustment of the high-frequency images. Dark channel priors are used to dehaze the low-frequency images, and a contrast-limited adaptive histogram equalization algorithm is combined to enhance the overall dark low-frequency images after dehazing. Finally, the processed high-frequency and low-frequency images are reconstructed to obtain the final enhanced image.

[0025] Preferably, in step 2, the collected sensor data is preprocessed, specifically as follows:

[0026] The system acquires sensor data, including the cows' diet, body temperature, and the environment inside the barn. It improves the traditional VAE model using unsupervised learning, and then uses the improved VAE model to detect and repair abnormal data from the sensor data.

[0027] Preferably, the image information feature extraction model is constructed based on an improved YOLOv8-ResNet50. The improvement process of YOLOv8-ResNet50 involves improving the YOLOv8 part, specifically: replacing the CBS module of the BottleNeck module in the traditional YOLOv8 with the MS-Block module, replacing the BottleNeck module in the C2f module of the traditional YOLOv8 with the MSBottleNeck module, replacing the C2f module in the backbone network of the traditional YOLOv8 with the MSC2f module, replacing the CBS module in the Neck part of the traditional YOLOv8 with the GSConv module, replacing the Bottleneck module in the C2f part of the Neck part of the traditional YOLOv8 with the GSBottleneck module, and adding an EMA attention mechanism to the Head part of the traditional YOLOv8.

[0028] Preferably, the sensor data information feature extraction model is a CNN-LSTM model.

[0029] Preferably, the health diagnosis model is a multi-task Transformer model.

[0030] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0031] This invention provides an artificial intelligence-based real-time diagnostic system and method for the health status of peripartum dairy cows. The system includes a data acquisition module, a data preprocessing module, a feature extraction module, a feature fusion module, and a health diagnosis module. The method includes acquiring image information and sensor data information from the cowshed; preprocessing the acquired image information and sensor data information; constructing an image information feature extraction model and a sensor data information feature extraction model; extracting features from the preprocessed image information and sensor data information based on the image information feature extraction model and the sensor data information feature extraction model to obtain image features and sensor data features; fusing the image features and sensor data features to obtain fused features; constructing a health diagnosis model; and diagnosing the fused features based on the health diagnosis model to obtain the health status information of the dairy cows. This invention pioneers a "multimodal perception-dynamic fusion-edge diagnosis" system. First, it achieves precise data binding through individualized fence design, overcomes fog / low light interference with dual-domain filtering image enhancement technology, and improves the VAE model to repair sensor outliers, ensuring input quality. Second, it uses MS-Block to enhance YOLOv8's feature extraction capabilities, GSConv to optimize computational efficiency, and the EMA attention mechanism to accurately locate lesion areas. Combined with CNN-LSTM, it captures the cross-period correlation between sudden changes in body temperature and abnormal feed intake. Finally, the multi-task Transformer uses a cross-modal attention mechanism to link visual lesions with sensor time-series events, simultaneously outputting disease classification, health scores, and risk warnings. This invention can overcome the limitations of single-modal diagnosis, establish a comprehensive health profile of peripartum dairy cows, and provide reliable decision support for smart farms. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A schematic diagram of the process for real-time diagnosis of periparturient dairy cow health status based on artificial intelligence, provided in an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of the improved YOLOv8 network structure;

[0035] Figure 3 This is a schematic diagram of the CNN-LSTM network structure. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] The purpose of this invention is to provide an artificial intelligence-based real-time diagnostic system and method for the health status of peripartum dairy cows. This invention pioneers a "multimodal perception-dynamic fusion-edge diagnosis" system. First, it achieves precise data binding through individualized fence design, overcomes fog / low light interference with dual-domain filtering image enhancement technology, and improves the VAE model to repair sensor anomalies, ensuring input quality. Second, it uses MS-Block to enhance the feature extraction capabilities of YOLOv8, GSConv to optimize computational efficiency, and the EMA attention mechanism to accurately locate lesion areas. It also uses CNN-LSTM to capture the cross-period correlation between sudden changes in body temperature and abnormal feed intake. Finally, the multi-task Transformer associates visual lesions with sensor time-series events through a cross-modal attention mechanism, simultaneously outputting disease classification, health scores, and risk warnings. This invention can overcome the limitations of single-modal diagnosis, establish a comprehensive health profile of peripartum dairy cows, and provide reliable decision support for smart farms.

[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] This invention provides an artificial intelligence-based real-time diagnostic system for the health status of peripartum dairy cows, comprising:

[0040] The data acquisition module is used to collect image information and sensor data information in the cattle shed;

[0041] The data preprocessing module is used to preprocess the acquired image information and sensor data.

[0042] The feature extraction module is used to extract features from the preprocessed image information and sensor data information;

[0043] The feature fusion module is used to fuse image features and sensor data features.

[0044] The health diagnosis module is used to build a health diagnosis model and perform diagnosis on the fused features based on the health diagnosis model.

[0045] like Figure 1As shown, the present invention also provides an artificial intelligence-based method for real-time diagnosis of peripartum dairy cow health status, applied to the aforementioned artificial intelligence-based real-time diagnosis system for peripartum dairy cow health status, comprising:

[0046] Step 1: Collect image information and sensor data from the cattle shed;

[0047] Step 2: Preprocess the acquired image information and sensor data;

[0048] Step 3: Construct an image information feature extraction model and a sensor data information feature extraction model. Based on the image information feature extraction model and the sensor data information feature extraction model, extract features from the preprocessed image information and sensor data information to obtain image features and sensor data features.

[0049] Step 4: Perform feature fusion on image features and sensor data features to obtain fused features;

[0050] Step 5: Construct a health diagnosis model, and diagnose the fused features based on the health diagnosis model to obtain the health status information of the dairy cows.

[0051] In step 1, image information and sensor data information from the cattle shed are collected, specifically as follows:

[0052] The cowshed uses individualized partitions, with each partition containing one dairy cow, one feed trough, and one water trough;

[0053] Among them, a high-precision strain gauge weighing sensor is embedded under the ground of the enclosure and covers the core area where the cows stand to collect the weight information of the cows. This application provides an embodiment in which the sensor is collected once every 10 minutes.

[0054] A cantilever beam weighing sensor is installed at the bottom of the feed tank to collect the weight information of the feed. Other conventional components can also be used to collect the start time and duration of each feeding, depending on specific needs. Based on the weight information, the weight of a single feeding and the total amount of feed can also be calculated.

[0055] The water tank can be equipped with a dual-redundant design of electromagnetic flow meter and weighing sensor, which can collect the amount of water drunk at one time, drinking frequency and average daily drinking volume.

[0056] Ear tag temperature sensors are installed in the cow's ears to collect the cow's body temperature information in real time. Depending on the specific needs, the ear tag temperature sensor can be replaced with an ear tag multi-functional sensor, which integrates temperature, triaxial accelerometer and microphone, and can collect the cow's temperature, activity level and rumination sounds.

[0057] Temperature and humidity sensors, light intensity sensors, and ammonia concentration sensors were evenly installed inside the cattle shed to collect environmental information.

[0058] Multiple cameras are evenly installed inside the cowshed to collect image information of the dairy cows;

[0059] Depending on specific needs, each dairy cow can be labeled with an ear tag RFID, and individualized pens can also be labeled, with each number corresponding to a dairy cow's label.

[0060] In step 2, the acquired image information is preprocessed, specifically as follows:

[0061] Image information of dairy cows is acquired and subjected to dual-domain filtering for image denoising, decomposing the cow images into low-frequency and high-frequency images. Wavelet thresholds for different high-frequency images are obtained based on Bayesian estimation, and wavelet denoising is performed using an improved Garrote thresholding function. Gamma transform is then used to correct the denoised high-frequency images, achieving filtering and contrast adjustment of the high-frequency images. Dark channel priors are used to dehaze the low-frequency images, and a contrast-limited adaptive histogram equalization algorithm is combined to enhance the overall darker low-frequency images after dehazing. The processed high-frequency and low-frequency images are then reconstructed to obtain the final enhanced image. This process is described in detail below:

[0062] First, the dual-domain filtering model is introduced. Dual-domain filtering image denoising (DDID) is a non-linear two-dimensional signal filtering method that combines the spatial proximity of pixels with the similarity of gray values ​​between pixels. Compared to Gaussian filters, DDID uses dual-domain filters and short-time Fourier transforms to perform multi-scale noise reduction on the image. Therefore, this invention selects a dual-domain filter to perform single-scale decomposition on the cow image. In the dual-domain filter, the pixel values ​​of the output image depend on the weighted combination of neighboring pixel values, defined as:

[0063]

[0064] In the formula, f(k, l) represents the original noisy image f; w(i, j, k, l) represents the weighting coefficients; f L (i, j) represents the output low-frequency image f L ;

[0065] The weighting coefficients w(i, j, k, l) depend on the spatial domain kernel and the pixel value domain kernel. The spatial domain kernel is defined as follows:

[0066]

[0067] In the formula, d(i,j,k,l) ​​represents the Gaussian weight based on spatial distance, (i,j) represents the coordinates of the neighboring pixels, (k,l) represents the coordinates of the center pixel, and σ represents the spatial domain variance.

[0068] The pixel value range kernel is defined as:

[0069]

[0070] In the formula, r(i, j, k, l) represents the Gaussian weights based on the similarity between pixels; σ 2 r Indicates the range variance;

[0071] The weighting coefficient w(i, j, k, l) is the product of the spatial domain kernel and the pixel value domain kernel, and its expression is:

[0072]

[0073] Since images still contain a large amount of noise after being filtered by the DDID algorithm, it is necessary to further refine the high-frequency image f obtained after DDID filtering. H Further noise reduction should be performed before enhancement; otherwise, while enhancing image details, noise will be amplified. H The low-frequency image f decomposed from the original noisy image f and the image f decomposed by the dual-domain filter L Differential is obtained because of the high-frequency image f H It is difficult to achieve f using traditional spatial domain filtering or value domain filtering methods. H To address noise removal in high-frequency frequencies, this invention employs an improved Garrote wavelet threshold function to construct a high-frequency denoising model, which is described below.

[0074] The improved two typical wavelet thresholding (WTD) models are defined as follows:

[0075] 1. Semisoft Denoising Model

[0076]

[0077] In the formula, μ T ω represents the high-frequency coefficients after wavelet denoising. i，j Let sgn(ω) represent the j-th high-frequency coefficient under the i-th level wavelet decomposition. i，j T1 and T2 represent the two thresholds of the wavelet threshold function.

[0078] 2. Garrote Denoising Model

[0079]

[0080] In the formula, μ T ω represents the high-frequency coefficients after wavelet denoising. i，j Let represent the j-th high-frequency coefficient under the i-th level wavelet decomposition, and T represent the wavelet threshold;

[0081] Semisoft denoising model can combine the advantages of soft and hard thresholding functions, but it requires the calculation of two thresholds, which has disadvantages such as large computational load and difficulty in algorithm implementation. Garrote denoising model can maintain image smoothness well and can preserve the edge feature information of the image to a certain extent. Its disadvantage is that the wavelet threshold T cannot adaptively adjust the threshold as the number of wavelet decomposition layers increases, which leads to blurring of the enhanced image.

[0082] In image acquisition, the two typical denoising models mentioned above exhibit varying degrees of drawbacks after image enhancement. This invention introduces an improved Garrote wavelet thresholding denoising model. This model is continuous throughout its domain, avoids fixed biases, is computationally simple, and can be applied to image denoising under different noise environments. Its expression is as follows:

[0083]

[0084] In the formula, s represents the adaptive weighting factor, s∈(0,1);

[0085] In this model, the adaptive weight factor s can be adaptively adjusted according to the noise coefficient distribution after wavelet decomposition, greatly improving the flexibility and practicality of the model. s is:

[0086] s=m / M (8)

[0087] In the formula, M represents the length of the wavelet high-frequency coefficients, and m represents the number of frequencies in the wavelet high-frequency coefficients that are greater than the threshold.

[0088] Since the noise coefficients of the acquired images vary, if the same threshold is used for all images to be processed, an excessively large threshold will cause effective wavelet coefficients below the threshold to be set to 0, resulting in blurred image details. Conversely, an excessively small threshold will leave a large amount of residual noise signal in the wavelet denoising process, reducing the denoising effect of the fH wavelet denoising algorithm for high-frequency images. Therefore, this invention employs a Bayesian estimation wavelet shrinkage threshold method to adaptively adjust the wavelet threshold. The specific calculation process is as follows:

[0089] 1. According to Bayesian estimation theory, after filtering using the DDID algorithm, the high-frequency image f... H It follows a mean of 0 and a variance of σ. x 2 The generalized Gaussian distribution is:

[0090]

[0091] 2. For a given parameter σ x Then, based on the Bayesian risk estimation function r(T), the optimal threshold T is found, and its calculation expression is:

[0092]

[0093] In the formula, σ 2 Represents the high-frequency image f H The noise variance; σ x Represents the high-frequency image f H Standard deviation;

[0094] 3. Noise variance σ 2 The Lubang median estimate proposed by Donoho is as follows:

[0095] σ=median(|ω i，j |) / 0.6745 (11)

[0096] In the formula, ω i，j Represents the high-frequency image f H The coefficient values ​​of the coordinates (i, j);

[0097] 4. The variance estimate for each noisy observation sub-band is obtained using the maximum likelihood estimation (ML) method, as follows:

[0098]

[0099] In the formula, n represents the high-frequency image f H Total number of pixels;

[0100] 5. Finally, σ can be obtained. x The calculation formula is:

[0101]

[0102] The above formula can be used to calculate the adaptive wavelet threshold T at different wavelet scales and overcome the shortcomings of fixed wavelet thresholds.

[0103] This invention employs a dehazing model based on dark channel priors for low-frequency images f L The image is processed and enhanced by combining a contrast-limited adaptive histogram equalization algorithm to improve the overall dark low-frequency image after dehazing. The processed high-frequency and low-frequency images are then reconstructed to obtain the final enhanced image. This part is not improved and can be achieved using existing technology, so it will not be described in detail.

[0104] In step 2, the collected sensor data is preprocessed, specifically as follows:

[0105] Acquire sensor data, including dairy cows' diet, body temperature, and the environment inside the barn. Improve the traditional VAE model based on unsupervised learning. Detect abnormal data from the sensor data using the improved VAE model and repair the detected abnormal data.

[0106] The improved VAE model is introduced. In anomaly detection, a threshold is often needed to determine the presence of abnormal data. The original VAE model is an image generation model, so a threshold needs to be determined when performing anomaly detection. However, a fixed threshold is prone to false positives and false negatives during the detection process. Anomalies are detected by comparing the error between the reconstructed data and the original data with the set threshold. If the threshold is set manually, false positives and false negatives are likely to occur. Therefore, a dynamic threshold method is introduced to improve the detection accuracy and precision of the model.

[0107] The dynamic threshold method can adjust a suitable threshold according to the actual situation of reconstruction error and the changes in time data to complete the anomaly detection function. This means that it can adapt to different data patterns, including seasonal, trend and sudden changes. It usually uses the statistical information of the data, such as mean and standard deviation, to calculate the anomaly threshold. The threshold can be updated in real time according to new data, which makes the threshold more representative and better able to capture outliers.

[0108] After processing the error between the reconstructed data and the original data, the threshold is determined, which uses the principles of mean and standard deviation, which will not be described here.

[0109] The process of anomaly detection and judgment is described below:

[0110] 1. Error calculation: First, for the input time series data X, the VAE model will generate the corresponding reconstructed sequence X'. Based on the differences between the samples in the two sequences and the average reconstruction error of each sample, the average reconstruction error of the entire batch is calculated, and the reconstruction error of the subsequence is calculated.

[0111] 2. Anomaly detection: Based on the reconstruction error obtained in the previous step, analyze and compare it with the obtained threshold to determine whether there is abnormal data in the input data.

[0112] If abnormal data is detected, it will be repaired, including removal, filling, etc. The choice can be made according to the specific needs, and there are no restrictions here.

[0113] The image information feature extraction model is built based on an improved YOLOv8-ResNet50; the improvement process of YOLOv8-ResNet50 involves improving the YOLOv8 part, specifically:

[0114] The YOLOv8 architecture will not be described here; instead, its improvements will be specifically introduced. The BottleNeck module of YOLOv8 consists of residual connections and two CBS modules cascaded together. The residual connections are used to alleviate the gradient vanishing problem, and the two cascaded CBS modules are used for feature extraction. Due to the fixed 3×3 convolutional kernel size and structure of CBS, its feature fusion capability is insufficient and it cannot effectively extract feature information at different scales. The MS-Block module, due to its hierarchical feature fusion and the use of different convolutional kernel sizes at different stages, effectively improves the multi-scale feature fusion capability of the model. Since MS-Block has good transferability, it can be directly used to replace the CBS module with a convolutional kernel size of 3×3.

[0115] The main components of the YOLOv8 backbone network include the CBS module and the C2f module composed of Bottleneck. This invention replaces the Bottleneck module with the MSBottleNeck module while maintaining the overall structure of C2f, forming the MSC2f module. By using the MSC2f module, the multi-scale feature extraction capability is increased, and the fusion of multi-scale feature information is realized. This reduces the loss of information during feature extraction and enables the network to pay more attention to and capture features more comprehensively.

[0116] The YOLOv8 backbone network consists of alternating CBS and C2f modules. The main function of the C2f module is to extract features from the input data. There are four C2f modules in the YOLOv8 backbone network. These modules are located in different positions and process feature maps of different scales. The MSC2f module is used to replace these four C2f modules to enhance the network's ability to process multi-scale information.

[0117] The initial input size of the YOLOv8 backbone network is 640×640×3, representing an image with 3 channels and a size of 640×640. As the network deepens, the input image is transformed into feature maps of gradually smaller sizes. In the shallow layers of the network, the output feature map is larger in size and has a smaller receptive field. The smaller receptive field helps the model extract local features and is suitable for detecting small targets. In the deep layers of the network, the output feature map is smaller in size and has a larger receptive field. The larger receptive field allows the model to focus more on the global features of the image and is helpful for detecting larger targets. In layers 4, 6, and 8, the size of the feature map gradually decreases from 160×160 to 80×80, 40×40, and 20×20, respectively. This invention introduces MS-Block in layers 5, 7, and 9. MS-Block selects different sizes of convolutional kernels according to the feature map of different sizes, which improves the detection accuracy of the model, increases the model's ability to extract features at multiple scales, and thus improves the overall performance of the model.

[0118] GSConv is mainly used for the Neck part of feature enhancement and feature fusion. The Neck part of YOLOv8 is mainly composed of CBS module and C2f module. In order to reduce the number of convolution parameters, the CBS module is replaced by GSConv module. By stacking GSConv modules, the network's ability to process features is improved, forming GSBottleneck module. Then, this module is used to replace the Bottleneck module in C2f, forming GSC2f module.

[0119] By using the GSC2f module, the number of network model parameters and computational cost are reduced without significantly affecting the network detection performance, enabling the improved network to meet the requirements of lightweight design.

[0120] To improve the detection capability of the model without increasing too many parameters, this invention introduces the EMA attention mechanism. The EMA attention mechanism retains accurate spatial location information in the channels while controlling the weight of each channel by encoding the information between channels, thereby highlighting important channels and enabling the model to focus on the more important parts of the features, thus improving the overall detection capability of the model.

[0121] This invention adds an EMA attention mechanism to the Head part of the network to further process the output feature map of the Neck part. The EMA attention mechanism can highlight important features before final classification and reduce the importance of irrelevant features, enabling the model to effectively acquire important information about the target to be detected, thereby improving the detection accuracy and overall performance of the model.

[0122] After a series of improvements, the improved YOLOv8 was finally formed, such as... Figure 2 As shown, the introduction of lightweight modules MS-Block and GSConv significantly reduces the network parameters. Furthermore, MS-Block's multi-scale feature extraction capability improves the model's ability to detect targets at different scales. The introduction of the EMA attention mechanism enhances the model's cross-spatial learning ability. The combination of these two mechanisms allows the model to acquire global information when facing features at different scales, thereby improving the overall performance of the model.

[0123] This invention does not limit the improvement of ResNet50; improvements can be made according to specific needs.

[0124] The sensor data information feature extraction model is a CNN-LSTM model, which is described below:

[0125] The CNN and LSTM network structures will not be described here; their structural diagrams are shown below. Figure 3 As shown, its parameters are shown in Table 1;

[0126] Table 1 CNN-LSTM Model Parameters

[0127]

[0128] Step 4: Perform feature fusion on image features and sensor data features to obtain fused features, specifically as follows:

[0129] The cross-modal Transformer model is used to fuse image features and sensor data features to obtain fused features. Since the cross-modal Transformer model is an existing model, it will not be described in detail here.

[0130] In step 5, a health diagnosis model is constructed, and the fused features are diagnosed based on the health diagnosis model to obtain the health status information of the dairy cows, specifically:

[0131] The health diagnosis model is a multi-task Transformer model, which adopts the BERT multi-task head extension architecture with Hugging Face, and has three tasks, namely...

[0132] The disease classification task has an output layer structure of fully connected layer + Softmax, and its output is a 4-dimensional probability vector: [mastitis, ketosis, laminitis, endometritis];

[0133] The health scoring task has an output layer structure of a fully connected layer + Sigmoid, and its output is a continuous value from 0 to 100.

[0134] The risk warning task has an output layer structure of a fully connected layer + Sigmoid, and its output is a 2-dimensional probability: [risk of dystocia, risk of postpartum paralysis].

[0135] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0136] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An artificial intelligence-based real-time diagnostic system for the health status of peripartum dairy cows, characterized in that, include: The data acquisition module is used to collect image information and sensor data information in the cattle shed; The data preprocessing module is used to preprocess the acquired image information and sensor data. The feature extraction module is used to extract features from the preprocessed image information and sensor data information; The feature fusion module is used to fuse image features and sensor data features. The health diagnosis module is used to build a health diagnosis model and perform diagnosis on the fused features based on the health diagnosis model.

2. A method for real-time diagnosis of the health status of peripartum dairy cows based on artificial intelligence, characterized in that, The system applied to the artificial intelligence-based real-time diagnostic system for peripartum dairy cow health status as described in claim 1 includes: Step 1: Collect image information and sensor data from the cattle shed; Step 2: Preprocess the acquired image information and sensor data; Step 3: Construct an image information feature extraction model and a sensor data information feature extraction model. Based on the image information feature extraction model and the sensor data information feature extraction model, extract features from the preprocessed image information and sensor data information to obtain image features and sensor data features. Step 4: Perform feature fusion on image features and sensor data features to obtain fused features; Step 5: Construct a health diagnosis model, and diagnose the fused features based on the health diagnosis model to obtain the health status information of the dairy cows.

3. The method according to claim 2, characterized in that, In step 1, image information and sensor data information from the cattle shed are collected, specifically as follows: The cowshed uses individualized partitions, with each partition containing one dairy cow, one feed trough, and one water trough. Weight sensors are installed in the feed trough, water trough, and inside the partition to collect the weight information of the feed, water, and dairy cow, respectively. Based on the weight information of the feed and water, the weight of a single feeding and the amount of water consumed at a single feeding are calculated to obtain the dairy cow's feeding information. Ear tag-type temperature sensors are installed in the cows' ears to collect their body temperature information in real time. Temperature and humidity sensors, light intensity sensors, and ammonia concentration sensors were evenly installed inside the cattle shed to collect environmental information. Multiple cameras are evenly placed inside the cowshed to collect image information of the dairy cows.

4. The method according to claim 3, characterized in that, In step 2, the acquired image information is preprocessed, specifically as follows: Image information of dairy cows is acquired and subjected to dual-domain filtering for image denoising, decomposing the cow images into low-frequency and high-frequency images. Wavelet thresholds for different high-frequency images are obtained based on Bayesian estimation, and wavelet denoising is performed using an improved Garrote threshold function. Gamma transform is then used to correct the denoised high-frequency images, achieving filtering and contrast adjustment of the high-frequency images. Dark channel priors are used to dehaze the low-frequency images, and a contrast-limited adaptive histogram equalization algorithm is combined to enhance the overall dark low-frequency images after dehazing. Finally, the processed high-frequency and low-frequency images are reconstructed to obtain the final enhanced image.

5. The method according to claim 4, characterized in that, In step 2, the collected sensor data is preprocessed, specifically as follows: The system acquires sensor data, including the cows' diet, body temperature, and the environment inside the barn. It improves the traditional VAE model using unsupervised learning, and then uses the improved VAE model to detect and repair abnormal data from the sensor data.

6. The method according to claim 5, characterized in that, The image information feature extraction model is built based on an improved YOLOv8-ResNet50. The improvement process of YOLOv8-ResNet50 involves improving the YOLOv8 part, specifically: replacing the CBS module of the BottleNeck module of the traditional YOLOv8 with the MS-Block module, replacing the BottleNeck module in the C2f module of the traditional YOLOv8 with the MSBottleNeck module, replacing the C2f module in the backbone network of the traditional YOLOv8 with the MSC2f module, replacing the CBS module in the Neck part of the traditional YOLOv8 with the GSConv module, replacing the Bottleneck module in the C2f part of the Neck part of the traditional YOLOv8 with the GSBottleneck module, and adding an EMA attention mechanism to the Head part of the traditional YOLOv8.

7. The method according to claim 6, characterized in that, The sensor data information feature extraction model is a CNN-LSTM model.

8. The method according to claim 7, characterized in that, The health diagnosis model is a multi-task Transformer model.

Images