System for analyzing health status of cattle herd by fusing biosensing and pattern visual recognition

Abstract

This invention discloses a cattle herd health status analysis system integrating biosensing and pattern visual recognition, belonging to the field of intelligent livestock monitoring technology. It includes a data acquisition module that simultaneously acquires time-series data of cattle physiological parameters and sequences of cattle shed behavior images; an anomaly analysis module that uses an adaptive threshold improved sliding window algorithm with dynamically adjusted window length to generate a sequence of marked physiological anomaly periods; a feature extraction module that extracts key points of image posture to form a sequence of behavioral posture feature vectors; and a cross-modal fusion module that aligns and fuses the two types of sequences according to timestamps to construct a cross-modal health status representation tensor, which is then input into a pre-trained hierarchical network to output the health status level of individual cattle. This scheme achieves cross-modal integration of physiological sensing and visual recognition, adapts to the dynamic fluctuation patterns of cattle physiological data, enriches health feature representation information, and can accurately distinguish individual health differences within a cattle herd.

Description

Technical Field

[0001] This invention belongs to the field of intelligent livestock monitoring technology, specifically a cattle herd health status analysis system that integrates biosensing and pattern visual recognition. Background Technology

[0002] In the livestock farming industry, cattle health monitoring generally employs a single monitoring method, relying on wearable sensors to collect time-series data of cattle physiological parameters, or on cameras to capture images of cattle behavior in the barn. Screening for abnormal physiological data typically uses conventional sliding window detection algorithms with fixed window lengths and thresholds. Health status determination relies solely on single-dimensional data for basic analysis, failing to achieve the integrated application of biosensor data and visual behavioral data.

[0003] Conventional detection modes using fixed windows and thresholds cannot adapt to the dynamic fluctuations in physiological parameters of cattle at different growth stages and activity levels, leading to biased or missed detections in time-series data anomaly identification. The single-dimensional data analysis model severs the intrinsic connection between physiological changes and behavioral posture changes in cattle; the two types of data cannot establish a temporal correlation, resulting in a limited dimension of health status analysis and an inability to fully reconstruct the true health status of cattle. It is necessary to optimize the anomaly detection logic to adapt to the dynamic changes in physiological time-series data, achieving accurate temporal matching and fusion of physiological time-series data and behavioral image features, thus enriching the representational dimensions of cattle health status. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; Therefore, this invention proposes a bovine herd health status analysis system that integrates biosensing and pattern visual recognition, comprising: The data acquisition module acquires the time-series data set of physiological parameters collected by the biosensor devices worn by each experimental cow in the target herd, as well as the sequence of cattle behavior images synchronously captured by the visual acquisition equipment fixed in the cowshed. The anomaly analysis module executes an improved sliding window anomaly detection algorithm on the physiological parameter time series data set to generate a physiological anomaly period marker sequence for each experimental cow. The improved sliding window anomaly detection algorithm is based on an adaptive threshold mechanism that dynamically adjusts the window length. The feature extraction module performs posture key point extraction on the cattle behavior image sequence to generate a behavior posture feature vector sequence for each experimental cattle. The cross-modal fusion module aligns and fuses the physiological abnormality period marker sequence with the behavioral posture feature vector sequence according to the timestamp to generate a cross-modal health status representation tensor for each experimental cow. The health status classification module inputs the cross-modal health status representation tensor into a pre-trained health status classification network and outputs the individual health status level of each experimental cow.

[0005] Furthermore, the step of performing an improved sliding window anomaly detection algorithm on the time-series physiological parameter data set to generate a sequence of physiologically abnormal time periods for each experimental cow specifically includes: The physiological parameter time series data set is grouped according to the individual identifier of each experimental cow to obtain the individual physiological parameter time series data corresponding to each experimental cow. The individual physiological parameter time series data includes body temperature sampling sequence, heart rate sampling sequence and rumination sound intensity sampling sequence. An improved sliding window anomaly detection algorithm is executed on the body temperature sampling sequence, the heart rate sampling sequence, and the rumination sound intensity sampling sequence, respectively. The working process of the improved sliding window anomaly detection algorithm is as follows: Initialize a sliding window of length L, place the sliding window at the beginning of the body temperature sampling sequence, and calculate the arithmetic mean and standard deviation of all sampling points within the sliding window; The value of each sampling point within the sliding window is compared with the arithmetic mean. When the absolute difference between the value of a sampling point and the arithmetic mean is greater than three times the standard deviation, the sampling point is marked as a candidate outlier. The number of candidate anomalies within the sliding window is counted. When the number of candidate anomalies exceeds a preset anomaly density threshold, the current position of the sliding window is marked as an anomaly window, and the length L of the sliding window is increased by one step unit to obtain the updated window length. When the number of candidate anomalies is less than or equal to the preset anomaly density threshold, the length L of the sliding window is reduced by one step unit to obtain the updated window length, and the sliding window is moved forward by one step unit. Repeat the operations of calculating the arithmetic mean and standard deviation, marking candidate outliers, judging the outlier window, and adjusting the length of the sliding window until the sliding window has traversed the entire body temperature sampling sequence; The time intervals covered by all abnormal windows in the body temperature sampling sequence are merged to obtain the body temperature abnormal time period marker sequence corresponding to the body temperature sampling sequence. The heart rate sampling sequence and the rumination sound intensity sampling sequence are processed according to the same working process to obtain the heart rate abnormal period marker sequence and the rumination abnormal period marker sequence; The abnormal body temperature time period marker sequence, the abnormal heart rate time period marker sequence, and the abnormal rumination time period marker sequence are time-series merged, and the union of the three sequences is taken as the physiological abnormal time period marker sequence for each experimental cow.

[0006] Further, the step of performing pose key point extraction on the cattle behavior image sequence to generate a behavioral pose feature vector sequence for each experimental cattle specifically includes: The local regions containing individual experimental cattle are extracted frame by frame from the sequence of images of cattle behavior, and image normalization processing is performed on the local regions to generate a normalized sequence of individual cattle images. The pre-trained posture keypoint detection network is invoked to perform keypoint regression processing on each frame of the normalized individual cattle image sequence, and outputs the pixel coordinates of multiple trunk keypoints of each experimental cattle in each frame image. The multiple trunk keypoints include the nose tip keypoint, the left ear root keypoint, the right ear root keypoint, the neck center keypoint, the scapula keypoint, the midpoint of the back midline keypoint, the tail root keypoint, the left forehoof keypoint, the right forehoof keypoint, the left hindhoof keypoint, and the right hindhoof keypoint. The posture description parameters of each experimental cow are calculated based on the pixel coordinates of multiple trunk key points in each frame of the image. The posture description parameters include the angle between the line connecting the nose tip key point and the center neck key point and the horizontal line, the angle between the line connecting the center neck key point and the midpoint of the back key point and the horizontal line, the length of the line connecting the scapula key point and the tail root key point, the horizontal distance between the left foreleg key point and the right foreleg key point, the horizontal distance between the left hind leg key point and the right hind leg key point, and the height of the midpoint of the back key point from the ground reference line. All pose description parameters calculated in the same frame are concatenated into a one-dimensional vector in a preset order, and this one-dimensional vector is used as the single-frame behavior pose feature vector corresponding to the frame. Arrange the single-frame behavioral posture feature vectors corresponding to all frame images in chronological order to generate a behavioral posture feature vector sequence for each experimental cow.

[0007] Furthermore, the pose key point detection network is based on an HRNet network model finely tuned on a cattle pose image dataset.

[0008] Further, the step of aligning and fusing the physiological abnormality period marker sequence with the behavioral posture feature vector sequence according to timestamps to generate a cross-modal health status representation tensor for each experimental cow specifically includes: Obtain the start and end times of each abnormal time period in the physiological abnormal time period marker sequence, and the timestamp corresponding to each single frame behavioral posture feature vector in the behavioral posture feature vector sequence; Extract all single-frame behavioral posture feature vectors from the behavioral posture feature vector sequence whose timestamps fall within any abnormal time period of the physiological abnormal time period marker sequence, and form an abnormal time period behavioral feature subsequence. Extract all single-frame behavioral posture feature vectors from the behavioral posture feature vector sequence whose timestamps do not fall within any abnormal time periods to form a normal time period behavioral feature subsequence. A dimension-wise averaging operation is performed on all single-frame behavioral pose feature vectors in the normal time period behavioral feature subsequence to obtain the normal behavior baseline feature vector; Perform a dimension-wise difference calculation between each single-frame behavior posture feature vector in the abnormal period behavior feature subsequence and the normal behavior baseline feature vector to obtain the deviation feature vector corresponding to each single-frame behavior posture feature vector. Each deviation feature vector is bound to the time period identifier information of its corresponding abnormal time period, and the binding results are stacked in chronological order to generate a cross-modal health status representation tensor for each experimental cow. The first dimension of the cross-modal health status representation tensor corresponds to the chronological order of the abnormal time period, and the second dimension corresponds to each dimension component of the deviation feature vector.

[0009] Furthermore, the time period identification information is the sequential number of the corresponding abnormal time period in the physiological abnormal time period marking sequence.

[0010] Further, the step of inputting the cross-modal health state representation tensor into a pre-trained health state classification network and outputting the individual health state level of each experimental cow specifically includes: The cross-modal health state representation tensor is flattened along the first dimension to generate a one-dimensional health state input feature vector. The one-dimensional health status input feature vector is input into the input layer of the pre-trained health status classification network, which is a multi-layer classification network composed of multiple fully connected layers stacked together. In the first fully connected layer of the pre-trained health state classification network, a linear transformation operation is performed on the one-dimensional health state input feature vector to generate a first intermediate feature vector. In the second fully connected layer of the pre-trained health status classification network, a linear transformation operation is performed on the first intermediate feature vector and then processed by an activation function to generate a second intermediate feature vector; The output of the pre-trained health status classification network is processed sequentially through the remaining fully connected layers, and linear transformation and activation function processing are performed on the output of the previous layer until the result vector of the output layer of the pre-trained health status classification network is obtained. The output vector of the output layer is subjected to a normalized exponential function transformation to obtain the probability distribution of each experimental cow belonging to each preset health status level. The health status level corresponding to the maximum probability is taken as the individual health status level of each experimental cow.

[0011] Furthermore, the pre-trained health status classification network is trained using a set of labeled cross-modal health status representation tensors, the labels of which are provided by veterinary experts based on clinical diagnostic results.

[0012] Furthermore, the improved sliding window anomaly detection algorithm is based on an adaptive threshold mechanism that dynamically adjusts the window length, specifically including: Set the initial length L (zero value), minimum length L (minimum value), and maximum length L (maximum value) of the sliding window, and set the initial value D (zero value) of the abnormal density threshold; During the process of the sliding window traversing the body temperature sampling sequence, each time the sliding window moves once, the current window density value is calculated based on the number of sampling points currently covered by the sliding window. The current window density value is the ratio of the number of candidate abnormal points in the sliding window to the current length of the sliding window. The current window density value is compared with the current value of the abnormal density threshold. When the current window density value is greater than the current value of the abnormal density threshold, the length of the sliding window is updated to the current length plus one step unit, but the updated length does not exceed the maximum value of the maximum length L. When the current window density value is less than or equal to the current value of the abnormal density threshold, the length of the sliding window is updated to the current length minus one step unit, but the updated length is not less than the minimum value of the minimum length L. After each update of the length of the sliding window, the current value of the abnormal density threshold is dynamically adjusted according to the updated sliding window length. The adjustment method is as follows: the current value of the abnormal density threshold is updated to the product of the initial value D zero of the abnormal density threshold and the adjustment number. The adjustment coefficient is the quotient obtained by dividing the initial length L zero of the sliding window by the updated sliding window length. Save the updated sliding window length and the updated anomaly density threshold for comparison operations during the next sliding window movement.

[0013] Furthermore, the initial value D of the abnormal density threshold is obtained by pre-analyzing historical normal physiological parameter data.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This algorithm optimizes the sliding window anomaly detection algorithm by leveraging an adaptive threshold mechanism that dynamically adjusts the window length. It breaks free from the constraints of fixed window size and fixed judgment thresholds, flexibly changing the window coverage range in accordance with the fluctuation rhythm of cattle physiological parameters in time-series data, and synchronously matching and adapting the judgment threshold to the current data distribution characteristics. This approach closely reflects the fluctuation differences in physiological data during different scenarios such as daily activities, rest, and feeding of cattle. The identification of anomalies in time-series data aligns with the changing patterns of the original data, mitigating the detection adaptation bias caused by fixed parameters.

[0015] By integrating the sequence of physiological abnormality period markers and the sequence of behavioral posture feature vectors using a unified timestamp, the independent state of biosensor physiological data and visual posture feature data is broken, allowing physiological abnormality information and behavioral posture change information to form a temporal correspondence. A multi-dimensional fused health status representation tensor is constructed to enrich the representation dimensions and information content of health status data, adapt to the parsing needs of hierarchical networks for complex health features, and refine the feature distinguishability between different health states. Attached Figure Description

[0016] Figure 1 This is a timing diagram of the cattle health status analysis system that integrates biosensing and pattern visual recognition as described in this invention. Figure 2 A flowchart for extracting behavioral pose feature vector sequences from pose key points; Figure 3 The flowchart for generating the tensor for cross-modal health state representation. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 The specific implementation of this cattle herd health status analysis system that integrates biosensing and pattern visual recognition is as follows.

[0019] The system comprises a data acquisition module, an anomaly analysis module, a feature extraction module, a cross-modal fusion module, and a health status classification module. The data acquisition module acquires time-series physiological parameter data collected by biosensors worn by each experimental cow in the target herd, as well as sequences of cow behavior images synchronously captured by visual acquisition devices fixed in the cowshed. The anomaly analysis module applies an improved sliding window anomaly detection algorithm to the physiological parameter time-series data, generating a sequence of physiological anomaly time periods for each experimental cow. This improved sliding window anomaly detection algorithm is based on an adaptive threshold mechanism that dynamically adjusts the window length. The feature extraction module performs posture keypoint extraction on the cow behavior image sequences, generating a sequence of behavioral posture feature vectors for each experimental cow. The cross-modal fusion module aligns and fuses the physiological anomaly time period marker sequence with the behavioral posture feature vector sequence according to timestamps, generating a cross-modal health status representation tensor for each experimental cow. The health status classification module inputs the cross-modal health status representation tensor into a pre-trained health status classification network, outputting the individual health status level for each experimental cow.

[0020] In one embodiment of the present invention, the step of executing an improved sliding window anomaly detection algorithm on the physiological parameter time series data set to generate a physiological abnormality period marker sequence for each experimental cow specifically includes: grouping the physiological parameter time series data set according to the individual identifier of each experimental cow to obtain individual physiological parameter time series data corresponding to each experimental cow, wherein the individual physiological parameter time series data includes body temperature sampling sequence, heart rate sampling sequence, and rumination sound intensity sampling sequence. The improved sliding window anomaly detection algorithm is executed on the body temperature sampling sequence, the heart rate sampling sequence, and the rumination sound intensity sampling sequence respectively. The working process of the improved sliding window anomaly detection algorithm is as follows: initializing a sliding window of length L, placing the sliding window at the beginning position of the body temperature sampling sequence, and calculating the arithmetic mean and standard deviation of all sampling points within the sliding window. Comparing the value of each sampling point within the sliding window with the arithmetic mean, when the absolute difference between the value of a sampling point and the arithmetic mean is greater than three times the standard deviation, the sampling point is marked as a candidate anomaly point. The number of candidate anomalies within the sliding window is counted. When the number of candidate anomalies exceeds a preset anomaly density threshold, the current position of the sliding window is marked as an anomaly window, and the length L of the sliding window is increased by one step unit to obtain the updated window length.

[0021] When the number of candidate outliers is less than or equal to the preset outlier density threshold, the length L of the sliding window is reduced by one step unit to obtain the updated window length, and the sliding window is moved forward by one step unit. The operations of calculating the arithmetic mean and standard deviation, marking candidate outliers, judging outlier windows, and adjusting the sliding window length are repeated until the sliding window traverses the entire body temperature sampling sequence. The time intervals covered by all outlier windows in the body temperature sampling sequence are merged to obtain the body temperature outlier time period labeling sequence corresponding to the body temperature sampling sequence. The heart rate sampling sequence and the rumination sound intensity sampling sequence are processed according to the same procedure to obtain the heart rate outlier time period labeling sequence and the rumination outlier time period labeling sequence. The body temperature outlier time period labeling sequence, the heart rate outlier time period labeling sequence, and the rumination outlier time period labeling sequence are time-series merged, and the union of the three sequences is taken as the physiological outlier time period labeling sequence for each experimental cow.

[0022] In practical implementation, taking a large-scale dairy farm as an example, 50 experimental cows were selected as the target herd. Each experimental cow was equipped with a neck collar-type biosensor device, which continuously collected body temperature, heart rate, and rumination sound intensity data for 72 hours at a sampling frequency of once per minute. At the same time, four high-definition visual acquisition devices installed in the cowshed synchronously captured the cow's behavioral image sequence at a frame rate of 2 frames per second. The data acquisition module transmitted the acquired physiological parameter time series data set and the cow behavioral image sequence to the anomaly analysis module. The anomaly analysis module grouped the physiological parameter time series data set according to the individual identifier of each experimental cow. The individual identifier is the electronic ear tag number, which yields the individual physiological parameter time series data corresponding to each experimental cow. The individual physiological parameter time series data includes body temperature sampling sequence, heart rate sampling sequence, and rumination sound intensity sampling sequence.

[0023] In some embodiments, an improved sliding window anomaly detection algorithm is executed on the body temperature sampling sequence. A sliding window of length L is initialized, with an initial value of 30 sampling points. The sliding window is placed at the beginning of the body temperature sampling sequence. The arithmetic mean and standard deviation of all 30 sampling points within the sliding window are calculated. The value of each sampling point within the sliding window is compared with the arithmetic mean. When the absolute difference between the value of a sampling point and the arithmetic mean is greater than three times the standard deviation, the sampling point is marked as a candidate anomaly. The number of candidate anomalies within the sliding window is counted. When the number of candidate anomalies exceeds a preset anomaly density threshold, an anomaly is detected. The initial value of the constant density threshold is set to 0.2. The current position of the sliding window is marked as an abnormal window, and the length L of the sliding window is increased by one step unit, with a step unit of 5 sampling points, resulting in an updated window length of 35 sampling points. When the number of candidate abnormal points is less than or equal to the abnormal density threshold, the length L of the sliding window is decreased by one step unit, resulting in an updated window length of 25 sampling points. The sliding window is then moved forward by one step unit. The operations of calculating the arithmetic mean and standard deviation, marking candidate abnormal points, judging abnormal windows, and adjusting the length of the sliding window are repeated until the sliding window traverses the entire body temperature sampling sequence.

[0024] In practice, a dynamic adaptive threshold mechanism is introduced during the adjustment of the sliding window length. The current value of the abnormal density threshold is updated according to the following formula: in: This represents the updated anomaly density threshold. This represents the initial value of the abnormal density threshold. This indicates the initial length of the sliding window. This indicates the updated length of the sliding window.

[0025] Optionally, all time intervals covered by abnormal windows in the body temperature sampling sequence are merged. Time intervals that overlap or are less than 2 minutes apart are merged to obtain a body temperature abnormal period marker sequence. The heart rate sampling sequence and rumination sound intensity sampling sequence are processed separately according to the same working process to obtain a heart rate abnormal period marker sequence and a rumination abnormal period marker sequence. The initial length of the sliding window of the heart rate sampling sequence is set to 20 sampling points, and the initial length of the sliding window of the rumination sound intensity sampling sequence is set to 40 sampling points. The initial value of the abnormal density threshold is set to 0.2 for both.

[0026] It is understandable that the abnormal body temperature time period marker sequence, abnormal heart rate time period marker sequence, and abnormal rumination time period marker sequence are temporally merged, and the union of the three sequences is taken as the physiological abnormal time period marker sequence for each experimental cow. For the experimental cow numbered N012, the abnormal body temperature time period marker sequence includes two time periods from 120 minutes to 135 minutes and from 240 minutes to 255 minutes, the abnormal heart rate time period marker sequence includes one time period from 122 minutes to 140 minutes, and the abnormal rumination time period marker sequence includes two time periods from 118 minutes to 130 minutes and from 238 minutes to 260 minutes. The physiological abnormal time period marker sequence obtained after merging is from 118 minutes to 140 minutes and from 238 minutes to 260 minutes.

[0027] In some embodiments, when performing the improved sliding window anomaly detection algorithm on the body temperature sampling sequence, the minimum length of the sliding window is set to 15 sampling points, and the maximum length is set to 60 sampling points. When the sliding window length exceeds the maximum length after updating, it remains at the maximum length; when it is lower than the minimum length, it remains at the minimum length. The step size of the sliding window is fixed at 1 sampling point each time, ensuring point-by-point traversal of the entire sequence. Optionally, for the anomaly detection process of the heart rate sampling sequence and the rumination sound intensity sampling sequence, their respective minimum and maximum lengths are set. The minimum length of the heart rate sampling sequence is 10 sampling points, and the maximum length is 40 sampling points. The minimum length of the rumination sound intensity sampling sequence is 20 sampling points, and the maximum length is 80 sampling points. After the traversal of each sequence, the time intervals covered by the anomaly window are merged in chronological order, and the merging interval threshold is set to 1 minute. It is understandable that after the physiological abnormality period marker sequence for each experimental cow is generated, the abnormality analysis module outputs the sequence to the cross-modal fusion module, while retaining the original body temperature abnormality period marker sequence, heart rate abnormality period marker sequence, and rumination abnormality period marker sequence for subsequent fine-grained analysis. When processing data from 50 experimental cows in batches, the time series data of individual physiological parameters of each experimental cow are independently executed using the above-mentioned sliding window abnormality detection algorithm. During the algorithm execution, the arithmetic mean and standard deviation of each sliding window are calculated in real time based on the current sampling point within the window, without relying on the statistical information of historical windows.

[0028] In one embodiment of the present invention, the step of performing a pose key point extraction operation on the bovine behavior image sequence to generate a behavioral pose feature vector sequence for each experimental bovine specifically includes: (See reference) Figure 2The local regions containing individual experimental cattle are extracted frame by frame from the sequence of cattle behavior images. Image normalization is then performed on these local regions to generate a normalized sequence of individual cattle images. A pre-trained posture keypoint detection network is then used to perform keypoint regression processing on each frame of the normalized sequence of individual cattle images, outputting the pixel coordinates of multiple trunk keypoints for each experimental cattle in each frame. These multiple trunk keypoints include the nose tip keypoint, left ear root keypoint, right ear root keypoint, neck center keypoint, scapula keypoint, midline keypoint of the back, tail root keypoint, left forehoof keypoint, right forehoof keypoint, left hindhoof keypoint, and right hindhoof keypoint. The posture description parameters of each experimental cow are calculated based on the pixel coordinates of multiple trunk key points in each frame of the image. These parameters include the angle between the line connecting the nose tip key point and the center neck key point and the horizontal line; the angle between the line connecting the center neck key point and the midpoint of the back midline key point and the horizontal line; the length of the line connecting the scapula key point and the tail root key point; the horizontal distance between the left and right forehoof key points; the horizontal distance between the left and right hindhoof key points; and the height of the midpoint of the back midline key point from the ground reference line. All posture description parameters calculated in the same frame are concatenated into a one-dimensional vector in a preset order, and this one-dimensional vector is used as the single-frame behavioral posture feature vector corresponding to that frame. The single-frame behavioral posture feature vectors corresponding to all frames are arranged in chronological order to generate the behavioral posture feature vector sequence for each experimental cow. The posture key point detection network is based on a finely tuned HRNet network model on the cattle posture image dataset.

[0029] In specific implementation, the same 50 experimental cattle as in the above embodiment are used as the target cattle herd. The feature extraction module receives a sequence of cattle behavior images synchronously captured by the visual acquisition device. The time span of the image sequence is 72 hours, the frame rate is 2 frames per second, and the resolution of each frame image is 1920×1080 pixels. The local region containing the individual experimental cattle is extracted from each frame of the cattle behavior image sequence. For the experimental cattle numbered N012, the minimum bounding box region containing the complete cattle is cropped from each frame of the original image using the YOLOv8 object detection algorithm. Image normalization processing is performed on this local region, and the size of the cropped image is uniformly scaled to 512×512 pixels. Then, the pixel value is normalized by subtracting the mean and dividing the variance to generate a normalized sequence of individual cattle images.

[0030] In some embodiments, a pre-trained pose keypoint detection network is invoked to perform keypoint regression processing on each frame of a normalized sequence of individual cattle images. The pose keypoint detection network is based on an HRNet network model fine-tuned on a cattle pose image dataset. The backbone network of the HRNet network model adopts the HRNet-W32 structure and is fine-tuned on a dataset of 20,000 cattle pose images annotated with 11 trunk keypoints. The network outputs the pixel coordinates of multiple trunk keypoints for each experimental cattle in each frame. These trunk keypoints include the nose tip keypoint, left ear root keypoint, right ear root keypoint, neck center keypoint, scapula keypoint, midline keypoint of the back, tail root keypoint, left forehoof keypoint, right forehoof keypoint, left hindhoof keypoint, and right hindhoof keypoint. In the image frame corresponding to the 180th minute of the test cow N012, the pixel coordinates of the key point of the nose tip are (320, 180), the pixel coordinates of the key point of the left ear base are (310, 160), the pixel coordinates of the key point of the right ear base are (335, 158), the pixel coordinates of the key point of the neck center are (340, 200), the pixel coordinates of the key point of the shoulder blade are (360, 250), the pixel coordinates of the key point of the midline of the back are (400, 300), the pixel coordinates of the key point of the tail base are (500, 320), the pixel coordinates of the key point of the left forehoof are (350, 400), the pixel coordinates of the key point of the right forehoof are (370, 405), the pixel coordinates of the key point of the left hindhoof are (480, 410), and the pixel coordinates of the key point of the right hindhoof are (510, 408).

[0031] In practice, the posture description parameters of each experimental cow are calculated based on the pixel coordinates of multiple trunk key points in each frame of the image. The posture description parameters include the angle between the line connecting the nose tip key point and the center neck key point and the horizontal line, the angle between the line connecting the center neck key point and the midpoint of the back key point and the horizontal line, the length of the line connecting the scapula key point and the tail root key point, the horizontal distance between the left forehoof key point and the right forehoof key point, the horizontal distance between the left hindhoo key point and the right hindhoof key point, and the height of the midpoint of the back key point from the ground reference line. The ground reference line is determined by the pre-marked horizontal line of the cowshed floor in the image, and its pixel ordinate value is 480. The height of the midpoint of the back key point from the ground reference line is calculated as the difference between the ordinate of the ground reference line and the ordinate of the midpoint of the back key point.

[0032] Optionally, the angle between the line connecting the key point of the nasal tip and the key point of the center of the neck and the horizontal line is calculated using the following formula: in: This indicates the angle between the line connecting the key point of the nose tip and the key point of the center of the neck, and the horizontal line. Represents the pixel coordinates of the key points at the tip of the nose. Represents the pixel coordinates of the key point at the center of the neck. Represents the arctangent function. It represents pi (π).

[0033] In practice, the angle between the line connecting the center point of the neck and the midpoint of the back midline and the horizontal line is calculated in the same way. The coordinates of the nose tip key point are replaced with the coordinates of the center point of the neck, and the coordinates of the center point of the neck are replaced with the coordinates of the midpoint of the back midline. The length of the line connecting the scapula key point and the tail root key point is calculated as the Euclidean distance between the two points. The horizontal distance between the left foreleg key point and the right foreleg key point is calculated as the absolute value of the difference between the x-coordinate of the right foreleg key point and the x-coordinate of the left foreleg key point. The horizontal distance between the left hind leg key point and the right hind leg key point is calculated as the absolute value of the difference between the x-coordinate of the right hind leg key point and the x-coordinate of the left hind leg key point.

[0034] Optionally, all posture description parameters calculated in the same frame are concatenated into a one-dimensional vector in a preset order. The preset order is: the angle between the line connecting the nose tip key point and the neck center key point and the horizontal line; the angle between the line connecting the neck center key point and the midpoint key point of the back midline and the horizontal line; the length of the line connecting the scapula key point and the tail root key point; the horizontal distance between the left forehoof key point and the right forehoof key point; the horizontal distance between the left hindhoof key point and the right hindhoof key point; and the height of the midpoint key point of the back midline from the ground reference line. This one-dimensional vector is used as the single-frame behavioral posture feature vector corresponding to the frame image. For the experimental cow numbered N012, in the frame image corresponding to the 180th minute, the single-frame behavioral posture feature vector is [15.3, 8.7, 210.5, 20.0, 30.0, 180.0], where the length is in pixels.

[0035] In some embodiments, the single-frame behavioral posture feature vectors corresponding to all frame images are arranged in chronological order to generate a behavioral posture feature vector sequence for each experimental cow. For an image sequence with a time span of 72 hours, 2 frames are generated per second, and each frame corresponds to a single-frame behavioral posture feature vector. Therefore, the behavioral posture feature vector sequence for each experimental cow contains 518,400 single-frame behavioral posture feature vectors. Each single-frame behavioral posture feature vector is a 6-dimensional vector, and the entire sequence is represented as a matrix of size 518,400×6, stored in ascending order of timestamps.

[0036] It is understandable that for each of the 50 experimental cattle, the feature extraction module independently performs the above-mentioned posture keypoint extraction operation. Individual image sequences of different experimental cattle are distinguished by individual identifiers in the target detection algorithm. After the behavioral posture feature vector sequence of each experimental cattle is generated, it is bound and stored with the corresponding individual identifier. It is also understandable that when calling the pre-trained posture keypoint detection network for keypoint regression processing, for each frame in the normalized individual cattle image sequence, the HRNet network model outputs a heatmap of 11 keypoints. Sub-pixel precision pixel coordinates are obtained by calculating the peak positions of the heatmap. For keypoints with a confidence level below 0.5 due to occlusion or motion blur, linear interpolation is used to complete the keypoints based on the coordinates of the same keypoint in the preceding and following frames.

[0037] In one embodiment of the present invention, the step of aligning and fusing the physiological abnormality period marker sequence and the behavioral posture feature vector sequence according to timestamps to generate a cross-modal health status representation tensor for each experimental cow specifically includes: (See reference) Figure 3 The process involves obtaining the start and end times of each abnormal time period in the physiological abnormal time period marker sequence, and the timestamp corresponding to each single-frame behavioral posture feature vector in the behavioral posture feature vector sequence. All single-frame behavioral posture feature vectors whose timestamps fall within any abnormal time period of the physiological abnormal time period marker sequence are extracted to form an abnormal time period behavioral feature subsequence. All single-frame behavioral posture feature vectors whose timestamps do not fall within any abnormal time period are extracted to form a normal time period behavioral feature subsequence. A dimensional averaging operation is performed on all single-frame behavioral posture feature vectors in the normal time period behavioral feature subsequence to obtain a normal behavioral baseline feature vector. Dimensional difference calculations are performed between each single-frame behavioral posture feature vector in the abnormal time period behavioral feature subsequence and the normal behavioral baseline feature vector to obtain a deviation feature vector corresponding to each single-frame behavioral posture feature vector. Each deviation feature vector is bound to the time period identifier information of its corresponding abnormal time period, and the binding results are stacked in chronological order to generate a cross-modal health status representation tensor for each experimental cow. The first dimension of the cross-modal health status representation tensor corresponds to the chronological order of the abnormal time period, and the second dimension corresponds to the components of each dimension of the deviation feature vector. The time period identifier information is the sequential number of the corresponding abnormal time period in the physiological abnormal time period labeling sequence.

[0038] In specific implementation, the same 50 experimental cattle as in the above embodiments are used as the target cattle herd. The cross-modal fusion module receives the physiological abnormal period marker sequence of each experimental cattle output from the anomaly analysis module and the behavioral posture feature vector sequence of each experimental cattle output from the feature extraction module. For the experimental cattle numbered N012, the physiological abnormal period marker sequence contains two abnormal periods. The start point of the first abnormal period is the 118th minute and the end point is the 140th minute. The start point of the second abnormal period is the 238th minute and the end point is the 260th minute. The timestamp corresponding to each single frame behavioral posture feature vector in the behavioral posture feature vector sequence is in seconds. The timestamp corresponding to the 118th minute is 7080 seconds, the timestamp corresponding to the 140th minute is 8400 seconds, the timestamp corresponding to the 238th minute is 14280 seconds, and the timestamp corresponding to the 260th minute is 15600 seconds.

[0039] In some embodiments, all single-frame behavioral posture feature vectors whose timestamps fall within any abnormal time period of the physiological abnormal time period marker sequence are extracted from the behavioral posture feature vector sequence to form an abnormal time period behavioral feature subsequence. For the experimental cow numbered N012, all single-frame behavioral posture feature vectors with timestamps ranging from 7080 seconds to 8400 seconds are extracted as the first abnormal time period behavioral feature subsequence, and all single-frame behavioral posture feature vectors with timestamps ranging from 14280 seconds to 15600 seconds are extracted as the second abnormal time period behavioral feature subsequence. The number of single-frame behavioral posture feature vectors contained in each abnormal time period behavioral feature subsequence is equal to the duration of the abnormal time period multiplied by 60 seconds per minute and then multiplied by 2 frames per second. The first abnormal time period lasts for 22 minutes and contains 2640 single-frame behavioral posture feature vectors, each of which is a 6-dimensional vector.

[0040] In practice, all single-frame behavioral posture feature vectors whose timestamps do not fall within any abnormal time periods are extracted from the behavioral posture feature vector sequence to form a normal time period behavioral feature subsequence. For the test cow numbered N012, the entire 72-hour time span is 259,200 seconds. Excluding the 3,480 seconds covered by the two abnormal time periods (22 minutes plus 22 minutes equals 44 minutes, or 2,640 seconds), the remaining 256,560 seconds correspond to 513,120 single-frame behavioral posture feature vectors. These vectors are arranged in the order of timestamps to form a normal time period behavioral feature subsequence.

[0041] Optionally, an dimensional averaging operation is performed on all single-frame behavior pose feature vectors in the normal time period behavior feature subsequence to obtain the normal behavior baseline feature vector. The normal time period behavior feature subsequence contains N single-frame behavior pose feature vectors, each of which is 6-dimensional. Let the j-th dimension component of the i-th vector be denoted as... Then the j-th dimension component of the normal behavior baseline feature vector Calculated as For the experimental cow numbered N012, the baseline feature vector for normal behavior is [14.2,7.5,208.3,19.8,29.5,178.6], where the meaning of each dimension is consistent with the preset order in the single-frame behavior posture feature vector.

[0042] In practice, a dimension-wise difference calculation is performed between the single-frame behavior posture feature vector and the normal behavior baseline feature vector in the behavior feature subsequence of the abnormal period, to obtain the deviation feature vector corresponding to each single-frame behavior posture feature vector. The j-th dimension component of the deviation feature vector is calculated according to the following formula: in: This represents the j-th dimension of the deviation feature vector component in the behavioral feature subsequence of the k-th abnormal time period. This represents the j-th dimension component of the single-frame action pose feature vector in this subsequence. This represents the j-th component of the baseline feature vector for normal behavior.

[0043] It can be understood that for the 2640 single-frame behavioral posture feature vectors in the behavioral feature subsequence of the first abnormal time period, each vector is subtracted dimension-by-dimensional from the normal behavioral baseline feature vector to obtain 2640 deviation feature vectors. Each deviation feature vector is still 6-dimensional and may contain positive, negative, or zero values, representing the direction and magnitude of the deviation of the posture description parameter in that dimension from the normal baseline, respectively. Optionally, each deviation feature vector is bound to the time period identifier information of its corresponding abnormal time period. The time period identifier information is the sequential number of the corresponding abnormal time period in the physiological abnormal time period marking sequence. For the experimental cow with the number N012, the sequential number of the first abnormal time period is 1, and the sequential number of the second abnormal time period is 2. The 2640 deviation feature vectors corresponding to the first abnormal time period are bound to sequential number 1, and the 2640 deviation feature vectors corresponding to the second abnormal time period are bound to sequential number 2.

[0044] In practice, the binding results are stacked in chronological order to generate a cross-modal health status representation tensor for each experimental cow. The first dimension of the cross-modal health status representation tensor corresponds to the chronological order of the abnormal periods, and the second dimension corresponds to the dimensional components of the deviation feature vectors. For the experimental cow numbered N012, the size of the cross-modal health status representation tensor is 2×2640×6, where the size of the first dimension of 2 represents the two abnormal periods, the size of the second dimension of 2640 represents the number of deviation feature vectors contained in each abnormal period, and the size of the third dimension of 6 represents the dimension of each deviation feature vector. For the experimental cow with only one abnormal period, the size of the cross-modal health status representation tensor is 1×M×6, where M is the number of deviation feature vectors in that abnormal period.

[0045] It is understandable that for experimental cattle whose physiological abnormality period marker sequence does not contain any abnormal periods, the physiological abnormality period marker sequence is an empty sequence. At this time, the cross-modal fusion module generates an empty tensor, and the health status classification module directly determines the experimental cattle corresponding to the empty tensor as healthy, without inputting it into the health status classification network for processing.

[0046] In some embodiments, after the cross-modal health status representation tensor is generated, it is bound and stored with the individual identifier of the corresponding experimental cow and transmitted to the health status classification module. For each of the 50 experimental cows, a cross-modal health status representation tensor of its own size is generated. The number of abnormal periods of different experimental cows is different, so the size of the tensor in the first dimension is different. When the health status classification module receives the tensor, it processes it according to the individual identifier of each cow.

[0047] In one embodiment of the present invention, the step of inputting the cross-modal health status representation tensor into a pre-trained health status classification network and outputting the individual health status level of each experimental cow specifically includes: flattening the cross-modal health status representation tensor along a first dimension to generate a one-dimensional health status input feature vector; inputting the one-dimensional health status input feature vector into the input layer of the pre-trained health status classification network, wherein the pre-trained health status classification network is a multi-layer classification network composed of multiple fully connected layers stacked together; performing a linear transformation operation on the one-dimensional health status input feature vector in the first fully connected layer of the pre-trained health status classification network to generate a first intermediate feature vector; performing a linear transformation operation on the first intermediate feature vector and then applying an activation function in the second fully connected layer of the pre-trained health status classification network to generate a second intermediate feature vector; and sequentially passing the remaining fully connected layers of the pre-trained health status classification network, performing linear transformation and activation function processing on the output of the previous layer layer by layer until the result vector of the output layer of the pre-trained health status classification network is obtained. The output vector of the output layer is subjected to a normalized exponential function transformation to obtain the probability distribution of each experimental cow belonging to each preset health status level. The health status level corresponding to the maximum probability is taken as the individual health status level of each experimental cow. The pre-trained health status classification network is trained using a labeled cross-modal health status representation tensor sample set, and the labels are marked by veterinary experts based on clinical diagnosis results.

[0048] In a specific implementation, taking the experimental cow numbered N012 in the above embodiment as an example, the health status classification module receives the cross-modal health status representation tensor output from the cross-modal fusion module. The size of this tensor is 2×2640×6, where the size of the first dimension is 2 to represent two abnormal time periods, the size of the second dimension is 2640 to represent the number of deviation feature vectors contained in each abnormal time period, and the size of the third dimension is 6 to represent the dimension of each deviation feature vector. The cross-modal health status representation tensor is flattened along the first dimension to generate a one-dimensional health status input feature vector. The flattening operation arranges the elements in the tensor into a one-dimensional vector in the order of the first dimension, the second dimension, and the third dimension. For a tensor of size 2×2640×6, the length of the flattened one-dimensional health status input feature vector is 31680.

[0049] In some embodiments, a one-dimensional health status input feature vector is input into the input layer of a pre-trained health status classification network. The pre-trained health status classification network is a multi-layer classification network composed of multiple fully connected layers stacked together. The network contains three fully connected hidden layers and one fully connected output layer. The number of neurons in the input layer is 31680, the number of neurons in the first fully connected hidden layer is 1024, the number of neurons in the second fully connected hidden layer is 512, and the number of neurons in the third fully connected hidden layer is 256. The number of neurons in the fully connected output layer is equal to the number of preset health status levels. The preset health status levels include four levels: healthy, sub-healthy, mild illness, and severe illness. Therefore, the number of neurons in the output layer is 4.

[0050] In the specific implementation, a linear transformation operation is performed on the one-dimensional health state input feature vector in the first fully connected layer of the pre-trained health state classification network to generate the first intermediate feature vector. The linear transformation operation is to multiply the input vector by the weight matrix of the first fully connected layer and add the bias vector. The length of the one-dimensional health state input feature vector is 31680, the size of the weight matrix of the first fully connected layer is 1024×31680, and the length of the bias vector is 1024. After the linear transformation, the first intermediate feature vector with a length of 1024 is obtained.

[0051] Optionally, in the second fully connected layer of the pre-trained health status classification network, a linear transformation operation is performed on the first intermediate feature vector and then processed by an activation function to generate a second intermediate feature vector. The activation function is a modified linear unit. The linear transformation operation is to multiply the first intermediate feature vector by the weight matrix of the second fully connected layer and add a bias vector. The weight matrix of the second fully connected layer has a size of 512×1024 and the bias vector has a length of 512. After the linear transformation, a vector of length 512 is obtained. Then, the modified linear unit function is applied element by element to set the negative values ​​to zero to obtain the second intermediate feature vector.

[0052] In practice, the remaining fully connected layers of the pre-trained health status classification network sequentially process the output of the previous layer with linear transformations and activation functions until the output vector of the pre-trained health status classification network is obtained. The third fully connected hidden layer performs a linear transformation on the second intermediate feature vector and then applies a modified linear unit activation function to obtain a third intermediate feature vector of length 256. The fully connected output layer performs a linear transformation on the third intermediate feature vector but does not apply an activation function, resulting in an output vector of length 4. Let the output vector be denoted as . ,in The non-normalized scores correspond to four levels: healthy, sub-healthy, mild illness, and severe illness.

[0053] It is understandable that performing a normalized exponential function transformation on the output layer's result vector yields the probability distribution of each experimental cow belonging to each preset health state level. The normalized exponential function transformation is performed according to the following formula: in: This indicates that the experimental cattle belong to the first... The probability of a preset health status level. This represents the first element in the output layer result vector. One portion, Represented by natural constant The base is an exponential function, where the denominator represents the summation of the exponents of the unnormalized scores for each of the four levels. and All values ​​are integers from 1 to 4.

[0054] In practice, the health status level corresponding to the maximum probability is taken as the individual health status level of each experimental cow. For the experimental cow numbered N012, the calculated probability distribution is as follows: The maximum value of 0.75 corresponds to the third level, which is mild illness. Therefore, the output individual health status level is mild illness.

[0055] In some embodiments, the pre-trained health status classification network is trained using a sample set of labeled cross-modal health status representation tensors. The labels are provided by veterinary experts based on clinical diagnostic results. The sample set contains 2000 samples, each of which is a cross-modal health status representation tensor collected from different experimental cattle and its corresponding health status level label. During training, the cross-entropy loss function and Adam optimizer are used, with a learning rate of 0.001, a batch size of 32, and 100 training epochs. The network weights are randomly initialized using a normal distribution. After training, the network parameters are fixed for actual deployment.

[0056] Optionally, before the training phase, a flattening operation is performed on each sample in the cross-modal health status representation tensor sample set to generate a one-dimensional health status input feature vector. The health status input feature vectors of all samples constitute the training feature matrix, and the corresponding labels constitute the training label vector. During training, the feature vectors of 32 samples are randomly selected from each batch and input into the network. The loss is calculated and backpropagation is used to update the network weights. It can be understood that for experimental cattle with an empty sequence as the marker sequence during physiological abnormality periods, the cross-modal fusion module outputs an empty tensor, and the health status classification module directly classifies the individual health status level of the experimental cattle as healthy, without processing by the health status classification network. Alternatively, after training, the pre-trained health status classification network is deployed in the health status classification module. For the cross-modal health status representation tensor generated in real time for each experimental cattle, the health status classification module outputs the individual health status level according to the aforementioned flattening, layer-by-layer linear transformation and activation, and normalized exponential function transformation process. The entire process is completed within 200 milliseconds after receiving the tensor.

[0057] In one embodiment of the present invention, the improved sliding window anomaly detection algorithm is based on an adaptive threshold mechanism that dynamically adjusts the window length. Specifically, it includes setting an initial length Lzero, a minimum length Lminimum, and a maximum length Lmaximum for the sliding window, and setting an initial value Dzero for the anomaly density threshold. During the sliding window's traversal of the body temperature sampling sequence, each time the sliding window moves once, a current window density value is calculated based on the number of sampling points currently covered by the sliding window. The current window density value is the ratio of the number of candidate anomaly points within the sliding window to the current length of the sliding window. The current window density value is compared with the current value of the anomaly density threshold. When the current window density value is greater than the current value of the anomaly density threshold, the length of the sliding window is updated to the current length plus one step unit, but the updated length does not exceed the maximum length Lmaximum.

[0058] When the current window density value is less than or equal to the current value of the abnormal density threshold, the length of the sliding window is updated to the current length minus one step unit, but the updated length is not less than the minimum length L. Each time the length of the sliding window is updated, the current value of the abnormal density threshold is dynamically adjusted based on the updated sliding window length. The adjustment method is as follows: the current value of the abnormal density threshold is updated to the product of the initial value D (zero value) of the abnormal density threshold and an adjustment factor, where the adjustment factor is the quotient obtained by dividing the initial length L (zero value) of the sliding window by the updated sliding window length. The updated sliding window length and the updated abnormal density threshold are saved for comparison operations during the next sliding window movement. The initial value D (zero value) of the abnormal density threshold is obtained through pre-analysis of historical normal physiological parameter data.

[0059] In specific implementation, taking the body temperature sampling sequence of a test cow with the number N012 as an example, the body temperature sampling sequence contains 4320 sampling points, with a sampling frequency of once per minute, for a total of 72 hours. The improved sliding window anomaly detection algorithm is based on an adaptive threshold mechanism that dynamically adjusts the window length. The specific implementation process is as follows: the initial length L of the sliding window is set to zero with 30 sampling points, the minimum length L is set to a minimum of 15 sampling points, and the maximum length L is set to a maximum of 60 sampling points. The initial value D of the anomaly density threshold is set to zero with 0.2. The initial value D of the anomaly density threshold is obtained by pre-analyzing historical normal physiological parameter data. That is, 100 windows with a length of 30 sampling points are randomly selected from the body temperature sampling sequences of healthy cattle in the past year in this farm. The ratio of the number of candidate anomalies in each window to the window length is calculated, and the 95th percentile of these ratios is taken as the zero value D.

[0060] In some embodiments, during the process of traversing the body temperature sampling sequence by the sliding window, each time the sliding window moves once, the current window density value is calculated based on the number of sampling points currently covered by the sliding window. The current window density value is the ratio of the number of candidate abnormal points in the sliding window to the current length of the sliding window. The sliding window is initially placed at the beginning of the body temperature sampling sequence, and the current length is 30 sampling points. If 2 candidate abnormal points are detected in the window, the current window density value is calculated as 2 divided by 30, which equals 0.0667. The current window density value of 0.0667 is compared with the current value of the abnormal density threshold of 0.2. Since 0.0667 is less than 0.2, the operation of reducing the length of the sliding window is performed.

[0061] In practice, when the current window density value is greater than the current value of the abnormal density threshold, the length of the sliding window is updated to the current length plus one step unit, with the step unit set to 5 sampling points. However, the updated length does not exceed the maximum length L, which is 60 sampling points. When the current window density value is less than or equal to the current value of the abnormal density threshold, the length of the sliding window is updated to the current length minus one step unit. However, the updated length is not less than the minimum length L, which is 15 sampling points. For the case where the window density value is 0.0667, which is less than 0.2, the sliding window length is reduced from 30 sampling points to 25 sampling points, and the sliding window is moved forward by one step unit, with the step unit being 1 sampling point, and the detection continues at the next position.

[0062] Optionally, each time the length of the sliding window is updated, the current value of the abnormal density threshold is dynamically adjusted according to the updated sliding window length. The adjustment method is as follows: the current value of the abnormal density threshold is updated to the product of the initial value D (zero value) of the abnormal density threshold and the adjustment factor. The adjustment factor is the quotient obtained by dividing the initial length L (zero value) of the sliding window by the updated sliding window length. In specific implementation, the update of the abnormal density threshold is performed according to the following formula: in: This represents the updated anomaly density threshold. The initial value for the abnormal density threshold is 0.2. This indicates the initial length of the sliding window is 30 sampling points. This indicates the updated sliding window length. When the sliding window length is updated from 30 sampling points to 25 sampling points, It equals 25, calculated as follows It is understandable that the updated sliding window length and the updated abnormal density threshold are saved for comparison operations during the next sliding window movement. That is, after the next movement, the window density value is calculated and compared using the currently saved sliding window length of 25 sampling points and the abnormal density threshold of 0.24.

[0063] In some embodiments, the traversal process described above continues, the sliding window moves to the next position, the current length is 25 sampling points, the anomaly density threshold is 0.24, 8 candidate anomalies are detected within the window, and the current window density value is 8 divided by 25 equals 0.32. Since 0.32 is greater than 0.24, the sliding window length is increased by one step unit, updating the sliding window length from 25 sampling points to 30 sampling points, but not exceeding the maximum length of 60 sampling points. The anomaly density threshold is dynamically adjusted based on the updated length of 30 sampling points, and the calculation is performed. The window is saved with a length of 30 sampling points and an abnormal density threshold of 0.2. The sliding window is then moved forward by one step unit.

[0064] In practice, when the current length of the sliding window has reached the maximum length L (60 sampling points), even if the current window density value is greater than the current value of the abnormal density threshold, the sliding window length will not be increased and will remain at 60 sampling points. Similarly, when the current length of the sliding window has reached the minimum length L (15 sampling points), even if the current window density value is less than or equal to the current value of the abnormal density threshold, the sliding window length will not be decreased and will remain at 15 sampling points.

[0065] It is understandable that during the traversal of the entire body temperature sampling sequence, the sliding window moves one sampling point from the beginning of the sequence each time until the right end of the sliding window reaches the end of the sequence. After each movement, the current window density value is calculated, compared with the abnormal density threshold, the window length is adjusted, the abnormal density threshold is dynamically adjusted, and the updated parameters are saved. For a body temperature sampling sequence with a length of 4320 sampling points, the window position is calculated a total of 4320 times.

[0066] Optionally, for the heart rate sampling sequence and the rumination sound intensity sampling sequence, the initial values ​​of the sliding window initial length, minimum length, maximum length, and abnormal density threshold are set respectively. The initial value of the sliding window L for the heart rate sampling sequence is set to 20 sampling points, the minimum length L is 10 sampling points, and the maximum length L is 40 sampling points. The initial value D for the abnormal density threshold is obtained by pre-analyzing historical normal heart rate parameter data and is set to 0.15. The initial value of the sliding window L for the rumination sound intensity sampling sequence is set to 40 sampling points, the minimum length L is 20 sampling points, and the maximum length L is 80 sampling points. The initial value D for the abnormal density threshold is obtained by pre-analyzing historical normal rumination sound intensity parameter data and is set to 0.25.

[0067] In some embodiments, each sampling sequence independently performs the above-described dynamic adjustment window length and adaptive threshold update process. Different sequences do not share the sliding window length and abnormal density threshold parameters. After the sliding window traversal is completed, the time intervals covered by all abnormal windows in each sequence are output for subsequent generation of body temperature abnormal period marker sequences, heart rate abnormal period marker sequences, and rumination abnormal period marker sequences.

[0068] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A system for analyzing the health status of a herd of cattle by fusing biosensing and pattern visual recognition, characterized in that, The system includes: The data acquisition module acquires the time-series data set of physiological parameters collected by the biosensor devices worn by each experimental cow in the target herd, as well as the sequence of cattle behavior images synchronously captured by the visual acquisition equipment fixed in the cowshed. The anomaly analysis module executes an improved sliding window anomaly detection algorithm on the physiological parameter time series data set to generate a physiological anomaly period marker sequence for each experimental cow. The improved sliding window anomaly detection algorithm is based on an adaptive threshold mechanism that dynamically adjusts the window length. The feature extraction module performs posture key point extraction on the cattle behavior image sequence to generate a behavior posture feature vector sequence for each experimental cattle. The cross-modal fusion module aligns and fuses the physiological abnormality period marker sequence with the behavioral posture feature vector sequence according to the timestamp to generate a cross-modal health status representation tensor for each experimental cow. The health status classification module inputs the cross-modal health status representation tensor into a pre-trained health status classification network and outputs the individual health status level of each experimental cow.

2. The system for analyzing the health status of a cow herd by fusing biosensing and pattern visual recognition according to claim 1, characterized in that, The steps of performing an improved sliding window anomaly detection algorithm on the time-series physiological parameter data set to generate a sequence of physiologically abnormal time periods for each experimental cow specifically include: The physiological parameter time series data set is grouped according to the individual identifier of each experimental cow to obtain the individual physiological parameter time series data corresponding to each experimental cow. The individual physiological parameter time series data includes body temperature sampling sequence, heart rate sampling sequence and rumination sound intensity sampling sequence. An improved sliding window anomaly detection algorithm is executed on the body temperature sampling sequence, the heart rate sampling sequence, and the rumination sound intensity sampling sequence, respectively. The working process of the improved sliding window anomaly detection algorithm is as follows: Initialize a sliding window of length L, place the sliding window at the beginning of the body temperature sampling sequence, and calculate the arithmetic mean and standard deviation of all sampling points within the sliding window; The value of each sampling point within the sliding window is compared with the arithmetic mean. When the absolute difference between the value of a sampling point and the arithmetic mean is greater than three times the standard deviation, the sampling point is marked as a candidate outlier. The number of candidate anomalies within the sliding window is counted. When the number of candidate anomalies exceeds a preset anomaly density threshold, the current position of the sliding window is marked as an anomaly window, and the length L of the sliding window is increased by one step unit to obtain the updated window length. When the number of candidate anomalies is less than or equal to the preset anomaly density threshold, the length L of the sliding window is reduced by one step unit to obtain the updated window length, and the sliding window is moved forward by one step unit. Repeat the operations of calculating the arithmetic mean and standard deviation, marking candidate outliers, judging the outlier window, and adjusting the length of the sliding window until the sliding window has traversed the entire body temperature sampling sequence; The time intervals covered by all abnormal windows in the body temperature sampling sequence are merged to obtain the body temperature abnormal time period marker sequence corresponding to the body temperature sampling sequence. The heart rate sampling sequence and the rumination sound intensity sampling sequence are processed according to the same working process to obtain the heart rate abnormal period marker sequence and the rumination abnormal period marker sequence; The abnormal body temperature time period marker sequence, the abnormal heart rate time period marker sequence, and the abnormal rumination time period marker sequence are time-series merged, and the union of the three sequences is taken as the physiological abnormal time period marker sequence for each experimental cow.

3. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 2, characterized in that, The steps of performing pose key point extraction on the cattle behavior image sequence to generate a behavior pose feature vector sequence for each experimental cattle specifically include: The local regions containing individual experimental cattle are extracted frame by frame from the sequence of images of cattle behavior, and image normalization processing is performed on the local regions to generate a normalized sequence of individual cattle images. The pre-trained posture keypoint detection network is invoked to perform keypoint regression processing on each frame of the normalized individual cattle image sequence, and outputs the pixel coordinates of multiple trunk keypoints of each experimental cattle in each frame image. The multiple trunk keypoints include the nose tip keypoint, the left ear root keypoint, the right ear root keypoint, the neck center keypoint, the scapula keypoint, the midpoint of the back midline keypoint, the tail root keypoint, the left forehoof keypoint, the right forehoof keypoint, the left hindhoof keypoint, and the right hindhoof keypoint. The posture description parameters of each experimental cow are calculated based on the pixel coordinates of multiple trunk key points in each frame of the image. The posture description parameters include the angle between the line connecting the nose tip key point and the center neck key point and the horizontal line, the angle between the line connecting the center neck key point and the midpoint of the back key point and the horizontal line, the length of the line connecting the scapula key point and the tail root key point, the horizontal distance between the left foreleg key point and the right foreleg key point, the horizontal distance between the left hind leg key point and the right hind leg key point, and the height of the midpoint of the back key point from the ground reference line. All pose description parameters calculated in the same frame are concatenated into a one-dimensional vector in a preset order, and this one-dimensional vector is used as the single-frame behavior pose feature vector corresponding to the frame. Arrange the single-frame behavioral posture feature vectors corresponding to all frame images in chronological order to generate a behavioral posture feature vector sequence for each experimental cow.

4. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 3, characterized in that, The pose keypoint detection network is based on an HRNet network model finely tuned on a cattle pose image dataset.

5. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 3, characterized in that, The step of aligning and fusing the physiological abnormality period marker sequence with the behavioral posture feature vector sequence according to timestamps to generate a cross-modal health status representation tensor for each experimental cow specifically includes: Obtain the start and end times of each abnormal time period in the physiological abnormal time period marker sequence, and the timestamp corresponding to each single frame behavioral posture feature vector in the behavioral posture feature vector sequence; Extract all single-frame behavioral posture feature vectors from the behavioral posture feature vector sequence whose timestamps fall within any abnormal time period of the physiological abnormal time period marker sequence, and form an abnormal time period behavioral feature subsequence. Extract all single-frame behavioral posture feature vectors from the behavioral posture feature vector sequence whose timestamps do not fall within any abnormal time periods to form a normal time period behavioral feature subsequence. A dimension-wise averaging operation is performed on all single-frame behavioral pose feature vectors in the normal time period behavioral feature subsequence to obtain the normal behavior baseline feature vector; Perform a dimension-wise difference calculation between each single-frame behavior posture feature vector in the abnormal period behavior feature subsequence and the normal behavior baseline feature vector to obtain the deviation feature vector corresponding to each single-frame behavior posture feature vector. Each deviation feature vector is bound to the time period identifier information of its corresponding abnormal time period, and the binding results are stacked in chronological order to generate a cross-modal health status representation tensor for each experimental cow. The first dimension of the cross-modal health status representation tensor corresponds to the chronological order of the abnormal time period, and the second dimension corresponds to each dimension component of the deviation feature vector.

6. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 5, characterized in that, The time period identification information is the sequential number of the corresponding abnormal time period in the physiological abnormal time period marking sequence.

7. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 5, characterized in that, The steps of inputting the cross-modal health status representation tensor into a pre-trained health status classification network and outputting the individual health status level of each experimental cow specifically include: The cross-modal health state representation tensor is flattened along the first dimension to generate a one-dimensional health state input feature vector. The one-dimensional health status input feature vector is input into the input layer of the pre-trained health status classification network, which is a multi-layer classification network composed of multiple fully connected layers stacked together. In the first fully connected layer of the pre-trained health state classification network, a linear transformation operation is performed on the one-dimensional health state input feature vector to generate a first intermediate feature vector. In the second fully connected layer of the pre-trained health status classification network, a linear transformation operation is performed on the first intermediate feature vector and then processed by an activation function to generate a second intermediate feature vector; The output of the pre-trained health status classification network is processed sequentially through the remaining fully connected layers, and linear transformation and activation function processing are performed on the output of the previous layer until the result vector of the output layer of the pre-trained health status classification network is obtained. The output vector of the output layer is subjected to a normalized exponential function transformation to obtain the probability distribution of each experimental cow belonging to each preset health status level. The health status level corresponding to the maximum probability is taken as the individual health status level of each experimental cow.

8. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 7, characterized in that, The pre-trained health status classification network is trained using a set of labeled cross-modal health status representation tensors, with the labels provided by veterinary experts based on clinical diagnostic results.

9. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 7, characterized in that, The improved sliding window anomaly detection algorithm is based on an adaptive threshold mechanism that dynamically adjusts the window length, specifically including: Set the initial length L (zero value), minimum length L (minimum value), and maximum length L (maximum value) of the sliding window, and set the initial value D (zero value) of the abnormal density threshold; During the process of the sliding window traversing the body temperature sampling sequence, each time the sliding window moves once, the current window density value is calculated based on the number of sampling points currently covered by the sliding window. The current window density value is the ratio of the number of candidate abnormal points in the sliding window to the current length of the sliding window. The current window density value is compared with the current value of the abnormal density threshold. When the current window density value is greater than the current value of the abnormal density threshold, the length of the sliding window is updated to the current length plus one step unit, but the updated length does not exceed the maximum value of the maximum length L. When the current window density value is less than or equal to the current value of the abnormal density threshold, the length of the sliding window is updated to the current length minus one step unit, but the updated length is not less than the minimum value of the minimum length L. After each update of the length of the sliding window, the current value of the abnormal density threshold is dynamically adjusted according to the updated sliding window length. The adjustment method is as follows: the current value of the abnormal density threshold is updated to the product of the initial value D zero of the abnormal density threshold and the adjustment number. The adjustment coefficient is the quotient obtained by dividing the initial length L zero of the sliding window by the updated sliding window length. Save the updated sliding window length and the updated anomaly density threshold for comparison operations during the next sliding window movement.

10. The bovine herd health status analysis system integrating biosensing and pattern visual recognition according to claim 9, characterized in that, The initial value D of the abnormal density threshold is obtained by pre-analyzing historical normal physiological parameter data.

Images