Cow video checking system and method for pasture management

Abstract

The invention provides a cattle video checking system and method for pasture management, and the system comprises an edge end and a cloud end, the edge end collects a cattle herd video stream through a camera, extracts cattle features including appearance, motion and posture features, and generates a local track fragment with a timestamp and spatial information based on an improved ByteTrack algorithm; and the cloud receives the features and the trajectory fragments and then carries out multi-modal feature fusion, constructs a complete cattle trajectory covering the pasture, implements spatio-temporal logic and multi-source data cross verification, and finally generates an inventory report containing cattle number and activity trajectory abstracts. According to the invention, non-contact, automatic and high-accuracy cattle checking is realized, and the efficiency and reliability of pasture management are improved.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of computer vision, Internet of Things and smart animal husbandry, specifically a video inventory system and method for cattle management in ranches. Background Technology

[0002] In the context of the large-scale development of animal husbandry, accurate inventory and behavioral monitoring of cattle are crucial for daily farm management. Traditional inventory mainly relies on manual counting or contact identification technologies such as RFID. The former is labor-intensive, inefficient, and easily affected by subjective factors, making it difficult to achieve high-frequency, large-scale accurate inventory. The latter, while enabling individual identification, requires attaching electronic tags to each cow, resulting in high deployment and maintenance costs. Furthermore, in scenarios with dense herds and frequent activity, it is prone to missed or misreading, and cannot provide continuous activity trajectories and behavioral analysis information for the cattle.

[0003] In recent years, computer vision-based automated monitoring technologies have been gradually applied to the livestock industry. However, existing video analysis methods still face significant challenges in complex pasture environments. For example, factors such as similar appearances of cattle, mutual occlusion, changes in lighting, and weather interference can easily lead to target tracking loss, trajectory breaks, or identity confusion, making it difficult to stably construct complete individual trajectories across multiple cameras and over long time series. Furthermore, most systems rely on only single visual features and lack effective fusion and reliability assessment of multi-dimensional features of cattle, resulting in shortcomings in inventory accuracy and system robustness, thus limiting their practical application in full-scene, automated pasture management. Summary of the Invention

[0004] In order to solve the technical problems mentioned in the background art, the present invention proposes a video inventory system and method for cattle for ranch management.

[0005] Therefore, the technical solution adopted by the present invention is as follows:

[0006] A video inventory system for cattle management in a ranch, the system comprising an edge terminal and a cloud terminal, including:

[0007] Data acquisition and processing module: The edge device acquires video streams of cattle based on preset cameras and preprocesses the video streams;

[0008] Feature fusion module: The edge device extracts cattle features based on the preprocessed video stream; the cattle features include appearance features, motion features and posture features; within a single camera video stream, a local trajectory segment containing timestamps and spatial information is output based on the cattle features;

[0009] The cattle features and local trajectory segments are uploaded to the cloud; the cloud performs feature fusion on the cattle features to obtain multimodal features;

[0010] Inventory module: When taking inventory of cattle, the cloud platform constructs cattle trajectories covering the pasture based on the multimodal features and local trajectory fragments, and performs integrity verification on the cattle trajectories. After passing the integrity verification, an inventory report is generated based on the cattle trajectories.

[0011] Furthermore, the extraction process of the appearance features is as follows:

[0012] The preprocessed video stream is input into a YOLOv8 model with added pixel-level segmentation branches to obtain detection boxes, detection box confidence, and segmentation masks;

[0013] The area within the segmentation mask is divided into three parts according to the preset proportions: head area, torso area, and leg area.

[0014] Within each region, color histogram features, texture features, and contour shape features are extracted and then stitched together to generate appearance features.

[0015] Furthermore, the process for extracting the pose features is as follows:

[0016] Based on the preset HRNet model and the detection box, the two-dimensional coordinates and confidence scores of each key point on the cow are obtained;

[0017] Based on the aforementioned key points, a system is constructed that includes ridge line features, head and neck posture features, limb proportion features, and joint angle features, and these features are then spliced ​​together to generate posture features.

[0018] Furthermore, the process of extracting the motion features is as follows:

[0019] The Farneback optical flow algorithm is used to sample optical flow vectors within the segmentation mask to obtain a set of optical flow vectors.

[0020] Motion velocity features, motion direction features, motion consistency features, and gait periodic features are extracted from the optical flow vector set and then spliced ​​to generate motion features.

[0021] Furthermore, the local trajectory segments are extracted based on the improved ByteTrack algorithm;

[0022] The improvements to the ByteTrack algorithm are as follows:

[0023] First, the improved ByteTrack algorithm features a proprietary two-stage association mechanism.

[0024] The first stage prioritizes matching with active tracking objects; the second stage associates the unmatched detection boxes from the first stage with tracking objects that are missing.

[0025] Secondly, the improved ByteTrack algorithm constructs a composite similarity model of location and features, calculated as follows:

[0026]

[0027] in, For test results With the tracked object similarity, For the test results, For test results Detection box and tracking object in The intersection-union ratio of the predicted bounding boxes. For test results Cattle characteristics and tracking objects Cosine similarity of historical average cattle characteristics and These are the position similarity weight coefficient and the feature similarity weight coefficient, respectively.

[0028] Thirdly, the improved ByteTrack algorithm is designed with a three-level state system of active, lost and terminated, and is equipped with a lost counter. When the tracked object has not been matched for a long time, it will be automatically marked as terminated and the complete trajectory will be output.

[0029] Fourthly, the improved ByteTrack algorithm adds a composite confidence evaluation model, as shown in the following formula:

[0030]

[0031] in, The confidence score for a local trajectory segment. The number of video frames contained in the local trajectory. The minimum reliable trajectory length threshold. This represents the standard deviation of cattle characteristics in a local trajectory segment over time. The characteristic standard deviation is the normalization parameter. It is a minimum value function.

[0032] Furthermore, during the generation of the local trajectory segment, the edge maintains a list of tracked objects, including a unique local ID, historical feature sequence, and

[0033] Historical bounding box sequence, tracking status, and loss counter.

[0034] Furthermore, the process of acquiring the multimodal features is as follows:

[0035] Calculate the reliability weights of the cattle features separately;

[0036] The reliability weights are normalized;

[0037] The multimodal features are obtained by weighted concatenation of the cattle features.

[0038] Furthermore, the steps for obtaining the cattle's trajectory are as follows:

[0039] Within a preset inventory period, the cosine similarity of the average multimodal features of the local trajectory segments from different cameras is calculated.

[0040] The local trajectories with a cosine similarity exceeding a preset similarity threshold are spliced ​​together to obtain the cattle trajectories, and each cattle is assigned a globally unique ID.

[0041] Furthermore, the integrity verification includes spatiotemporal logic verification and multi-source data cross-verification;

[0042] The inventory report is in list format and includes: the globally unique ID of each cow, the inventory period, the number of cows, and a summary of the cow's trajectory.

[0043] The bovine trajectory summary includes the coverage area, total activity duration, and key time period locations.

[0044] A method for video inventory of cattle for ranch management, comprising:

[0045] Step 1: This includes the edge device and the cloud. The edge device collects video streams of the cattle herd based on preset cameras and preprocesses the video streams.

[0046] Step 2: The edge device extracts cattle features based on the preprocessed video stream; the cattle features include appearance features, motion features, and posture features; within the single camera video stream, a local trajectory segment containing timestamps and spatial information is output based on the cattle features;

[0047] The cattle features and local trajectory segments are uploaded to the cloud; the cloud performs feature fusion on the cattle features to obtain multimodal features;

[0048] Step 3: During the cattle inventory, the cloud platform constructs cattle trajectories covering the pasture based on the multimodal features and local trajectory fragments, and performs integrity verification on the cattle trajectories. After passing the integrity verification, an inventory report is generated based on the cattle trajectories.

[0049] Compared with the prior art, the advantages of the present invention are as follows:

[0050] 1. By deploying cameras and visual analysis algorithms, this invention completely avoids the reliance on human labor and intrusive operations of traditional manual counting or contact identification methods, thereby reducing deployment and maintenance costs while improving inventory efficiency and the objectivity and accuracy of results.

[0051] 2. By fusing appearance, posture and motion multimodal features, and combining an improved multi-stage tracking association mechanism and confidence assessment, this invention effectively overcomes the problems of tracking loss or identity confusion caused by factors such as cattle occlusion, similar appearance, and changes in lighting, and can stably generate complete individual trajectories across cameras and time periods. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the system execution flow of the present invention;

[0054] Figure 2 This is a schematic diagram of the cattle feature structure of the present invention;

[0055] Figure 3 This is a schematic diagram of the inventory report structure of the present invention. Detailed Implementation

[0056] To achieve the above objectives, the present invention provides a video inventory system for cattle in ranch management. Please refer to the provided technical solution. Figure 1 ,include:

[0057] Data acquisition and processing module: The edge device acquires video streams of cattle based on preset cameras and preprocesses the video streams;

[0058] Multiple network cameras are deployed at the edge to cover key areas of the ranch, including feeding areas, watering areas, rest areas, and passageways;

[0059] Each camera captures a video stream at a frame rate of no less than 25 frames per second and a resolution of no less than 1920×1080 pixels to ensure that the detailed features of the cattle can be clearly captured.

[0060] There is an appropriate overlap area between the cameras, with the overlap rate set to 10%-20%, to facilitate subsequent cross-camera trajectory association;

[0061] Based on the geographical layout of the ranch, each camera was calibrated, and its three-dimensional coordinates, orientation angle, pitch angle, and field of view were recorded.

[0062] In this embodiment, the specific operations for video stream preprocessing are as follows:

[0063] First, image denoising is performed using a temporal filtering algorithm. The three consecutive frames are weighted and averaged with weight coefficients of 0.25, 0.5, and 0.25, respectively, to reduce the interference of random noise while preserving motion information.

[0064] Secondly, illumination normalization processing is performed. Since the illumination of the pasture environment varies greatly, an adaptive histogram equalization method is adopted to divide the image into 8×8 sub-blocks. Histogram equalization is performed independently on each sub-block, and then bilinear interpolation is used to smooth the boundaries of adjacent sub-blocks to obtain an image with uniform illumination.

[0065] For severe weather conditions such as rain and snow, the overall brightness and contrast of the image are detected. When the brightness standard deviation is less than 15 or the contrast is less than 0.3, the weather compensation module is activated to perform dehazing through the dark channel prior algorithm and appropriately improve the image contrast.

[0066] The preprocessed video frames are compressed in JPEG format with a quality factor of 85 to reduce data transmission while maintaining image quality.

[0067] Feature fusion module: The edge device extracts cattle features based on the preprocessed video stream; the cattle features include appearance features, motion features and posture features; within a single camera video stream, a local trajectory segment containing timestamps and spatial information is output based on the cattle features;

[0068] The cattle features and local trajectory segments are uploaded to the cloud; the cloud performs feature fusion on the cattle features to obtain multimodal features;

[0069] Appearance features, motion features, and posture features are all in vector form.

[0070] The specific steps of feature extraction are as follows; please refer to [link / reference]. Figure 2 :

[0071] The process of extracting appearance features is as follows:

[0072] The preprocessed video stream is input into the YOLOv8 model with pixel-level segmentation branches to output bounding boxes, confidence scores, and segmentation masks.

[0073] The detection box is specifically in the format of (x, y, w, h), which marks the position of the cow in each frame of the image, where x and y are the center coordinates of the detection box, and w and h are the width and height of the detection box;

[0074] The area within the segmentation mask is divided into three parts according to a preset ratio: head area, torso area, and leg area. For example, the ratio is: head occupies the front 20%, torso occupies the middle 60%, and legs occupies the back 20%.

[0075] Within each region, color histogram features, texture features, and contour shape features are extracted and then stitched together to generate appearance features;

[0076] The color histogram features are statistically analyzed in the HSV color space. The H channel is divided into 18 intervals, and the S and V channels are each divided into 8 intervals, forming a color histogram vector of 18×8×8=1152 dimensions. To reduce the dimensionality, principal component analysis is used to compress it to 64 dimensions, retaining more than 95% of the variance information.

[0077] Texture features were extracted using the Local Binary Pattern Operator. The region was divided into a 4×4 grid, and rotation-invariant uniform LBP patterns were calculated within each grid to obtain statistical histograms of 59 basic patterns, totaling 16×59=944 dimensions. Principal component analysis was then used to compress the histograms to 32 dimensions.

[0078] The contour shape features are represented using Fourier descriptors. The system extracts the boundary contour of the segmentation mask, performs a discrete Fourier transform on the contour coordinate sequence, and takes the amplitude and phase of the first 16 low-frequency coefficients to form a 32-dimensional shape description vector.

[0079] The process of extracting pose features is as follows:

[0080] Based on the preset HRNet model, the system outputs the two-dimensional coordinates and confidence scores of each key point on the cow. The system only retains key points with a confidence score greater than 0.5 for subsequent processing.

[0081] In this embodiment, key points include: the tip of the nose, both eyes, the base of both ears, the neck, the shoulders, the midpoint of the spine, the buttocks, the base of the tail, and the shoulder joints, elbow joints, wrist joints, hip joints, knee joints, and ankle joints of the limbs.

[0082] Based on the detected key points, a posture feature is constructed that includes ridge line features, head and neck posture features, limb proportion features, and joint angle features;

[0083] The specific steps for obtaining ridge features are as follows:

[0084] The spine line connecting the key anatomical points such as the neck, shoulder, midpoint of the spine, buttocks, and tail root is constructed into a cubic spline curve.

[0085] The curvature of the curve at each key point is calculated, and the curvature of K uniformly distributed key points is selected to form the ridge feature. In this embodiment, K is 10, which can maintain low computational complexity while ensuring feature distinguishability.

[0086] Calculate the average vector from the midpoint between the nose tip and the midpoint between the eyes to the key points on the neck, and the vector from the neck to the shoulder. Use the angle between the two vectors as the head and neck pose feature, and normalize the angle to [0, 1].

[0087] The specific steps for obtaining limb proportion features are as follows:

[0088] Calculate the Euclidean distance between key points of the limbs, including the length of the forelimb, the length of the hindlimb, the length of the trunk, and the width of the trunk, for a total of four length measurements;

[0089] Normalizing the forelimb length, hindlimb length, and trunk width by dividing by the trunk length yields the limb proportions, which are the ratio of forelimb to trunk length, the ratio of hindlimb to trunk length, and the ratio of trunk width to length, respectively.

[0090] The joint angles of the limbs are calculated using the law of cosines as joint angle characteristics, including the shoulder, elbow and wrist angles of the forelimbs, and the hip, knee and ankle angles of the hindlimbs.

[0091] The process of obtaining motion features is as follows:

[0092] The Farneback optical flow algorithm is used to sample optical flow vectors within the segmentation mask region of the cattle to obtain a set of optical flow vectors;

[0093] In this embodiment, uniform grid sampling is used, with a grid spacing of 8 pixels.

[0094] Based on the optical flow vector set, motion velocity features, motion direction features, motion consistency features, and gait periodic features are extracted and concatenated into motion features;

[0095] The overall movement speed of the cattle is represented by calculating the average magnitude of all optical flow vectors, as shown in the following formula:

[0096]

[0097] Where N is the number of optical flow vector sampling points within the segmentation mask region. and The first The components of the optical flow vector in the x and y directions at each sampling point The average speed of the cattle, expressed in pixels per frame;

[0098] The motion direction feature is obtained by calculating the principal direction of the optical flow vector. The specific operation is as follows:

[0099] By statistically analyzing the histogram of optical flow direction, dividing 360 degrees into 36 intervals of 10 degrees each, and finding the peak interval in the histogram, the center angle of the interval is the direction of motion.

[0100] The standard deviation of the optical flow vector direction is used as a motion consistency feature; a smaller standard deviation indicates that the movement direction of each part of the cattle is consistent, while a larger standard deviation indicates that there is complex local movement. The calculation of the standard deviation of the direction takes into account the periodicity of the angle and uses a cyclic statistical method.

[0101] The specific steps for obtaining gait periodic features are as follows:

[0102] A time-series analysis of motion velocity was performed on 30 consecutive frames. The main frequency component was extracted using Fast Fourier Transform, and the amplitude and frequency value of the main frequency were taken as gait periodic features.

[0103] The improved ByteTrack algorithm is used to extract local trajectory segments from a single camera. The specific improvements are as follows:

[0104] First, the improved ByteTrack algorithm features a proprietary two-stage association mechanism.

[0105] The first phase prioritizes matching with active tracking targets to ensure the stability of cattle tracking in typical ranch scenarios.

[0106] The second phase will specifically target the detection boxes that were not matched in the first phase and associate them with the tracked objects in the lost state, so as to recall the trajectory of cattle that are blocked by fences, companions, or moving across areas, adapting to the special tracking scenarios of ranches.

[0107] If a detection result that still does not match after two-stage association is initialized as a new tracking object and a new local ID is assigned if the confidence of its detection box is higher than 0.7, it is then assigned a new local ID.

[0108] Secondly, the improved ByteTrack algorithm constructs a composite similarity model of location and features, calculated as follows:

[0109]

[0110] in, For test results With the tracked object similarity, The detection results include the bounding box, cattle features, and confidence level. For test results Detection box and tracking object in The intersection-union ratio (IUU) of the predicted bounding boxes is calculated as the ratio of the pixel overlap between the detected boxes and the predicted bounding boxes to the total area. For test results Cattle characteristics and tracking objects Cosine similarity of historical average cattle characteristics and These are the position similarity weight coefficient and the feature similarity weight coefficient, respectively, which were obtained through statistical optimization of experimental data, with typical values ​​of 0.4 and 0.6.

[0111] Thirdly, the improved ByteTrack algorithm is designed with a three-level state system of active, lost, and terminated, and is equipped with a lost counter. When the tracked object has not been matched for a long time, it will be automatically marked as terminated and the complete trajectory will be output to avoid unnecessary computing power consumption.

[0112] Fourthly, the improved ByteTrack algorithm adds a composite confidence evaluation model, as shown in the following formula:

[0113]

[0114] in, The confidence score for a local trajectory segment. The length of the trajectory is the number of video frames contained within a local trajectory, representing the duration of the trajectory. This is a minimum reliable trajectory length threshold; trajectories shorter than this threshold are considered invalid. This is a preset empirical threshold for cattle tracking scenarios in pastures. This represents the standard deviation of cattle characteristics in a local trajectory segment over time. The characteristic standard deviation normalization parameter is used to... Mapped to the [0, 1] interval, this value is derived from historical data tracking of cattle on the ranch, and is typically set to 0.3. It is a minimum value function.

[0115] The confidence score is used to evaluate the reliability of local trajectory segments, ensuring that the local trajectories uploaded to the cloud have sufficient biometric identification capabilities, and providing a reliable data foundation for subsequent cross-camera trajectory association.

[0116] During the generation of local trajectory segments, a list of tracking objects is maintained at the edge, including:

[0117] Unique Local ID: A unique identifier assigned to each cow tracked within the field of view of a single camera, valid only within the local tracking range of that camera;

[0118] Historical feature sequence: A temporal set of appearance, pose, and motion feature vectors extracted from the edge of the tracked object in past tracking frames;

[0119] Historical bounding box sequence: A temporal record of the coordinates of the cow detection boxes in previously successfully matched frames, with data sourced from the effective detection output of the YOLOv8 model. Based on this sequence, the algorithm can predict the bounding box position of the current frame using models such as Kalman filtering, providing a positional reference for calculating the intersection-over-union (IoU).

[0120] Tracking Status: Marks the real-time tracking status of the tracked object, divided into three categories:

[0121] Active: In the current frame or the most recent frame, the object successfully matched the detection result, and the tracking link remains uninterrupted;

[0122] Loss: No corresponding detection result was found in the current frame, but the loss counter did not reach the threshold, so it was determined to be a temporary "loss of connection";

[0123] Termination: When the loss counter reaches the threshold of 30 frames, it is determined that the object has left the camera's field of view, the tracking is terminated and the complete local trajectory is output.

[0124] Loss Counter: A counting tool that counts the number of frames in which the tracked object is in a lost state. When a frame of the tracked object does not match the detection result, the counter is incremented by 1; if a subsequent rematch is successful, the counter is reset to zero; once the count reaches 30 frames, the tracking state is switched to termination to avoid invalid tracking consuming computing power.

[0125] In this embodiment, the first stage uses the Hungarian algorithm to match detection boxes with a confidence level ≥ 0.6 with active tracking objects, with a similarity threshold of 0.5, to ensure tracking stability in normal scenarios. The second stage specifically associates the unmatched detection boxes with a confidence level ≥ 0.6 from the first stage with tracking objects in a lost state: on the one hand, the similarity threshold is lowered to 0.3, and on the other hand... The accuracy was improved from 0.6 to 0.7, enabling precise recall of lost targets by leveraging the unique appearance and posture characteristics of cattle.

[0126] Upload local trajectory segments with confidence scores greater than or equal to the preset upload threshold to the cloud;

[0127] In this embodiment, the upload threshold is set to 0.5, based on the following:

[0128] According to the confidence calculation formula, when the trajectory length L≥10 frames and the feature stability is good, C≥0.5. 10 frames correspond to an observation duration of about 0.4 seconds, which can basically confirm that the detected object is a real cow rather than a short-term false detection.

[0129] Experimental statistics show that among trajectory segments with a confidence level below 0.5, approximately 70% are false detections, occlusion-induced spurious trajectories, or extremely brief detections. The specific steps for fusing cattle features based on an attention mechanism are as follows:

[0130] Calculate the reliability weights of cattle characteristics, including:

[0131] Reliability weight of appearance features:

[0132]

[0133] in, The occlusion penalty coefficient was obtained through experimental optimization. The occlusion ratio is obtained by evaluating the integrity of the segmented mask, and its value ranges from [0, 1], where 0 represents no occlusion and 1 represents complete occlusion. A typical value is 2.0. This is the illumination penalty coefficient, obtained through experimental optimization, with a typical value of 1.5. The illumination anomaly level is assessed through the local variance of image brightness, with a value range of [0, 1], where 0 indicates normal illumination and 1 indicates severe illumination anomaly. It is a natural exponential function;

[0134] Reliability weights of pose features:

[0135]

[0136] in, For the first The confidence scores for each keypoint are output by the HRNet model, with values ​​ranging from [0, 1]. The number of key points;

[0137] Reliability weights of motion features:

[0138]

[0139] in, To adjust the parameters and control the rate at which reliability deteriorates with speed deviation, a typical value is 0.02. The optimal movement speed is measured in pixels per frame. At this speed, the motion features are most stable and reliable. This speed is based on statistics of the normal walking speed of cattle.

[0140] The reliability weights are normalized using the following formula:

[0141]

[0142] in, These correspond to three characteristics: appearance, posture, and movement.

[0143] The normalized weights satisfy ;

[0144] Weighted concatenation of cattle features yields multimodal features:

[0145]

[0146] in, , and These represent the normalized appearance, posture, and motion features, respectively, with the semicolon indicating a vector concatenation operation.

[0147] In this embodiment, to reduce the complexity of subsequent calculations, an autoencoder is used to compress the multimodal features to 128 dimensions.

[0148] Inventory module: When taking inventory of cattle, the cloud platform constructs cattle trajectories covering the pasture based on the multimodal features and local trajectory fragments, and performs integrity verification on the cattle trajectories. After passing the integrity verification, an inventory report is generated based on the cattle trajectories.

[0149] The steps to obtain the cattle's trajectory are as follows:

[0150] Calculate the cosine similarity of the average multimodal features of local trajectory segments from different cameras during the inventory period. The inventory period is a settable time period, such as 8:00-18:00 on the same day.

[0151] Local trajectories with cosine similarity exceeding a preset similarity threshold are spliced ​​together to obtain complete cow trajectories, and each cow is assigned a globally unique ID.

[0152] In this embodiment, the similarity threshold is set to 0.8;

[0153] Integrity verification includes spatiotemporal logic verification and multi-source data cross-verification. For example, the time difference between adjacent segments of the spliced ​​trajectory should be ≤5 minutes, and the total coverage time of adult cattle trajectories during the inventory period should be ≥6 hours.

[0154] Anomaly classification is performed on the abnormal verification results:

[0155] For minor anomalies, such as time breaks of 1-5 minutes or speeds slightly exceeding reasonable limits, the system will automatically re-trigger local trajectory matching without manual intervention.

[0156] For moderate anomalies, such as spatial jumps: mark the abnormal trajectory segments, generate a list of areas to be reviewed, and prompt manual review of the corresponding time period video;

[0157] For severe anomalies, such as trajectory crossing the boundary or speed exceeding 5m / s: directly remove the abnormal trajectory segment, record the cause of the anomaly (such as weather or equipment failure), and supplement the video analysis of the area.

[0158] The inventory report is in list format and includes the inventory period, number of cattle, and a summary of cattle movement. Please refer to [link / reference]. Figure 3 ;

[0159] The number of cattle is the same as the number of cattle tracks.

[0160] The cattle trajectory summary contains core activity information of the cattle, including: coverage area, total activity duration and key time period location. For example, a certain cow passed through the rest area, passage, feeding area and rest area in sequence during the inventory period, with a total activity duration of 9.2 hours. It was in the rest area from 8:00 to 8:30 and in the passage from 8:35 to 8:50, etc.

[0161] The current inventory result is compared with the adjacent inventory results. If the number of cattle is inconsistent, an abnormal warning is triggered.

[0162] A method for video inventory of cattle for ranch management, comprising:

[0163] Step 1: This includes the edge device and the cloud. The edge device collects video streams of the cattle herd based on preset cameras and preprocesses the video streams.

[0164] Step 2: The edge device extracts cattle features based on the preprocessed video stream; the cattle features include appearance features, motion features, and posture features; within the single camera video stream, a local trajectory segment containing timestamps and spatial information is output based on the cattle features;

[0165] The cattle features and local trajectory segments are uploaded to the cloud; the cloud performs feature fusion on the cattle features to obtain multimodal features;

[0166] Step 3: During the cattle inventory, the cloud platform constructs cattle trajectories covering the pasture based on the multimodal features and local trajectory fragments, and performs integrity verification on the cattle trajectories. After passing the integrity verification, an inventory report is generated based on the cattle trajectories.

[0167] This invention proposes a video inventory system and method for cattle management in ranches. By deploying a network of key cameras on the ranch, the system collects and analyzes cattle video streams in real time at the edge, extracting multimodal features such as appearance, posture, and movement of each cow, and generating local trajectory segments using an improved tracking algorithm. Subsequently, cross-camera trajectory association, multi-source data verification, and fusion are performed in the cloud to automatically construct a continuous and complete individual activity trajectory covering the entire ranch. This solves the problems of low efficiency, error-proneness, and inability to provide behavioral trajectories in traditional manual or contact-based inventory methods, achieving non-contact, automated, and highly reliable cattle inventory and behavior monitoring.

[0168] In summary, this invention achieves highly robust continuous individual tracking in complex pasture environments by integrating multimodal cattle features with an improved tracking algorithm, effectively solving the problems of low efficiency and susceptibility to interference in traditional methods. At the same time, by leveraging cloud-based trajectory fusion and multi-source verification mechanisms, the accuracy and reliability of inventory results are significantly improved. Finally, a structured report containing behavioral trajectories is output in an automated and contactless manner, providing efficient and reliable technical support for refined pasture management and decision-making.

[0169] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video inventory system for cattle in ranch management, the system comprising an edge terminal and a cloud terminal, characterized in that, include: Data acquisition and processing module: The edge device acquires video streams of cattle based on preset cameras and preprocesses the video streams; Feature fusion module: The edge end extracts cattle features based on the preprocessed video stream; The characteristics of the cattle include appearance, movement, and posture. Within a single-camera video stream, a local trajectory segment containing timestamps and spatial information is output based on the cattle characteristics. The cattle features and local trajectory segments are uploaded to the cloud; the cloud performs feature fusion on the cattle features to obtain multimodal features; Inventory module: When taking inventory of cattle, the cloud platform constructs cattle trajectories covering the pasture based on the multimodal features and local trajectory fragments, and performs integrity verification on the cattle trajectories. After passing the integrity verification, an inventory report is generated based on the cattle trajectories.

2. The system according to claim 1, characterized in that, The extraction process of the appearance features is as follows: The preprocessed video stream is input into a YOLOv8 model with added pixel-level segmentation branches to obtain detection boxes, detection box confidence, and segmentation masks; The area within the segmentation mask is divided into three parts according to the preset proportions: head area, torso area, and leg area. Within each region, color histogram features, texture features, and contour shape features are extracted and then stitched together to generate appearance features.

3. The system according to claim 2, characterized in that, The process of extracting the pose features is as follows: Based on the preset HRNet model and the detection box, the two-dimensional coordinates and confidence scores of each key point on the cow are obtained; Based on the aforementioned key points, a system is constructed that includes ridge line features, head and neck posture features, limb proportion features, and joint angle features, and these features are then spliced ​​together to generate posture features.

4. The system according to claim 3, characterized in that, The process of extracting the motion features is as follows: The Farneback optical flow algorithm is used to sample optical flow vectors within the segmentation mask to obtain a set of optical flow vectors. Motion velocity features, motion direction features, motion consistency features, and gait periodic features are extracted from the optical flow vector set and then spliced ​​to generate motion features.

5. The system according to claim 4, characterized in that, The local trajectory segments are extracted based on the improved ByteTrack algorithm; The improvements to the ByteTrack algorithm are as follows: First, the improved ByteTrack algorithm features a proprietary two-stage association mechanism. The first stage prioritizes matching with active tracking objects; the second stage associates the unmatched detection boxes from the first stage with tracking objects that are missing. Secondly, the improved ByteTrack algorithm constructs a composite similarity model of location and features, calculated as follows: in, For test results With the tracked object similarity, For the test results, For test results Detection box and tracking object in The intersection-union ratio of the predicted bounding boxes. For test results Cattle characteristics and tracking objects Cosine similarity of historical average cattle characteristics and These are the position similarity weight coefficient and the feature similarity weight coefficient, respectively. Thirdly, the improved ByteTrack algorithm is designed with a three-level state system of active, lost and terminated, and is equipped with a lost counter. When the tracked object has not been matched for a long time, it will be automatically marked as terminated and the complete trajectory will be output. Fourthly, the improved ByteTrack algorithm adds a composite confidence evaluation model, as shown in the following formula: in, The confidence score for a local trajectory segment. The number of video frames contained in the local trajectory. The minimum reliable trajectory length threshold. This represents the standard deviation of cattle characteristics in a local trajectory segment over time. The characteristic standard deviation is the normalization parameter. It is a minimum value function.

6. The system according to claim 1, characterized in that, During the generation of the local trajectory segment, the edge maintains a list of tracked objects, including a unique local ID, historical feature sequence, and Historical bounding box sequence, tracking status, and loss counter.

7. The system according to claim 1, characterized in that, The process of obtaining the multimodal features is as follows: Calculate the reliability weights of the cattle features separately; The reliability weights are normalized; The multimodal features are obtained by weighted concatenation of the cattle features.

8. The system according to claim 1, characterized in that, The steps for obtaining the cattle's trajectory are as follows: Within a preset inventory period, the cosine similarity of the average multimodal features of the local trajectory segments from different cameras is calculated. The local trajectories with a cosine similarity exceeding a preset similarity threshold are spliced ​​together to obtain the cattle trajectories, and each cattle is assigned a globally unique ID.

9. The system according to claim 1, characterized in that, The integrity verification includes spatiotemporal logic verification and multi-source data cross-verification; The inventory report is in list format and includes: the globally unique ID of each cow, the inventory period, the number of cows, and a summary of the cow's trajectory. The bovine trajectory summary includes the coverage area, total activity duration, and key time period locations.

10. A method for video inventory of cattle for ranch management, implemented using the system described in any one of claims 1-9, characterized in that, include: Step 1: This includes the edge device and the cloud. The edge device collects video streams of the cattle herd based on preset cameras and preprocesses the video streams. Step 2: The edge end extracts cattle features based on the preprocessed video stream; The characteristics of the cattle include appearance, movement, and posture. Within a single-camera video stream, a local trajectory segment containing timestamps and spatial information is output based on the cattle characteristics. The cattle features and local trajectory segments are uploaded to the cloud; the cloud performs feature fusion on the cattle features to obtain multimodal features; Step 3: During the cattle inventory, the cloud platform constructs cattle trajectories covering the pasture based on the multimodal features and local trajectory fragments, and performs integrity verification on the cattle trajectories. After passing the integrity verification, an inventory report is generated based on the cattle trajectories.

Images