Sheep drinking water behavior intelligent recognition method and device based on space-time feature fusion

Abstract

This invention discloses a method and device for intelligent recognition of sheep drinking behavior based on spatiotemporal feature fusion. The method acquires left and right views of a flock of sheep using a binocular vision imaging system, and uses a binocular vision image analysis algorithm to obtain the individual sheep's contour information, three-dimensional coordinates, and pixel coordinates, tracking and recording movement trajectory data. A spatiotemporal feature fusion model is constructed, combining drinking behavior parameters such as the distance between the sheep's mouth and the water surface, the neck bending angle, and the frequency of head swaying to extract and fuse spatial and temporal behavioral features, generating a spatiotemporal fusion feature vector. A classifier then determines whether an individual sheep is in a drinking behavior state. The device includes an image acquisition unit, an image analysis unit, a trajectory recording unit, a feature extraction unit, a feature fusion unit, and a behavior recognition unit. These units work collaboratively to achieve high-precision automated recognition of sheep drinking behavior, providing effective technical support for intelligent farming.

Description

Technical Field

[0001] This invention relates to the field of sheep drinking behavior recognition, and more particularly to a method and apparatus for intelligent recognition of sheep drinking behavior based on spatiotemporal feature fusion. Background Technology

[0002] As the livestock industry develops towards large-scale and intelligent operations, precise monitoring of sheep behavior has become a crucial link in improving breeding efficiency and ensuring the health of sheep flocks. Sheep drinking behavior, as an important indicator reflecting their physiological state, can directly reflect their health level and environmental adaptation. Accurately identifying sheep drinking behavior helps to promptly detect abnormalities in the flock and achieve scientific breeding. However, current sheep drinking behavior identification technologies have many limitations.

[0003] Existing technologies face two major challenges. First, while traditional manual observation methods can obtain information about sheep drinking behavior to some extent, they are extremely labor-intensive and heavily influenced by the observer's subjectivity, failing to meet the needs of real-time, continuous monitoring of large numbers of sheep in large-scale farming scenarios. Second, some automated recognition technologies based on single vision or simple algorithms perform poorly in complex farming environments. These technologies often fail to effectively integrate the spatial and temporal characteristics of sheep drinking behavior. When sheep posture changes or occlusion occurs, it is difficult to accurately extract key behavioral parameters, leading to a significant decrease in recognition accuracy and failing to provide a reliable basis for farming decisions. Summary of the Invention

[0004] In order to overcome the shortcomings and deficiencies of the existing technology, the present invention provides a method and device for intelligent recognition of sheep drinking behavior based on spatiotemporal feature fusion.

[0005] The technical solution adopted in this invention is an intelligent recognition method for sheep drinking behavior based on spatiotemporal feature fusion, comprising the following steps:

[0006] Step S1: Use a binocular vision imaging system to acquire images of the sheep's activity area, and obtain the left and right views of the sheep at the same time. The left and right views constitute a binocular vision image pair.

[0007] Step S2: The binocular vision image pair is processed by a binocular vision image analysis algorithm to extract the contour information of the individual sheep in the image, and the three-dimensional coordinates of the individual sheep in space are determined based on the contour information. At the same time, the pixel coordinates of the individual sheep in the image are obtained.

[0008] Step S3: Track and record the three-dimensional coordinates and pixel coordinates of the sheep in each frame of the image according to the time series, and generate the spatial motion trajectory data of the sheep and the two-dimensional motion trajectory data in the image.

[0009] Step S4: Construct a behavior spatiotemporal feature fusion model. The behavior spatiotemporal feature fusion model includes a spatial feature extraction module and a temporal feature extraction module. Input the three-dimensional coordinates and two-dimensional motion trajectory data of the sheep individual into the spatial feature extraction module and the temporal feature extraction module respectively to extract the spatial behavior features and temporal behavior features of the sheep individual.

[0010] Step S5: Based on various parameters of sheep drinking behavior, including but not limited to the distance between the sheep's mouth and the water surface, the sheep's neck bending angle, and the sheep's head swinging frequency, the spatial and temporal behavioral features are fused to obtain a spatiotemporal fusion feature vector of individual sheep behavior.

[0011] Step S6: Using a pre-trained classifier, classify and judge the spatiotemporal fusion feature vector to identify whether an individual sheep is in a drinking behavior state.

[0012] Furthermore, in the aforementioned spatiotemporal feature fusion model, the spatial feature extraction module uses the following formula to extract features from the three-dimensional coordinates of an individual sheep: ,in, This represents the extracted spatial behavior feature vector. For individual sheep in continuous The number of three-dimensional coordinates in a frame image For the first Weighting coefficients for the three-dimensional coordinates of a frame image. It is a three-dimensional coordinate feature mapping function constructed based on sheep drinking behavior parameters. The sheep individuals were in the 1st month of the 2nd month. The three-dimensional coordinates in the frame image, the feature mapping function The study comprehensively considers the influence of the distance between the sheep's snout and the water surface on the coordinate weights, as well as the manifestation of the sheep's neck bending angle in spatial coordinate changes.

[0013] Furthermore, in the aforementioned spatiotemporal feature fusion model, the time feature extraction module processes the two-dimensional motion trajectory data of an individual sheep using the following formula: ,in, This represents the extracted time-based behavioral feature vector. The adjustment parameters for time feature extraction, The time interval between two adjacent frames. This represents the change in two-dimensional pixel coordinates of an individual sheep between two adjacent image frames. It is a time feature calculation function designed based on the sheep's head swaying frequency. By calculating the time interval and pixel coordinate changes, it obtains features related to the time characteristics of sheep drinking behavior.

[0014] Furthermore, in step S5, the process of fusing spatial behavior features and temporal behavior features adopts the following formula: ,in, This is the fused spatiotemporal fusion feature vector. and The fusion weights are spatial and temporal behavioral features, respectively. By adjusting these two weights, the fused feature vector can more accurately reflect the spatiotemporal characteristics of sheep drinking behavior. The determination of the fusion weights is based on the importance of various parameters of sheep drinking behavior in the spatial and temporal dimensions.

[0015] Furthermore, in step S2, when determining the three-dimensional coordinates of an individual sheep, the binocular vision image analysis algorithm calculates the disparity of corresponding points of the individual sheep in the left and right views, and establishes the relationship between disparity and depth by combining the calibration parameters of the binocular vision imaging system, thereby obtaining the three-dimensional coordinates of the individual sheep in space. In addition, when extracting the contour information of the individual sheep, an edge detection and region growing method is adopted, and the accuracy of contour extraction is optimized based on the possible changes in body posture caused by the sheep's drinking behavior.

[0016] Furthermore, in step S4, before inputting the data into the spatiotemporal feature fusion model, the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep are normalized. The normalization formula is as follows: ,in, For the normalized data, The original data, and These are the maximum and minimum values ​​in the original data, respectively. Through this normalization process, the relevant data of sheep drinking behavior parameters at different scales are processed in the behavior spatiotemporal feature fusion model.

[0017] Furthermore, in step S3, the three-dimensional coordinates and pixel coordinates of individual sheep are tracked and recorded according to the time series, specifically including:

[0018] Step 3.1: Mark and store the three-dimensional coordinates and pixel coordinates of the individual sheep in the first frame image to establish an initial coordinate record;

[0019] Step 3.2: In each subsequent frame, the sheep in the current frame is matched with the sheep in the previous frame using an image matching algorithm to obtain the three-dimensional coordinates and pixel coordinates of the corresponding sheep.

[0020] Step 3.3: Associate the matched coordinates of the sheep in the current frame with the stored coordinate sequence to update the sheep's motion trajectory data;

[0021] Step 3.4: Determine if the end of the image sequence has been reached. If not, return to sub-step two to continue processing the next frame. If the end has been reached, the recording of the sheep's individual movement trajectory data is complete.

[0022] Further, in step S4, the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep are input into the spatial feature extraction module and temporal feature extraction module of the behavior spatiotemporal feature fusion model, respectively, specifically including:

[0023] Step 4.1: Perform feature filtering on the three-dimensional coordinate data of individual sheep, and select key coordinate dimensions that are related to the spatial characteristics of sheep drinking behavior;

[0024] Step 4.2: Convert the format of the filtered 3D coordinate data to meet the input requirements of the spatial feature extraction module;

[0025] Step 4.3: Perform time series segmentation processing on the two-dimensional motion trajectory data of individual sheep, dividing the continuous trajectory data into multiple time segments;

[0026] Step 4.4: Input the segmented time data into the time feature extraction module for processing.

[0027] Furthermore, in step S5, spatial and temporal behavioral features are fused based on various parameters of the sheep's drinking behavior, specifically including:

[0028] Step S5.1: Establish a correlation table between sheep drinking behavior parameters and spatial and temporal behavioral characteristics, and clarify how each parameter is reflected in the characteristics;

[0029] Step S5.2: Based on the aforementioned association table, perform weighted adjustments on the spatial and temporal behavioral features to highlight the features closely related to sheep drinking behavior;

[0030] Step S5.3: Combine the weighted spatial behavior features and temporal behavior features to form preliminary fusion features;

[0031] Step S5.4: Perform dimensionality reduction on the preliminary fusion features to remove redundant information and obtain the final spatiotemporal fusion feature vector.

[0032] A smart recognition device for sheep drinking behavior based on spatiotemporal feature fusion includes:

[0033] The image acquisition unit is equipped with a binocular vision imaging system to acquire images of the sheep's activity area and obtain binocular vision image pairs.

[0034] An image analysis unit, connected to the image acquisition unit, is used to process the binocular vision image pair using a binocular vision image analysis algorithm to extract the outline information of the individual sheep and determine the three-dimensional coordinates and pixel coordinates of the individual sheep.

[0035] The trajectory recording unit, connected to the image analysis unit, is used to track and record the three-dimensional coordinates and pixel coordinates of an individual sheep according to a time series, and generate motion trajectory data.

[0036] The feature extraction unit, connected to the trajectory recording unit, is used to construct a behavior spatiotemporal feature fusion model, and inputs the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep into the spatial feature extraction module and temporal feature extraction module of the model respectively to extract spatial behavior features and temporal behavior features;

[0037] The feature fusion unit, connected to the feature extraction unit, is used to fuse the spatial behavior features and temporal behavior features based on various parameters of the sheep's drinking behavior to obtain a spatiotemporal fusion feature vector.

[0038] The behavior recognition unit, connected to the feature fusion unit, is used to classify and judge the spatiotemporal fusion feature vector through a pre-trained classifier to identify whether an individual sheep is in a drinking behavior state.

[0039] Beneficial Effects: This invention proposes an intelligent recognition method and device for sheep drinking behavior based on spatiotemporal feature fusion. The device utilizes an image acquisition unit equipped with a binocular vision imaging system to automatically acquire images of the sheep flock. Combined with an image analysis unit and a trajectory recording unit, it achieves unmanned, continuous monitoring, replacing manual observation and recording of each sheep, significantly reducing manpower and time consumption. Existing automation technologies struggle to handle complex farming scenarios and suffer from poor recognition accuracy. This method constructs a spatiotemporal feature fusion model to deeply analyze sheep drinking behavior characteristics from both spatial and temporal dimensions. Spatially, the binocular vision image analysis algorithm determines the three-dimensional coordinates of individual sheep by calculating disparity, and uses a combination of edge detection and region growing to accurately extract the contour information of sheep due to changes in drinking posture. Temporally, it analyzes the two-dimensional motion trajectory data of individual sheep based on parameters such as the frequency of head swaying. Based on key drinking behavior parameters such as the distance between the sheep's mouth and the water surface and the angle of neck bending, the extracted spatial and temporal behavioral features are deeply integrated to form a spatiotemporal fusion feature vector that can comprehensively reflect the characteristics of sheep's drinking behavior. Finally, a pre-trained classifier is used for accurate judgment, which effectively solves the problem of recognition errors caused by insufficient feature extraction and inaccurate fusion in complex scenarios, and provides efficient and reliable technical support for intelligent farming. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method steps of the present invention;

[0041] Figure 2 This is a diagram showing the unit composition of the device of the present invention. Detailed Implementation

[0042] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] like Figure 1 As shown, the intelligent recognition method for sheep drinking behavior based on spatiotemporal feature fusion includes the following steps:

[0044] Step S1: Use a binocular vision imaging system to acquire images of the sheep's activity area, and obtain the left and right views of the sheep at the same time. The left and right views constitute a binocular vision image pair.

[0045] Specifically, in this step, a binocular vision imaging system is used to acquire images of the sheep's activity area. The core objective is to obtain the left and right views of the sheep at the same moment, which together constitute a binocular vision image pair. The binocular vision imaging system simulates how human eyes observe objects, consisting of two cameras positioned at different locations. By setting a reasonable baseline distance (i.e., the horizontal distance between the two cameras), typically between 10-30 centimeters, it ensures effective acquisition of the object's stereoscopic information. By simultaneously triggering both cameras to capture images within a very short time interval (usually less than 0.1 seconds), the activity scene of the sheep is captured, ensuring temporal consistency between the left and right views, thus providing a reliable data foundation for subsequent 3D information calculations.

[0046] This step provides the raw data for the entire intelligent recognition method of sheep drinking behavior. The binocular vision image pairs contain rich information such as the sheep's position and posture in space, serving as a crucial basis for subsequent sheep contour extraction, 3D coordinate calculation, and behavioral feature analysis. Only by acquiring high-quality, accurate binocular vision image pairs can subsequent steps accurately recognize sheep drinking behavior. In terms of implementation, the binocular vision imaging system is typically installed above or to the side of the sheep's drinking area, ensuring the camera's field of view completely covers the sheep's activity area. Attention should be paid to the installation height and angle; generally, the height is between 1.5 and 3 meters, and the angle should be such that it can capture the sheep vertically or at approximately 45 degrees to avoid image information loss or distortion due to viewing angle issues.

[0047] Step S2: The binocular vision image pair is processed by a binocular vision image analysis algorithm to extract the contour information of the individual sheep in the image, and the three-dimensional coordinates of the individual sheep in space are determined based on the contour information. At the same time, the pixel coordinates of the individual sheep in the image are obtained.

[0048] Specifically, the acquired binocular vision image pairs are processed using a binocular vision image analysis algorithm. This step has two key tasks: extracting the contour information of individual sheep in the image and determining their three-dimensional coordinates in space, and simultaneously obtaining their pixel coordinates within the image. The binocular vision image analysis algorithm first preprocesses the left and right views to enhance image contrast and clarity for better feature extraction. Then, using an edge detection algorithm, such as the Canny edge detection algorithm, by setting appropriate high and low thresholds (generally, the high threshold is between 150-250, and the low threshold is 1 / 3-1 / 2 of the high threshold), the edge contours of the sheep in the image are detected. Finally, combined with a region growing algorithm, pixels within the edge contour are clustered based on their grayscale, color, and other features, thus completely extracting the sheep's contour information.

[0049] When determining the 3D and pixel coordinates of an individual sheep, based on the parallax principle of binocular vision, the parallax of corresponding points of the sheep in the left and right views is calculated. Combined with intrinsic parameters (such as focal length and principal point coordinates) and extrinsic parameters (such as rotation matrix and translation vector) obtained from prior calibration of the binocular vision imaging system, a model relating parallax to depth is established, thereby calculating the sheep's 3D coordinates in space. Simultaneously, the pixel coordinates of the sheep's outline in the image are directly recorded. This coordinate information reflects the sheep's position on the 2D image plane. This step is crucial for converting 2D image information into 3D spatial information, providing accurate positional data for subsequent analysis of the sheep's movement trajectory and behavioral characteristics. The accuracy of this step directly affects the overall precision of the recognition method.

[0050] Step S3: Track and record the three-dimensional coordinates and pixel coordinates of the sheep in each frame of the image according to the time series, and generate the spatial motion trajectory data of the sheep and the two-dimensional motion trajectory data in the image.

[0051] Specifically, the 3D coordinates and pixel coordinates of individual sheep in each frame of the image are tracked and recorded according to the time sequence. This operation aims to generate spatial trajectory data and 2D trajectory data of the sheep in the image. In the implementation process, the 3D coordinates and pixel coordinates of the sheep in the first frame are first marked and stored to establish initial coordinate records for subsequent tracking. Starting from the second frame, image matching algorithms, such as feature-based matching algorithms (SIFT, SURF, etc.), are used to find feature points in the current frame that correspond to the sheep in the previous frame. By calculating the similarity between feature points (generally using Euclidean distance or Hamming distance, with a threshold set between 0.6 and 0.8), the sheep in the current frame are accurately matched with those in the previous frame, thereby obtaining the corresponding 3D coordinates and pixel coordinates of the sheep.

[0052] The matched coordinates of the sheep in the current frame are associated with the stored coordinate sequence, and the sheep's movement trajectory data is updated in chronological order. This matching and updating operation is repeated until all image frames are processed, ultimately completing the recording of the sheep's movement trajectory data. The significance of this step lies in connecting discrete single-frame coordinate information to form a continuous movement trajectory, which can intuitively reflect the sheep's movement state and behavioral pattern changes over a period of time. This provides dynamic data support for subsequent extraction of sheep's behavioral features and is an important foundation for analyzing the spatiotemporal characteristics of sheep drinking behavior.

[0053] Step S4: Construct a behavior spatiotemporal feature fusion model. The behavior spatiotemporal feature fusion model includes a spatial feature extraction module and a temporal feature extraction module. Input the three-dimensional coordinates and two-dimensional motion trajectory data of the sheep individual into the spatial feature extraction module and the temporal feature extraction module respectively to extract the spatial behavior features and temporal behavior features of the sheep individual.

[0054] Specifically, a spatiotemporal feature fusion model of behavior is constructed. This model includes a spatial feature extraction module and a temporal feature extraction module. The core of this step is to input the three-dimensional coordinates and two-dimensional motion trajectory data of an individual sheep into the corresponding modules to extract the spatial and temporal behavioral features of the sheep. The spatial feature extraction module mainly focuses on the static and dynamic information of the sheep, such as its position and posture in space. In implementation, the three-dimensional coordinate data of the sheep is first filtered and preprocessed to remove noise data and outliers. Then, by designing specific calculation rules, such as calculating the relative positional relationships and angle changes between different parts of the sheep (head, neck, body, etc.), key features that can reflect the spatial characteristics of the sheep's drinking behavior are extracted, such as the relative height of the sheep's mouth to the water surface and the degree of curvature of the sheep's neck. These features can reflect the sheep's drinking posture and positional changes in space.

[0055] The temporal feature extraction module focuses on analyzing the changing patterns of sheep's individual movement trajectories over time. The two-dimensional movement trajectory data of each sheep is segmented chronologically, dividing the continuous trajectory into multiple time segments. Then, parameters such as the sheep's speed, acceleration, and oscillation frequency within each time segment are calculated. Through statistical analysis of these parameters, features related to the temporal characteristics of the sheep's drinking behavior are extracted, such as the rhythm of the sheep's head oscillation during drinking and the duration of the drinking action. The collaborative work of these two modules enables the comprehensive extraction of sheep's behavioral features from both spatial and temporal dimensions, providing rich information for subsequent feature fusion and behavior recognition.

[0056] Step S5: Based on various parameters of sheep drinking behavior, including but not limited to the distance between the sheep's mouth and the water surface, the sheep's neck bending angle, and the sheep's head swinging frequency, the spatial and temporal behavioral features are fused to obtain a spatiotemporal fusion feature vector of individual sheep behavior.

[0057] Specifically, based on various parameters of sheep drinking behavior, including the distance between the sheep's mouth and the water surface, the angle of neck bending, and the frequency of head swaying, the extracted spatial and temporal behavioral features are fused to obtain a spatiotemporal fusion feature vector of individual sheep behavior. In the fusion process, it is first necessary to determine the correlation between various sheep drinking behavior parameters and spatial and temporal behavioral features. Based on the importance of these parameters in sheep drinking behavior, different features are assigned corresponding weights. For example, the distance between the sheep's mouth and the water surface plays an important role in determining whether the sheep is drinking and can be assigned a higher weight; while the frequency of head swaying is crucial in reflecting the rhythm of drinking actions and is also given an appropriate weight.

[0058] Then, the spatial and temporal behavioral features are weighted and combined according to a set weight. The two feature vectors are then fused using a specific calculation method to form a new feature vector, namely the spatiotemporal fused feature vector. This fused feature vector integrates key information about sheep drinking behavior in both spatial and temporal dimensions, enabling a more comprehensive and accurate description of an individual sheep's drinking behavior. Compared to single spatial or temporal features, it has stronger representational capabilities, laying a solid foundation for subsequent accurate behavior recognition using a classifier.

[0059] Step S6: Using a pre-trained classifier, classify and judge the spatiotemporal fusion feature vector to identify whether an individual sheep is in a drinking behavior state.

[0060] Specifically, a pre-trained classifier classifies the generated spatiotemporal fusion feature vectors to identify whether a sheep is in a drinking behavior state. During the training phase, the classifier uses a large number of labeled spatiotemporal fusion feature vectors of sheep drinking and non-drinking behaviors as training samples. Machine learning algorithms, such as Support Vector Machines (SVM) and Random Forests, learn the feature patterns and classification boundaries of these samples to establish an accurate classification model. In practical applications, the spatiotemporal fusion feature vector of the sheep to be identified is input into the trained classifier. Based on the learned classification rules and model parameters, the classifier analyzes and judges the feature vector, outputting a classification result indicating whether the sheep is in a drinking or non-drinking behavior state.

[0061] This step is the final stage in the entire intelligent sheep drinking behavior recognition method, and its accuracy directly determines the effectiveness of the recognition method. By employing a suitable classifier and scientific training methods, the spatiotemporal fusion feature vectors extracted and fused in the previous steps can be fully utilized to quickly and accurately identify the drinking behavior of individual sheep. This provides strong technical support for monitoring and managing sheep behavior in livestock farming, helps to understand the health status and behavioral habits of sheep flocks in a timely manner, and enables scientific breeding and precise management.

[0062] Preferably, in the behavior spatiotemporal feature fusion model, the spatial feature extraction module uses the following formula to extract features from the three-dimensional coordinates of an individual sheep: ,in, This represents the extracted spatial behavior feature vector. For individual sheep in continuous The number of three-dimensional coordinates in a frame image For the first Weighting coefficients for the three-dimensional coordinates of a frame image. It is a three-dimensional coordinate feature mapping function constructed based on sheep drinking behavior parameters. The sheep individuals were in the 1st month of the 2nd month. The three-dimensional coordinates in the frame image, the feature mapping function The study comprehensively considers the influence of the distance between the sheep's snout and the water surface on the coordinate weights, as well as the manifestation of the sheep's neck bending angle in spatial coordinate changes.

[0063] Specifically, the spatial feature extraction module in the spatiotemporal feature fusion model operates as follows: This module extracts features from the three-dimensional coordinates of individual sheep, introducing weight coefficients to measure the importance of the three-dimensional coordinates in different frames during feature extraction. Its constructed three-dimensional coordinate feature mapping function fully considers the influence of the distance between the sheep's mouth and the water surface on the coordinate weights, as well as the manifestation of the sheep's neck bending angle in spatial coordinate changes. In practice, the three-dimensional coordinates of individual sheep are first collected from multiple consecutive frames of images. Then, based on the aforementioned influencing factors, corresponding weights are assigned to each coordinate. After calculation and processing, a feature vector that accurately reflects the spatial characteristics of the sheep's drinking behavior is output, providing crucial spatial dimensional information for subsequent behavior recognition.

[0064] Preferably, in the behavior spatiotemporal feature fusion model, the time feature extraction module processes the two-dimensional motion trajectory data of an individual sheep using the following formula: ,in, This represents the extracted time-based behavioral feature vector. The adjustment parameters for time feature extraction, The time interval between two adjacent frames. This represents the change in two-dimensional pixel coordinates of an individual sheep between two adjacent image frames. It is a time feature calculation function designed based on the sheep's head swaying frequency. By calculating the time interval and pixel coordinate changes, it obtains features related to the time characteristics of sheep drinking behavior.

[0065] Specifically, the temporal feature extraction module processes the two-dimensional motion trajectory data of individual sheep. This module controls the intensity of temporal feature extraction by adjusting parameters. Based on the time interval between two adjacent image frames and the change in the two-dimensional pixel coordinates of the individual sheep, a temporal feature calculation function is designed, incorporating the sheep's head-swaying frequency. In practice, the two-dimensional motion trajectory data of individual sheep between consecutive image frames is analyzed. The calculation function is used to mine features related to the temporal characteristics of sheep drinking behavior, generating a temporal behavior feature vector. This provides data support for comprehensively characterizing the temporal dimension of sheep drinking behavior.

[0066] Preferably, in step S5, the process of fusing spatial behavior features and temporal behavior features adopts the following formula: ,in, This is the fused spatiotemporal fusion feature vector. and The fusion weights are spatial and temporal behavioral features, respectively. By adjusting these two weights, the fused feature vector can more accurately reflect the spatiotemporal characteristics of sheep drinking behavior. The determination of the fusion weights is based on the importance of various parameters of sheep drinking behavior in the spatial and temporal dimensions.

[0067] Specifically, in step S5, the fusion process of spatial and temporal behavioral features involves setting fusion weights and weighting the spatial and temporal behavioral features based on the importance of the sheep drinking behavior parameters in the spatial and temporal dimensions. In practice, the correlation between each behavioral parameter and the spatiotemporal features is first determined, and then the fusion weights are reasonably allocated. The two feature vectors are then linearly combined to obtain the fused spatiotemporal feature vector. This vector integrates the key information of sheep drinking behavior in the spatiotemporal dimensions, and compared to a single feature vector, it can more comprehensively and accurately describe the characteristics of sheep drinking behavior, improving the accuracy of subsequent classification judgments.

[0068] Preferably, in step S2, when determining the three-dimensional coordinates of an individual sheep, the binocular vision image analysis algorithm calculates the disparity of corresponding points of the individual sheep in the left and right views, and establishes the relationship between disparity and depth by combining the calibration parameters of the binocular vision imaging system, thereby obtaining the three-dimensional coordinates of the individual sheep in space. Furthermore, when extracting the contour information of the individual sheep, an edge detection and region growing method is adopted, and the accuracy of contour extraction is optimized based on the possible changes in body posture caused by the sheep's drinking behavior.

[0069] Specifically, when determining the 3D coordinates of an individual sheep, the algorithm utilizes the principle of binocular vision imaging. It calculates the disparity between corresponding points in the left and right views and establishes the relationship between disparity and depth by combining system calibration parameters, thereby obtaining the sheep's 3D coordinates in space. For extracting the sheep's contour information, a combination of edge detection and region growing is employed. To address potential changes in body posture during drinking, the parameters of the contour extraction algorithm are optimized to improve accuracy. This ensures that the acquired sheep coordinates and contour information accurately reflect their spatial state, laying the foundation for subsequent behavioral analysis.

[0070] Preferably, in step S4, before inputting the data into the spatiotemporal feature fusion model, the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep are normalized. The normalization formula is as follows: ,in, For the normalized data, The original data, and These are the maximum and minimum values ​​in the original data, respectively. Through this normalization process, the relevant data of sheep drinking behavior parameters at different scales can be effectively processed in the behavior spatiotemporal feature fusion model.

[0071] Specifically, before inputting the three-dimensional coordinates and two-dimensional motion trajectory data of individual sheep into the spatiotemporal feature fusion model, normalization processing is performed. By mapping the raw data to a specific interval, the interference of data at different scales on the model processing is eliminated, allowing various data related to sheep drinking behavior parameters, such as distance, angle, and coordinate changes, to be effectively processed in the model using a unified standard. During implementation, the maximum and minimum values ​​in the raw data are determined, and the data is transformed according to normalization rules to ensure that the model can accurately extract effective features from the data, thereby improving the model's processing efficiency and recognition accuracy.

[0072] Preferably, in step S3, tracking and recording the three-dimensional coordinates and pixel coordinates of individual sheep according to a time series specifically includes:

[0073] Step 3.1: Mark and store the three-dimensional coordinates and pixel coordinates of the individual sheep in the first frame image to establish an initial coordinate record;

[0074] Step 3.2: In each subsequent frame, the sheep in the current frame is matched with the sheep in the previous frame using an image matching algorithm to obtain the three-dimensional coordinates and pixel coordinates of the corresponding sheep.

[0075] Step 3.3: Associate the matched coordinates of the sheep in the current frame with the stored coordinate sequence to update the sheep's motion trajectory data;

[0076] Step 3.4: Determine if the end of the image sequence has been reached. If not, return to sub-step two to continue processing the next frame. If the end has been reached, the recording of the sheep's individual movement trajectory data is complete.

[0077] Specifically, the coordinates of the individual sheep in the first frame image are first marked and stored to establish an initial data record. Then, in subsequent image frames, an image matching algorithm is used to achieve inter-frame correspondence matching based on the feature information of the individual sheep to obtain the coordinates of the individual sheep in each frame. Then, the newly obtained coordinates are associated with the stored coordinate sequence to update the motion trajectory data. Finally, by determining whether the end of the image sequence has been reached, the process is controlled to loop until all image frames are processed to form complete motion trajectory data of the individual sheep, providing continuous and accurate coordinate information for behavioral feature extraction.

[0078] Preferably, in step S4, the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep are input into the spatial feature extraction module and temporal feature extraction module of the behavior spatiotemporal feature fusion model, respectively, specifically including:

[0079] Step 4.1: Perform feature filtering on the three-dimensional coordinate data of individual sheep, and select key coordinate dimensions that are related to the spatial characteristics of sheep drinking behavior;

[0080] Step 4.2: Convert the format of the filtered 3D coordinate data to meet the input requirements of the spatial feature extraction module;

[0081] Step 4.3: Perform time series segmentation processing on the two-dimensional motion trajectory data of individual sheep, dividing the continuous trajectory data into multiple time segments;

[0082] Step 4.4: Input the segmented time data into the time feature extraction module for processing.

[0083] Specifically, for the three-dimensional coordinate data of individual sheep, the coordinate dimensions closely related to the spatial characteristics of the sheep's drinking behavior are first selected, redundant information is removed, and then the format is converted to adapt it to the input requirements of the spatial feature extraction module. For the two-dimensional motion trajectory data, it is segmented according to time sequence, divided into multiple time segments, and then the data of each time segment is input into the time feature extraction module in sequence. Through this series of operations, it is ensured that the data input to the model can accurately correspond to the functional requirements of each feature extraction module, thereby improving the targeting and effectiveness of feature extraction.

[0084] Preferably, in step S5, the spatial and temporal behavioral features are fused based on various parameters of the sheep's drinking behavior, specifically including:

[0085] Step S5.1: Establish a correlation table between sheep drinking behavior parameters and spatial and temporal behavioral characteristics, and clarify how each parameter is reflected in the characteristics;

[0086] Step S5.2: Based on the aforementioned association table, perform weighted adjustments on the spatial and temporal behavioral features to highlight the features closely related to sheep drinking behavior;

[0087] Step S5.3: Combine the weighted spatial behavior features and temporal behavior features to form preliminary fusion features;

[0088] Step S5.4: Perform dimensionality reduction on the preliminary fusion features to remove redundant information and obtain the final spatiotemporal fusion feature vector.

[0089] Specifically, firstly, a correlation table between sheep drinking behavior parameters and spatiotemporal features is established to clarify the manifestation of each parameter in spatial and temporal behavioral features; then, based on the correlation table, the spatiotemporal features are weighted and adjusted to enhance the weight of features closely related to drinking behavior; next, the weighted features are spliced ​​and combined to form preliminary fusion features; finally, through dimensionality reduction processing, redundant information is removed to obtain a concise and effective spatiotemporal fusion feature vector that can represent the spatiotemporal characteristics of sheep drinking behavior, providing high-quality feature data for behavior recognition.

[0090] like Figure 2 As shown, the intelligent recognition device for sheep drinking behavior based on spatiotemporal feature fusion includes:

[0091] The image acquisition unit is equipped with a binocular vision imaging system to acquire images of the sheep's activity area and obtain binocular vision image pairs.

[0092] An image analysis unit, connected to the image acquisition unit, is used to process the binocular vision image pair using a binocular vision image analysis algorithm to extract the outline information of the individual sheep and determine the three-dimensional coordinates and pixel coordinates of the individual sheep.

[0093] The trajectory recording unit, connected to the image analysis unit, is used to track and record the three-dimensional coordinates and pixel coordinates of an individual sheep according to a time series, and generate motion trajectory data.

[0094] The feature extraction unit, connected to the trajectory recording unit, is used to construct a behavior spatiotemporal feature fusion model, and inputs the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep into the spatial feature extraction module and temporal feature extraction module of the model respectively to extract spatial behavior features and temporal behavior features;

[0095] The feature fusion unit, connected to the feature extraction unit, is used to fuse the spatial behavior features and temporal behavior features based on various parameters of the sheep's drinking behavior to obtain a spatiotemporal fusion feature vector.

[0096] The behavior recognition unit, connected to the feature fusion unit, is used to classify and judge the spatiotemporal fusion feature vector through a pre-trained classifier to identify whether an individual sheep is in a drinking behavior state.

[0097] A method and device for intelligent recognition of sheep drinking behavior based on spatiotemporal feature fusion. This device, utilizing a binocular vision imaging system in the image acquisition unit, automatically and in real-time acquires images of the sheep's activity area, obtaining left and right views at the same time. Subsequently, through the close cooperation of modules such as the image analysis unit and the trajectory recording unit, the entire process from image acquisition to behavioral trajectory data generation can be automated without manual observation and recording of each sheep individually. This significantly saves manpower, improves monitoring efficiency, and enables continuous, unmanned monitoring of large-scale sheep drinking behavior.

[0098] To address the challenge of insufficient recognition accuracy in complex livestock farming scenarios using existing automated technologies, this method constructs a spatiotemporal feature fusion model. In the spatial dimension, a binocular vision image analysis algorithm calculates the disparity between corresponding points in the left and right views, combined with system calibration parameters, to accurately obtain the three-dimensional coordinates of an individual sheep in space. Simultaneously, an edge detection and region growing approach is employed to fully consider the posture changes of sheep while drinking water, accurately extracting individual contour information. In the temporal dimension, based on behavioral parameters such as the frequency of sheep head swaying, the two-dimensional motion trajectory data is meticulously analyzed. Subsequently, based on key drinking behavior parameters such as the distance between the sheep's mouth and the water surface, and the neck bending angle, the extracted spatial and temporal behavioral features are deeply fused to form a spatiotemporal fusion feature vector that comprehensively reflects the characteristics of sheep drinking behavior. Finally, a pre-trained classifier performs accurate judgment. This series of operations effectively overcomes the recognition errors caused by insufficient feature extraction and inaccurate fusion in complex scenarios, significantly improving recognition accuracy and providing reliable technical support for intelligent farming, thus promoting the development of animal husbandry towards a more efficient and precise direction.

[0099] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," "link," and "fix" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0100] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for intelligent recognition of sheep drinking behavior based on spatio-temporal feature fusion, characterized in that, Includes the following steps: Step S1: Use a binocular vision imaging system to acquire images of the sheep's activity area, and obtain the left and right views of the sheep at the same time. The left and right views constitute a binocular vision image pair. Step S2: The binocular vision image pair is processed by a binocular vision image analysis algorithm to extract the contour information of the individual sheep in the image, and the three-dimensional coordinates of the individual sheep in space are determined based on the contour information. At the same time, the pixel coordinates of the individual sheep in the image are obtained. Step S3: Track and record the three-dimensional coordinates and pixel coordinates of the sheep in each frame of the image according to the time series, and generate the spatial motion trajectory data of the sheep and the two-dimensional motion trajectory data in the image. Step S4: Construct a behavior spatiotemporal feature fusion model. The behavior spatiotemporal feature fusion model includes a spatial feature extraction module and a temporal feature extraction module. Input the three-dimensional coordinates and two-dimensional motion trajectory data of the sheep individual into the spatial feature extraction module and the temporal feature extraction module respectively to extract the spatial behavior features and temporal behavior features of the sheep individual. Step S5: Based on various parameters of sheep drinking behavior, including but not limited to the distance between the sheep's mouth and the water surface, the sheep's neck bending angle, and the sheep's head swinging frequency, the spatial and temporal behavioral features are fused to obtain a spatiotemporal fusion feature vector of individual sheep behavior. Step S6: Using a pre-trained classifier, classify and judge the spatiotemporal fusion feature vector to identify whether an individual sheep is in a drinking behavior state; In the aforementioned spatiotemporal feature fusion model, the spatial feature extraction module uses the following formula to extract features from the three-dimensional coordinates of an individual sheep: ,in, This represents the extracted spatial behavior feature vector. For individual sheep in continuous The number of three-dimensional coordinates in a frame image For the first Weighting coefficients for the three-dimensional coordinates of a frame image. It is a three-dimensional coordinate feature mapping function constructed based on sheep drinking behavior parameters. The sheep individuals were in the 1st month of the 2nd month. The three-dimensional coordinates in the frame image, the feature mapping function The study comprehensively considers the influence of the distance between the sheep's snout and the water surface on the coordinate weights, as well as the manifestation of the sheep's neck bending angle in spatial coordinate changes. In the aforementioned spatiotemporal feature fusion model, the time feature extraction module processes the two-dimensional motion trajectory data of individual sheep using the following formula: ,in, This represents the extracted time-based behavioral feature vector. The adjustment parameters for time feature extraction, The time interval between two adjacent frames. This represents the change in two-dimensional pixel coordinates of an individual sheep between two adjacent image frames. It is a time feature calculation function designed based on the sheep's head swaying frequency. By calculating the time interval and pixel coordinate changes, it obtains features related to the time characteristics of sheep drinking behavior.

2. The method according to claim 1, wherein, In step S5, the process of fusing spatial behavior features and temporal behavior features adopts the following formula: ,in, This is the fused spatiotemporal fusion feature vector. and The fusion weights are spatial and temporal behavioral features, respectively. By adjusting these two weights, the fused feature vector can more accurately reflect the spatiotemporal characteristics of sheep drinking behavior. The determination of the fusion weights is based on the importance of various parameters of sheep drinking behavior in the spatial and temporal dimensions.

3. The method according to claim 1, wherein, In step S2, when determining the three-dimensional coordinates of an individual sheep, the binocular vision image analysis algorithm calculates the disparity of corresponding points of the individual sheep in the left and right views, and establishes the relationship between disparity and depth by combining the calibration parameters of the binocular vision imaging system, thereby obtaining the three-dimensional coordinates of the individual sheep in space. Furthermore, when extracting the contour information of the individual sheep, an edge detection and region growing method is adopted, and the accuracy of contour extraction is optimized based on the possible changes in body posture caused by the sheep's drinking behavior.

4. The method according to claim 1, wherein, In step S4, before inputting the data into the spatiotemporal feature fusion model, the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep are normalized. The normalization formula is as follows: ,in, For the normalized data, The original data, and These are the maximum and minimum values ​​in the original data, respectively. Through this normalization process, the relevant data of sheep drinking behavior parameters at different scales are processed in the behavior spatiotemporal feature fusion model.

5. The method according to claim 1, wherein, In step S3, the three-dimensional coordinates and pixel coordinates of individual sheep are tracked and recorded according to the time series, specifically including: Step 3.1: Mark and store the three-dimensional coordinates and pixel coordinates of the individual sheep in the first frame image to establish an initial coordinate record; Step 3.2: In each subsequent frame, the sheep in the current frame is matched with the sheep in the previous frame using an image matching algorithm to obtain the three-dimensional coordinates and pixel coordinates of the corresponding sheep. Step 3.3: Associate the matched coordinates of the sheep in the current frame with the stored coordinate sequence to update the sheep's motion trajectory data; Step 3.4: Determine if the end of the image sequence has been reached. If not, return to sub-step two to continue processing the next frame. If the end has been reached, the recording of the sheep's individual movement trajectory data is complete.

6. The method according to claim 1, wherein, In step S4, the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep are input into the spatial feature extraction module and temporal feature extraction module of the behavior spatiotemporal feature fusion model, respectively, specifically including: Step 4.1: Perform feature filtering on the three-dimensional coordinate data of individual sheep, and select key coordinate dimensions that are related to the spatial characteristics of sheep drinking behavior; Step 4.2: Convert the format of the filtered 3D coordinate data to meet the input requirements of the spatial feature extraction module; Step 4.3: Perform time series segmentation processing on the two-dimensional motion trajectory data of individual sheep, dividing the continuous trajectory data into multiple time segments; Step 4.4: Input the segmented time data into the time feature extraction module for processing.

7. The method according to claim 1, wherein, In step S5, spatial and temporal behavioral features are fused based on various parameters of the sheep's drinking behavior, specifically including: Step S5.1: Establish a correlation table between sheep drinking behavior parameters and spatial and temporal behavioral characteristics, and clarify how each parameter is reflected in the characteristics; Step S5.2: Based on the aforementioned association table, perform weighted adjustments on the spatial and temporal behavioral features to highlight the features closely related to sheep drinking behavior; Step S5.3: Combine the weighted spatial behavior features and temporal behavior features to form preliminary fusion features; Step S5.4: Perform dimensionality reduction on the preliminary fusion features to remove redundant information and obtain the final spatiotemporal fusion feature vector.

8. The device for intelligent recognition of sheep drinking behavior based on spatiotemporal feature fusion, characterized in that, The device is applied to the intelligent recognition method for sheep drinking behavior based on spatiotemporal feature fusion as described in claim 1, comprising: The image acquisition unit is equipped with a binocular vision imaging system to acquire images of the sheep's activity area and obtain binocular vision image pairs. An image analysis unit, connected to the image acquisition unit, is used to process the binocular vision image pair using a binocular vision image analysis algorithm to extract the outline information of the individual sheep and determine the three-dimensional coordinates and pixel coordinates of the individual sheep. The trajectory recording unit, connected to the image analysis unit, is used to track and record the three-dimensional coordinates and pixel coordinates of an individual sheep according to a time series, and generate motion trajectory data. The feature extraction unit, connected to the trajectory recording unit, is used to construct a behavior spatiotemporal feature fusion model, and inputs the three-dimensional coordinates and two-dimensional motion trajectory data of the individual sheep into the spatial feature extraction module and temporal feature extraction module of the model respectively to extract spatial behavior features and temporal behavior features; The feature fusion unit, connected to the feature extraction unit, is used to fuse the spatial behavior features and temporal behavior features based on various parameters of the sheep's drinking behavior to obtain a spatiotemporal fusion feature vector. The behavior recognition unit, connected to the feature fusion unit, is used to classify and judge the spatiotemporal fusion feature vector through a pre-trained classifier to identify whether an individual sheep is in a drinking behavior state.

Images