Intelligent monitoring method and system for rumination behaviors of cattle herds

Abstract

The invention relates to the technical field of intelligent livestock management, in particular to an intelligent monitoring method and system for rumination behaviors of cattle individuals, and solves the technical problem of low monitoring accuracy in the prior art. The method comprises the following steps: acquiring cattle herd monitoring video data, and performing cattle mouth part identification on image frames in the cattle herd monitoring video data through a target identification model to obtain a cattle mouth area image in each image frame; performing shielding detection on the cattle mouth area image in each image frame, and screening out a target cattle mouth area image from the cattle mouth area images in each image frame according to an obtained shielding proportion; processing the screened target cattle mouth area image through an optical flow algorithm, and generating an optical flow field sequence representing cattle mouth movement; and inputting the optical flow field sequence into a preset classifier, judging whether the cattle is in a rumination behavior, and outputting a monitoring result of the rumination behavior.

Description

Technical Field

[0001] This invention relates to the field of intelligent livestock management technology, specifically to an intelligent monitoring method and system for individual rumination behavior in cattle herds. Background Technology

[0002] Rumination is a core digestive physiological behavior in ruminants such as cattle. It not only affects the efficiency of nutrient absorption but also provides crucial information for farmers and veterinarians to assess the health of cattle. Therefore, accurate monitoring of individual rumination behavior in cattle is of great significance for large-scale livestock management. With the development of computer vision technology, intelligent monitoring methods based on surveillance video are gradually replacing traditional manual observation and becoming the mainstream approach for monitoring rumination behavior in cattle.

[0003] Current common monitoring technologies are usually based on monitoring videos of cattle herds, using the movement of cattle in the videos to intelligently monitor rumination behavior. However, in actual farming scenarios, cattle are frequently active, and their walking, feeding, and other behaviors can easily affect the monitoring of rumination behavior, leading to low monitoring accuracy. Summary of the Invention

[0004] To address the technical problem of low accuracy in existing monitoring technologies, the present invention aims to provide an intelligent monitoring method and system for individual rumination behavior in cattle herds. The specific technical solution adopted is as follows: This application provides an intelligent monitoring method for individual rumination behavior in cattle, including: Acquire cattle herd monitoring video data, and use a target recognition model to identify the mouth area of ​​cattle in the image frames of the cattle herd monitoring video data to obtain the mouth area image of each image frame. Occlusion detection is performed on the cow mouth region image in each image frame, and the target cow mouth region image is selected from the cow mouth region images in each image frame according to the obtained occlusion ratio; The selected target cow mouth region image is processed using an optical flow algorithm to generate an optical flow field sequence representing the movement of the cow mouth. The optical flow field sequence is input into a preset classifier to determine whether the cattle are ruminating and outputs the monitoring results of rumination behavior.

[0005] In one possible implementation, the method includes: For each image frame, the cow mouth region image is divided into multiple sub-blocks; For each sub-block, the gradient magnitude matrix of the sub-block is calculated based on the gradient operator; the gradient magnitude matrix includes the gradient magnitude of each pixel in the sub-block. Based on the calculated gradient magnitude matrix, calculate the occlusion ratio of the cow's mouth region in each image frame; The image of the cow's mouth region in each image frame with an occlusion ratio less than a preset ratio threshold is taken as the target cow's mouth region image.

[0006] In one possible implementation, the method includes: For each sub-block, the mean matrix and standard deviation matrix corresponding to the sub-block are determined based on the gradient magnitude matrix of the corresponding sub-block in the cow mouth region image of each image frame. The mean matrix includes the mean gradient magnitude of the pixels in the corresponding sub-block in the cow mouth region image of each image frame, and the standard deviation matrix includes the standard deviation of the gradient magnitude of the pixels in the corresponding sub-block in the cow mouth region image of each image frame. Based on the gradient magnitude matrix, the corresponding mean matrix, and the standard deviation matrix of the sub-block, the normalization matrix of the sub-block is determined; the normalization matrix is ​​used to characterize the degree of deviation of the gradient magnitude of each pixel in the sub-block from the standard reference value. For each sub-block, a binary mask is generated based on the normalized matrix of the sub-block; wherein, the mask value in the binary mask corresponds one-to-one with the pixel of the sub-block, and the mask value is used to characterize whether the corresponding pixel is occluded; The occlusion ratio of the bull's mouth region in each image frame is calculated based on the binary mask.

[0007] In one possible implementation, the preset ratio threshold is determined in the following way: Obtain multiple sets of image samples of the cow's mouth region; For each group of cow mouth region image samples, the occlusion ratio of the cow mouth region image in the cow mouth region image sample and the average occlusion ratio of the cow mouth region image sample are calculated based on the gradient magnitude matrix. Based on the distribution of the average occlusion ratio of multiple sets of bull mouth region image samples, they are sorted according to the size of the average occlusion ratio, and the average occlusion ratio at the preset position in the sorting result is selected as the preset ratio threshold.

[0008] In one possible implementation, the method includes: The motion vectors of pixels in the corresponding target bull's mouth region image in consecutive image frames are calculated using an optical flow algorithm. Generate an optical flow field sequence for the target bull's mouth region image based on the motion vectors of the pixels.

[0009] In one possible implementation, the optical flow field sequence is represented by a tensor consisting of a time dimension, a height dimension, a width dimension, and a channel dimension; wherein the time dimension represents the number of consecutive image frames, the height and width dimensions are used to represent the size of the bull's mouth region image, and the channel dimension is used to represent the motion vectors in the horizontal and vertical directions.

[0010] In one possible implementation, the pre-defined classifier is trained in the following way: Acquire training set data, which includes optical flow field sequences and labels for images of the cow's mouth region; the labels are used to characterize whether the cow corresponding to the image of the cow's mouth region is in rumination behavior. Using the training set data, a neural network algorithm is used to train a preset classifier, resulting in a well-trained preset classifier; the neural network algorithm uses a binary classification loss function for classification.

[0011] In one possible implementation, the monitoring results of rumination behavior include an image frame containing an image of the mouth region of the target cow during rumination behavior.

[0012] In one possible implementation, the method further includes: Image filtering is performed on image frames in the cattle herd monitoring video data.

[0013] This application provides an intelligent monitoring system for individual rumination behavior in cattle, including: The cattle herd monitoring module is used to acquire cattle herd monitoring video data; The target recognition module is used to identify the mouth area of ​​cattle in the image frames of cattle monitoring video data through the target recognition model, and obtain the image of the mouth area of ​​cattle in each image frame. The monitoring video processing module is used to perform occlusion detection on the cow mouth region image in each image frame. Based on the obtained occlusion ratio, the target cow mouth region image is selected from the cow mouth region image in each image frame. The selected target cow mouth region image is processed by the optical flow algorithm to generate an optical flow field sequence representing the movement of the cow mouth. The monitoring result output module is used to input the optical flow field sequence into a preset classifier to determine whether the cattle are ruminating and output the monitoring results of rumination behavior.

[0014] The present invention has the following beneficial effects: In view of the problem of low monitoring accuracy in existing technologies, this application provides an intelligent monitoring method and system for individual rumination behavior in cattle, realizing intelligent monitoring of individual rumination behavior in cattle. First, this application can accurately focus on the monitoring object by acquiring cattle monitoring video data and identifying the mouth area image, avoiding interference from other irrelevant areas (such as the cow's body and background environment) to subsequent analysis, ensuring the relevance of the monitoring data. Second, by detecting occlusion and filtering the target mouth area image according to the occlusion ratio, invalid data caused by severe occlusion can be eliminated, reducing the impact of invalid data on subsequent optical flow field generation and judgment results, improving data quality. By generating an optical flow field sequence representing the movement of the cow's mouth through an optical flow algorithm, the movement of the cow's mouth can be converted into quantifiable feature data, providing an objective basis for judging rumination behavior. Thus, this application can output monitoring results through a preset classifier, realizing automated judgment of rumination behavior, improving monitoring efficiency, and is particularly suitable for large-scale farming scenarios. The above technical solution of this application significantly improves the accuracy and efficiency of monitoring individual rumination behavior in cattle, meeting the basic needs of large-scale farming for rumination behavior monitoring. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A system architecture diagram of an intelligent monitoring system for individual rumination behavior in cattle provided in an embodiment of the present invention; Figure 2 A flowchart illustrating an intelligent monitoring method for individual rumination behavior in cattle, provided as an embodiment of the present invention; Figure 3 A flowchart illustrating another intelligent monitoring method for individual rumination behavior in cattle provided in an embodiment of the present invention; Figure 4 This is a flowchart illustrating another intelligent monitoring method for individual rumination behavior in cattle, provided as an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an intelligent monitoring method and system for individual rumination behavior in cattle according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] In view of the problem of low monitoring accuracy in existing technologies, this application provides an intelligent monitoring method and system for individual rumination behavior in cattle, realizing intelligent monitoring of individual rumination behavior in cattle. First, this application can accurately focus on the monitoring object by acquiring cattle monitoring video data and identifying the mouth area image, avoiding interference from other irrelevant areas (such as the cow's body and background environment) to subsequent analysis, ensuring the relevance of the monitoring data. Second, by detecting occlusion and filtering the target mouth area image according to the occlusion ratio, invalid data caused by severe occlusion can be eliminated, reducing the impact of invalid data on subsequent optical flow field generation and judgment results, improving data quality. By generating an optical flow field sequence representing the movement of the cow's mouth through an optical flow algorithm, the movement of the cow's mouth can be converted into quantifiable feature data, providing an objective basis for judging rumination behavior. Thus, this application can output monitoring results through a preset classifier, realizing automated judgment of rumination behavior, improving monitoring efficiency, and is particularly suitable for large-scale farming scenarios. The above technical solution of this application significantly improves the accuracy and efficiency of monitoring individual rumination behavior in cattle, meeting the basic needs of large-scale farming for rumination behavior monitoring.

[0020] The following description, in conjunction with the accompanying drawings, details the specific scheme of an intelligent monitoring method and system for individual rumination behavior in cattle provided by this invention.

[0021] Please see Figure 1 The diagram illustrates a system architecture of an intelligent monitoring system for individual rumination behavior in cattle, according to an embodiment of the present invention. The intelligent monitoring system for individual rumination behavior in cattle includes: a cattle monitoring module, a target recognition module, a monitoring video processing module, and a monitoring result output module.

[0022] The cattle herd monitoring module is used to acquire cattle herd monitoring video data.

[0023] For example, the cattle herd monitoring module may include several high-resolution PTZ cameras. The deployment location of the PTZ cameras can be determined according to the layout of the farm. The rotation angle of the PTZ cameras can be set to cover the cattle herd area. The frame rate can be set between 30FPS and 60FPS, which can be adjusted according to the size of the cattle herd. The larger the herd, the higher the density of individual cattle in the monitoring video, requiring a higher frame rate; when the herd is small, the density of cattle in a single frame will not be too high, and the frame rate can be appropriately reduced.

[0024] The target recognition module is used to identify the mouth area of ​​cattle in the image frames of cattle monitoring video data through a target recognition model, and obtain the image of the mouth area of ​​cattle in each image frame.

[0025] For example, the target recognition module may include an image preprocessing unit and a model inference unit. The image preprocessing unit is used to filter the collected cattle herd monitoring video data. The model inference unit is used to deploy the target recognition model. The image preprocessing unit can be implemented using a field-programmable gate array (FPGA) chip, with a filtering processing speed of ≥30 frames / second. The model inference unit uses a graphics processing unit (GPU) chip, with a single-frame image recognition time of ≤50ms, ensuring real-time performance.

[0026] The monitoring video processing module is used to perform occlusion detection on the cow mouth region image in each image frame. Based on the obtained occlusion ratio, the target cow mouth region image is selected from the cow mouth region images in each image frame. The selected target cow mouth region image is processed by optical flow algorithm to generate an optical flow field sequence representing the movement of the cow mouth.

[0027] The monitoring result output module is used to input the optical flow field sequence into a preset classifier to determine whether the cattle are ruminating and output the monitoring results of rumination behavior.

[0028] It should be noted that the various embodiments of this application can be referenced or learned from each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referenced from each other without limitation.

[0029] Please see Figure 2 The diagram illustrates a flowchart of an intelligent monitoring method for individual rumination behavior in cattle, according to an embodiment of the present invention. The method includes the following steps: Step 201: Acquire cattle herd monitoring video data, and use a target recognition model to identify the mouth area of ​​the cattle in the image frames of the cattle herd monitoring video data, so as to obtain the mouth area image of each image frame.

[0030] It should be noted that rumination is the process by which cattle regurgitate food from their stomachs into their mouths to chew it again after a period of time. The dynamic characteristics of rumination are mainly reflected in the mouth area of ​​cattle. Therefore, in order to avoid the activities of other parts of the cattle's body (such as the rise and fall of the abdomen during breathing) from affecting the monitoring of rumination, this application can identify the mouth area of ​​cattle from the image frame for subsequent monitoring.

[0031] The cattle herd monitoring video data can be collected by pan-tilt cameras deployed within the breeding area. The frame rate can be flexibly set according to the number of cattle. For example, when the number of cattle is 30-50, the frame rate is set to 30 FPS; when the number of cattle exceeds 50, the frame rate is set to 60 FPS to ensure that the details of each cow's mouth movements can be clearly captured. A target recognition model is used to accurately locate the cow's mouth region in the image frame. Since the cow's mouth typically occupies less than 5% of the entire image, it is a typical small target. Therefore, this application can use a model specifically designed for small target recognition to ensure recognition accuracy.

[0032] For example, the target recognition model can utilize a real-time target detection model (You Only Look Once, YOLO) architecture. During the model training phase, multiple image frames are selected from historical monitoring videos of the farm. The cow's mouth region and non-cow's mouth regions in the image frames are labeled as 1 and 0 respectively. These labeled image frames are used as training samples. The EIoU loss function can be used during training, which can more accurately calculate the deviation between the predicted bounding box and the ground truth bounding box, and is suitable for small target detection. The initial learning rate is set to 0.001, and multiple iterations are performed. After training, the filtered image frames are input into the model, and the model outputs the coordinate information of the cow's mouth region in each image frame. Based on the coordinate information, the corresponding cow's mouth region image is output for subsequent unified processing.

[0033] In some embodiments, to improve the accuracy of subsequent identification, this application may also perform image filtering processing on the image frames in the cattle herd monitoring video data before identifying the cow's mouth part through the target recognition model.

[0034] The role of image filtering is to eliminate interference pixels caused by equipment noise and lighting fluctuations during the acquisition process. For example, median filtering can be used to filter salt-and-pepper noise (such as black and white noise caused by poor contact of monitoring equipment), and Gaussian filtering can be used to filter Gaussian noise (such as blurry noise in low light environment). Through filtering, the signal-to-noise ratio of the image can be further improved, providing a clearer input image for the target recognition model, thereby improving the accuracy of cow mouth recognition.

[0035] Step 202: Perform occlusion detection on the cow mouth region image in each image frame, and select the target cow mouth region image from the cow mouth region images in each image frame according to the obtained occlusion ratio.

[0036] It should be noted that during cattle herd monitoring, ruminant cattle are typically in a still state, lying down or standing, except when chewing. Cattle not in a ruminant state are prone to various disturbing physiological activities (such as pacing back and forth to eat, head raising and lowering, tail sweeping, etc.), which frequently obscure the image of ruminant cattle for short periods during herd monitoring. This obscuring of the image severely affects the generation of the optical flow field, leading to errors in the monitoring of ruminant behavior.

[0037] Therefore, in order to eliminate the influence of occlusion on the optical flow field, this application can perform occlusion detection on the image of the cow's mouth region, analyze the occlusion ratio, and then use the cow's mouth region image with low occlusion ratio and high information confidence to conduct subsequent rumination behavior monitoring, thereby improving the accuracy and efficiency of monitoring individual rumination behavior in cattle herds.

[0038] Among them, occlusion detection refers to determining whether there is a part in the cow's mouth region image that is occluded by other objects (such as the bodies of other cows or breeding equipment). The occlusion ratio can be represented by the ratio of the number of pixels in the occluded area to the total number of pixels in the cow's mouth region image.

[0039] Step 203: Process the selected target cow mouth region image using an optical flow algorithm to generate an optical flow field sequence representing the movement of the cow mouth.

[0040] Among them, the optical flow algorithm refers to the algorithm that calculates the pixel motion vector by analyzing the motion trajectory of pixels in continuous image frames. The "closing-opening" motion of the cow's mouth in rumination has a certain periodicity, and the corresponding pixel motion vector will also show regular changes. The optical flow field sequence can intuitively reflect this motion pattern, which is convenient for subsequent identification of rumination behavior.

[0041] In one possible implementation, this application can use an optical flow algorithm to calculate the motion vectors of pixels in the corresponding target bull's mouth region image in consecutive image frames, and generate an optical flow field sequence of the target bull's mouth region image based on the motion vectors of the pixels.

[0042] For example, the optical flow algorithm can employ the DeepFlow optical flow algorithm, selecting 30 consecutive frames of the target cow's mouth region image (covering one complete rumination action), and calculating the motion vectors of each pixel in the horizontal direction (u-axis) and vertical direction (v-axis) between adjacent frames. The optical flow field is the set of motion vectors of all pixels in the target cow's mouth region image, and the optical flow field sequence is a sequence of multiple consecutive optical flow fields arranged in temporal order.

[0043] In some embodiments, the optical flow field sequence is represented by a tensor consisting of a time dimension, a height dimension, a width dimension, and a channel dimension.

[0044] In this context, the time dimension represents the number of consecutive image frames, the height and width dimensions characterize the size of the cow's mouth region image, and the channel dimension characterizes the motion vectors in the horizontal and vertical directions. For example, the dimensions of an optical flow field sequence can be represented as [64, H, W, 2], where 64 is a fixed number of consecutive image frames (i.e., the time dimension), sufficient to cover the complete motion cycle of a typical rumination behavior, H and W are the height and width of the input cow's mouth region image, respectively, and the channel dimension is set to 2 to represent the motion vectors in the horizontal and vertical directions, respectively.

[0045] Step 204: Input the optical flow field sequence into the preset classifier to determine whether the cattle are in rumination behavior and output the monitoring results of rumination behavior.

[0046] The preset classifier is a binary classification model trained based on rumination behavior features, used to distinguish between rumination behavior and non-rumination behavior (such as feeding, head raising, and stillness).

[0047] In some embodiments, the preset classifier is trained by: acquiring training set data, which includes the optical flow field sequence and labels of the bull's mouth region image; and using the training set data, training the preset classifier with a neural network algorithm to obtain the trained preset classifier.

[0048] For example, the preset classifier can be a temporal classification model built based on a three-dimensional convolutional neural network or a convolutional long short-term memory network. The input data of this temporal classification model is a 4-dimensional tensor in the above-mentioned [64, H, W, 2] format, which can simultaneously extract motion texture features of the cow's mouth region from the spatial dimension (H, W) and learn the periodic pattern of rumination behavior from the temporal dimension (64). During training, a binary classification cross-entropy loss function can be used for optimization.

[0049] The label is used to characterize whether the cow in the image corresponding to the cow's mouth region is ruminating. The neural network algorithm uses a binary classification loss function for classification. For example, a label of 1 indicates rumination, and a label of 0 indicates non-rumination. The binary classification loss function can be the cross-entropy loss function or the sigmoid activation function used for binary classification tasks.

[0050] In some embodiments, the monitoring results of rumination behavior include image frames containing images of the mouth region of a target cow in the process of rumination.

[0051] Based on the above technical solution, this application can accurately focus on the monitoring object by acquiring cattle herd monitoring video data and identifying the cattle's mouth area image, avoiding interference from other irrelevant areas (such as the cattle's body and background environment) to subsequent analysis, thus ensuring the relevance of the monitoring data. Secondly, by detecting occlusion and filtering the target cattle's mouth area image according to the occlusion ratio, invalid data caused by severe occlusion can be eliminated, reducing the impact of invalid data on subsequent optical flow field generation and judgment results, and improving data quality. By generating an optical flow field sequence representing the movement of the cattle's mouth through an optical flow algorithm, the movement of the cattle's mouth can be transformed into quantifiable feature data, providing an objective basis for judging rumination behavior. Thus, this application can output monitoring results through a preset classifier, realizing automated judgment of rumination behavior, improving monitoring efficiency, and is especially suitable for large-scale farming scenarios. The above technical solution of this application significantly improves the accuracy and efficiency of monitoring individual rumination behavior in cattle herds, meeting the basic needs of large-scale farming for rumination behavior monitoring.

[0052] As one possible embodiment of this application, combined with Figure 2 ,like Figure 3 As shown, step 202 above can be achieved through the following steps: Step 301: For the cow mouth region image in each image frame, divide the cow mouth region image into multiple sub-blocks.

[0053] The purpose of sub-block division is to achieve accurate detection of local occlusion. If the entire cow's mouth region is analyzed, it is easy to overlook small-area occlusions (such as cow hair obscuring the corners of the mouth). After dividing into sub-blocks, the occlusion situation can be analyzed block by block. For example, this application can evenly divide the cow's mouth region image into multiple sub-blocks (such as 4×4, a total of 16 sub-blocks). The number of sub-blocks can be adjusted according to the image size to ensure that each sub-block contains sufficient cow's mouth details without causing excessive computation due to its small size.

[0054] Step 302: For each sub-block, calculate the gradient magnitude matrix of the sub-block based on the gradient operator.

[0055] The gradient magnitude matrix includes the gradient magnitude of each pixel in the sub-block. The gradient operator is used to calculate the edge intensity (i.e., gradient magnitude) of a pixel.

[0056] It should be noted that ruminating cattle exhibit periodic horizontal or vertical chewing behavior, including both vertical and horizontal periodicity, while non-ruminating cattle show non-periodic chewing or large up-and-down head-raising while feeding. In images of the cow's mouth region without occlusion, the texture features and edge gradient changes in the mouth area remain within a certain range as the cow's normal physiological activities continue. However, when occlusion occurs by other cattle or objects, the texture features of the mouth region undergo truncated changes. For example, the gradient direction of pixels within the upper and lower contours of the mouth of a lying-down ruminating cow is horizontal. When occluded by the leg of another upright-walking cow, the gradient direction of pixels in the occluded area of ​​the mouth contour changes from horizontal to vertical. Alternatively, pixels inside the mouth that originally had a zero gradient magnitude may gain more non-zero gradient magnitudes and more gradient directions upon the appearance of the occluding object.

[0057] In other words, the occluded image will have more edge information due to the addition of information from other objects. Therefore, this application can construct the gradient magnitude matrix of each sub-block in the bull's mouth region image to characterize the gradient magnitude distribution of the bull's mouth region image, analyze the difference in gradient magnitude distribution between the occluded and unoccluded bull's mouth region images, and thus evaluate the occlusion situation of each bull's mouth region image.

[0058] Step 303: Based on the calculated gradient magnitude matrix, calculate the occlusion ratio of the bull's mouth region in each image frame.

[0059] For example, this application can determine the number of occluded pixels by analyzing the difference between the gradient magnitude matrix of the sub-block in the bull's mouth region image and the gradient magnitude matrix of the sub-block in the unoccluded bull's mouth region image, thereby determining the proportion of occluded pixels, i.e. the occlusion ratio of the bull's mouth region image.

[0060] Step 304: Take the image of the cow's mouth region in each image frame where the occlusion ratio is less than a preset ratio threshold as the target cow's mouth region image.

[0061] The preset ratio threshold is the critical value for determining whether the image of the cow's mouth region is valid.

[0062] In some embodiments, the preset proportion threshold is determined as follows: Multiple sets of bull mouth region image samples are acquired. For each set of bull mouth region image samples, the occlusion proportion corresponding to the bull mouth region image and the average occlusion proportion of the bull mouth region image samples are calculated based on the gradient magnitude matrix. Based on the distribution of the average occlusion proportions of the multiple sets of bull mouth region image samples, they are sorted according to the magnitude of the average occlusion proportion. The average occlusion proportion located at a preset position in the sorting result is selected as the preset proportion threshold. For example, the preset position can be set to the third quartile (i.e., the 75th percentile) in the sorting result to ensure that the occlusion degree of the target bull mouth region image after filtering is lower than that of most samples.

[0063] Based on the above technical solution, this application can capture local small-area occlusions by dividing the data into sub-blocks, avoiding invalid data from being ignored due to local occlusions entering subsequent steps. The gradient magnitude matrix of each sub-block is calculated based on the gradient operator to evaluate the gradient magnitude distribution of the cow mouth region image. According to the calculated gradient magnitude matrix, the occlusion ratio of the cow mouth region image in each image frame is calculated. Thus, the cow mouth region images with a low occlusion ratio are filtered out by the ratio threshold, providing high-quality data for subsequent rumination behavior recognition and significantly improving the accuracy and efficiency of monitoring individual rumination behavior in cattle.

[0064] As one possible embodiment of this application, combined with Figure 3 ,like Figure 4 As shown, step 303 above can be achieved through the following steps: Step 401: For each sub-block, determine the mean matrix and standard deviation matrix corresponding to the sub-block based on the gradient magnitude matrix of the corresponding sub-block in the bull's mouth region image of each image frame.

[0065] The mean matrix comprises the mean gradient magnitude of pixels in the corresponding sub-block of the cow's mouth region image in each image frame, and the standard deviation matrix comprises the standard deviation of the gradient magnitude of pixels in the corresponding sub-block of the cow's mouth region image in each image frame. The mean reflects the overall level of the gradient magnitude of pixels in the sub-block, while the standard deviation reflects the dispersion of the gradient magnitude.

[0066] For example, for any given pixel, this application can obtain the gradient magnitude of that pixel across multiple consecutive image frames based on the gradient magnitude matrix, and further calculate the mean and standard deviation to construct the mean matrix and standard deviation matrix corresponding to each sub-block. The number of image frames can be determined based on the duration of a single movement of the cow's mouth. In different embodiments, it can be set according to actual conditions. The longer the chewing time, the larger the number of image frames, thereby obtaining as many complete dynamic features as possible. For example, this application can set it to 120.

[0067] Step 402: Determine the standardization matrix of the sub-block based on the gradient magnitude matrix, the corresponding mean matrix, and the standard deviation matrix of the sub-block.

[0068] The normalization matrix is ​​used to characterize the degree of deviation of the gradient magnitude of each pixel in the sub-block from the standard reference value. Through normalization, the influence of the dimension of the gradient magnitude can be eliminated, so as to facilitate the horizontal analysis of different sub-blocks.

[0069] For example, the normalized matrix of a sub-block satisfies the following formula: in, For image frames Neutron block The standardized matrix, The coordinates of this sub-block in the image of the cow's mouth region are: For image frames Neutron block The gradient magnitude matrix, For sub-blocks The corresponding mean matrix, For sub-blocks The corresponding standard deviation matrix. It is composed of extremely small positive numbers (such as A matrix is ​​constructed to avoid the denominator being zero.

[0070] Step 403: For each sub-block, generate a binary mask based on the normalized matrix of the sub-block.

[0071] In a binary mask, the mask values ​​correspond one-to-one with the pixels of the sub-blocks, and the mask values ​​represent whether the corresponding pixels are occluded. A binary mask converts a normalized matrix into a binary matrix, with each value corresponding to whether a pixel is occluded. For example, a mask value of 0 indicates that the pixel is not occluded, and a mask value of 1 indicates that the pixel is occluded.

[0072] For example, the above standardization process can transform the gradient magnitude matrix of sub-blocks in a non-standard normally distributed cow mouth region image into a standardized matrix that follows a standard normal distribution. This application can calculate the significance level based on the stability of the observations (e.g., slight differences in color at the cow mouth contour of different individual cows). Values ​​to ensure that each element in the normalized matrix is ​​within the specified range. The probability within a certain value satisfies the significance level. The larger the probability value, the higher the precision but also the greater the computational cost. This application uses 0.05 as the threshold, i.e., determines a... The value makes each element in the normalized matrix... The probability of finding a value within the specified range is 95%. The calculation equation is as follows: in, The number of image frames. This represents the number of pixels in the sub-block. For image frames Neutron block pixels The element values ​​in the standardized matrix, As an indicator function, when the matrix All elements are less than or equal to When the condition is met, the function value is 1; otherwise, the function value is zero.

[0073] This application can be determined through the above methods. Value, and thus based on A binary mask is generated. The binary mask satisfies the following formula: in, For image frames Neutron block pixels The mask value in a binary mask. For image frames Neutron block pixels The element values ​​in the normalization matrix. When the normalization result of pixel u exceeds the distribution range of most elements in the normalization matrix obtained during normal bullsnout motion, it indicates that the gradient magnitude at pixel u deviates more from the standard normal distribution, and the greater the possibility of a significant change in gradient magnitude due to occlusion.

[0074] Step 404: Calculate the occlusion ratio of the cow mouth region image in each image frame based on the binary mask.

[0075] For example, the occlusion ratio corresponding to the cow's mouth region image satisfies the following formula: in, For image frames The occlusion ratio corresponding to the middle bull's mouth region image. This represents the number of pixels with a mask value of 1 in the cow's mouth region image. For image frames The number of pixels in the image of the cow's mouth region.

[0076] Based on the above technical solution, this application determines the mean matrix and standard deviation matrix of the corresponding sub-block in the bull's mouth region image of each image frame by using the gradient magnitude matrix. This allows for the evaluation of the overall level and dispersion of the gradient magnitude of each pixel in the continuous image. Furthermore, the gradient magnitude matrix is ​​standardized based on the mean matrix and standard deviation matrix, eliminating the influence of dimensions and unifying the occlusion judgment standard for different sub-blocks. In this way, this application can intuitively display the occlusion distribution through a binary mask, thereby achieving quantitative analysis of the occlusion ratio and providing a more accurate basis for subsequent target image screening, further improving the reliability of overall monitoring.

[0077] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0078] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. An intelligent monitoring method for individual rumination behavior in cattle, characterized in that, The method includes: Acquire cattle herd monitoring video data, and use a target recognition model to identify the mouth area of ​​cattle in the image frames of the cattle herd monitoring video data to obtain the mouth area image of each image frame. Occlusion detection is performed on the cow mouth region image in each image frame, and the target cow mouth region image is selected from the cow mouth region images in each image frame according to the obtained occlusion ratio; The selected target cow mouth region image is processed using an optical flow algorithm to generate an optical flow field sequence representing the movement of the cow mouth. The optical flow field sequence is input into a preset classifier to determine whether the cattle are ruminating and output the monitoring results of rumination behavior.

2. The intelligent monitoring method for individual rumination behavior in cattle according to claim 1, characterized in that, The step of performing occlusion detection on the cow mouth region image in each image frame, and selecting the target cow mouth region image from the cow mouth region images in each image frame according to the obtained occlusion ratio, includes: For each image frame, the cow mouth region image is divided into multiple sub-blocks; For each sub-block, the gradient magnitude matrix of the sub-block is calculated based on the gradient operator; the gradient magnitude matrix includes the gradient magnitude of each pixel in the sub-block; Based on the calculated gradient magnitude matrix, calculate the occlusion ratio of the cow's mouth region in each image frame; The image of the cow's mouth region in each image frame where the occlusion ratio is less than a preset ratio threshold is taken as the target cow's mouth region image.

3. The intelligent monitoring method for individual rumination behavior in cattle according to claim 2, characterized in that, The step of calculating the occlusion ratio of the bull's mouth region in each image frame based on the calculated gradient magnitude matrix includes: For each sub-block, the mean matrix and standard deviation matrix corresponding to the sub-block are determined based on the gradient magnitude matrix of the corresponding sub-block in the cow mouth region image of each image frame; the mean matrix includes the mean gradient magnitude of the pixels in the corresponding sub-block in the cow mouth region image of each image frame, and the standard deviation matrix includes the standard deviation of the gradient magnitude of the pixels in the corresponding sub-block in the cow mouth region image of each image frame. Based on the gradient magnitude matrix, the corresponding mean matrix, and the standard deviation matrix of the sub-block, the normalization matrix of the sub-block is determined; the normalization matrix is ​​used to characterize the degree of deviation of the gradient magnitude of each pixel in the sub-block from the standard reference value. For each sub-block, a binary mask is generated based on the normalized matrix of the sub-block; wherein, the mask value in the binary mask corresponds one-to-one with the pixel of the sub-block, and the mask value is used to characterize whether the corresponding pixel is occluded; The occlusion ratio of the bull's mouth region in each image frame is calculated based on the binary mask.

4. The intelligent monitoring method for individual rumination behavior in cattle according to claim 2, characterized in that, The preset ratio threshold is determined in the following way: Obtain multiple sets of image samples of the cow's mouth region; For each group of bull mouth region image samples, the occlusion ratio of the bull mouth region image in the bull mouth region image sample and the average occlusion ratio of the bull mouth region image sample are calculated according to the gradient magnitude matrix. Based on the distribution of the average occlusion ratio of multiple sets of bull mouth region image samples, they are sorted according to the magnitude of the average occlusion ratio, and the average occlusion ratio at a preset position in the sorting result is selected as the preset ratio threshold.

5. The intelligent monitoring method for individual rumination behavior in cattle according to claim 1, characterized in that, The process of processing the selected target cow mouth region image using an optical flow algorithm to generate an optical flow field sequence characterizing the movement of the cow mouth includes: The motion vectors of pixels in the corresponding target bull's mouth region image in consecutive image frames are calculated using an optical flow algorithm. An optical flow sequence of the target bull's mouth region image is generated based on the motion vectors of the pixels.

6. The intelligent monitoring method for individual rumination behavior in cattle according to claim 1, characterized in that, The optical flow field sequence is represented by a tensor consisting of a time dimension, a height dimension, a width dimension, and a channel dimension; wherein the time dimension represents the number of consecutive image frames, the height dimension and the width dimension are used to represent the size of the bull's mouth region image, and the channel dimension is used to represent the motion vectors in the horizontal and vertical directions.

7. The intelligent monitoring method for individual rumination behavior in cattle according to claim 1, characterized in that, The preset classifier is trained in the following way: Acquire training set data, which includes optical flow field sequences and labels for images of the cow's mouth region; the labels are used to characterize whether the cow corresponding to the image of the cow's mouth region is in rumination behavior. Using the training set data, a neural network algorithm is used to train a preset classifier to obtain a trained preset classifier; the neural network algorithm uses a binary classification loss function for classification.

8. The intelligent monitoring method for individual rumination behavior in cattle according to claim 1, characterized in that, The monitoring results of the rumination behavior include the image frame containing the image of the mouth region of the target cow in the rumination behavior.

9. The method according to claim 1, characterized in that, Before identifying the mouth region of cattle in the image frames of the cattle monitoring video data using a target recognition model to obtain the mouth region image in each image frame, the method further includes: Image filtering processing is performed on the image frames in the cattle herd monitoring video data.

10. An intelligent monitoring system for individual rumination behavior in cattle, characterized in that, The system includes: The cattle herd monitoring module is used to acquire cattle herd monitoring video data; The target recognition module is used to identify the mouth area of ​​cattle in the image frames of the cattle monitoring video data through the target recognition model, and obtain the mouth area image of each image frame. The monitoring video processing module is used to perform occlusion detection on the cow mouth region image in each image frame, and to filter out the target cow mouth region image from the cow mouth region image in each image frame according to the obtained occlusion ratio. The filtered target cow mouth region image is processed by optical flow algorithm to generate an optical flow field sequence characterizing the movement of the cow mouth. The monitoring result output module is used to input the optical flow field sequence into a preset classifier to determine whether the cattle are ruminating and output the monitoring results of rumination behavior.

Images