A computer vision-based method and system for monitoring individual rumination behavior in sheep flocks

Abstract

This invention relates to a computer vision-based method and system for monitoring sheep rumination behavior, comprising: S1: constructing an efficient monitoring system for acquiring video data of rumination behavior to obtain individual rumination behavior data of sheep in a flock; S2: preprocessing the video data to obtain individual data of multiple target sheep in the video; S3: extracting the mouth region data of sheep based on the Gunnar Farneback dense optical flow algorithm to obtain the motion area of ​​sheep rumination activity; S4: constructing a sheep rumination data model, obtaining the width variation pattern by extracting the width of the mouth motion area in each frame, and finally obtaining the rumination data of sheep. By marking the head position of each sheep, the acquisition of rumination data of multiple target ruminating sheep is achieved. This invention calculates the variation pattern of the mouth width of sheep during chewing activities in rumination using video data to obtain the rumination behavior pattern of the target. It has a simple structure, accurate rumination monitoring, and is more suitable for the requirements of animal welfare farming, providing a guarantee for rumination behavior monitoring.

Description

Technical Field

[0001] This invention relates to the field of modern intelligent animal husbandry, specifically to a method and system for monitoring individual rumination behavior in sheep flocks based on computer vision. Background Technology

[0002] In recent years, the domestic and international livestock industry has been developing from traditional models towards intelligent, precise, and large-scale operations. Driven by national policies strongly supporting herbivorous livestock farming, meat production has steadily increased, and consumption is also rising. Lamb, in particular, is increasingly favored for its tender fibers, high protein content, and low cholesterol. Traditional grazing methods are restricted by relevant laws and regulations due to their destructive impact on the grassland ecosystem. Therefore, standardized farms currently have significant development potential. However, a disadvantage of standardized farms is that penned sheep have relatively weak disease resistance, leading to more disease outbreaks and lower farm efficiency. Therefore, to improve enterprise efficiency and lamb quality, it is essential to promptly and accurately identify unhealthy sheep, reduce manual labor, and utilize modern science and technology to achieve automated sheep health monitoring.

[0003] Sheep behavior can be seen as a feedback mechanism resulting from the interaction between their internal physiological activities and the surrounding environment, providing the fastest and most direct evidence of their health status. Experimental studies have shown a close relationship between rumination data and the health of ruminants. Like cattle, sheep exhibit very typical rumination behavior; therefore, studying sheep rumination behavior is of great significance for sheep health. Healthy sheep ruminate continuously and vigorously, while sick sheep ruminate slowly and weakly, sometimes even ceasing altogether. The resumption of rumination after treatment is a key indicator of recovery. Therefore, by studying the patterns of sheep rumination behavior, we can gain early insight into their current health status. Real-time monitoring of sheep rumination behavior allows for the early detection of illness and the resolution of potential health risks before obvious pathological signs appear.

[0004] One of the most challenging issues in ruminant research has always been the monitoring of rumination behavior data. Traditionally, this has relied on manual observation by researchers, using counters, stopwatches, and other methods to monitor a sheep's rumination at specific times. With advancements in scientific sensor technology, computer vision, and various recognition algorithms, a small number of researchers have adopted automated detection devices to record sheep rumination behavior. These primarily involve contact-based rumination behavior detection, including pressure sensor-based, sound sensor-based, and accelerometer-based methods. Both methods involve attaching different sensors to the target animal. However, many of these monitoring devices are contact-based, fixed to or placed on the animal's head, and the overall system size is relatively large. Due to battery life and other issues, they are unsuitable for long-term carrying by livestock and are not well-suited for large-scale farming. This is not only because these intelligent monitoring methods and systems rely on various types of sensors, but also because portability is a significant problem, affecting normal rumination activity and, in severe cases, even stopping rumination altogether. To avoid these problems, some researchers are dedicated to the design and research of non-contact rumination behavior monitoring systems. Animal behavior monitoring methods based on video analytics can effectively reduce labor costs and are more intelligent, making them more suitable for large-scale farms. Lu Mingzhou et al.'s patent, "A Method for Monitoring Sheep Rumination Behavior Based on Acoustic Analysis," provides a method for sheep identification, but because it is a contact-based method, it is not conducive to the overall data collection of the sheep flock and is prone to causing stress responses to the target. Peng Dongqiao et al.'s patent, "A Non-Contact Monitoring Method for Ruminant Feeding Status," provides a method for monitoring the behavioral status of ruminants. It re-identifies and tracks ruminants based on acquired image data to obtain their behavioral status, but it only distinguishes rumination behavior from other behaviors and does not address the acquisition of rumination behavior data.

[0005] Therefore, how to monitor the rumination behavior of individual sheep in a non-contact manner has become an urgent problem to be solved. Summary of the Invention

[0006] The technical problem solved by this invention is to overcome the shortcomings of existing technologies and provide a method and system for monitoring individual rumination behavior in sheep flocks based on computer vision. By calculating the change pattern of the mouth width of sheep during chewing activities through video data, the rumination behavior pattern of the target can be obtained. The non-contact recognition method is simple in structure and accurate in rumination monitoring. Moreover, non-contact monitoring is more suitable for the requirements of animal welfare breeding and provides a data foundation for future research on rumination behavior monitoring.

[0007] The technical solution of this invention is: a method for monitoring the rumination behavior of individual sheep based on computer vision, comprising the following steps:

[0008] Step S1: Build a monitoring system to obtain video stream data of individual sheep rumination activities from daily behavior;

[0009] Step S2: Preprocess the video data in the video stream data obtained in Step S1. The preprocessing includes cropping the video data, framing the video data, and obtaining individual data of different target sheep in the video. The individual data includes head position coordinates and mouth width data.

[0010] Step S3: Use computer vision analysis technology to process two adjacent frames of video data to obtain mouth movement change data; use Gunnar Farneback dense optical flow algorithm to detect the movement area of ​​the sheep's head, calculate the optical flow vector generated by the movement of all pixels in the time domain between two frames, and then obtain the magnitude and angle of the optical flow vector. Then, perform threshold segmentation on the optical flow vector to realize the detection of the target moving parts. Finally, use different colors to encode the optical flow so that pixels with the same optical flow size are connected to obtain an optical flow map with multiple color blocks, thus completing the acquisition of mouth movement change data.

[0011] Step S4: Extract sheep rumination data based on mouth movement change data. First, preprocess the obtained optical flow image, and use the Canny algorithm to extract the contour of the model in each frame. On this basis, use the minAreaRect algorithm to find the minimum bounding rectangle and output the corner point data of the bounding rectangle. The difference is used to obtain the width data of each frame. Determining the width data of each frame is to obtain the mouth width change pattern of the target sheep during rumination. The chewing information of the sheep is obtained through this change pattern. By marking the head position of each sheep, the rumination data of multiple target ruminating sheep can be obtained, and finally the individual rumination data of the sheep flock is obtained.

[0012] Furthermore, step S1 specifically includes:

[0013] Step S11: Design the monitoring layout to obtain video data of rumination activities from daily behavior: By dividing the actual sheepfold, the monitoring unit that can be covered most clearly by the monitoring equipment or the combination of two adjacent monitoring units is obtained. In order to maximize the area of ​​the monitoring unit covered by the monitoring equipment, the monitoring equipment is used in combination of one or two units, and finally the monitoring layout with the largest overall monitoring coverage area is achieved, so that the activity range of the sheep in the sheepfold is covered by the equipment.

[0014] Step S12: Optimize the monitoring layout with the largest coverage area to obtain video data of rumination activities from daily behavior: To achieve the best monitoring effect, another monitoring layout perpendicular to the monitoring layout with the largest coverage area is designed in the opposite direction of each monitoring unit. This ensures that each position on the monitoring unit has a perpendicular monitoring view. By superimposing the two sets of monitoring devices, data from the front of the sheep can be obtained no matter how the sheep move. This optimizes the video monitoring layout with the largest coverage area and uses multi-source video streams to obtain video data of individual rumination behavior in the sheepfold, effectively solving the problem of sheep blocking each other in the sheepfold and making it impossible to efficiently obtain target daily behavior data.

[0015] Furthermore, step S2 specifically includes:

[0016] Step S21: Convert the video data exported from the monitoring system to MP4 format video, and trim the video to the required length.

[0017] Step S22: Convert the video into a continuous image; at the same time, read the first frame image and use the mouse interaction program to obtain the head position data and mouth width data of all sheep in the image, laying the foundation for subsequent multi-target sheep rumination monitoring.

[0018] The second step, step S3, specifically includes:

[0019] Step S31: Use the OpenCV platform to obtain mouth motion change data, and calculate the optical flow vector of all pixels in the temporal domain between two frames based on the Gunnar Farneback dense optical flow algorithm: by constructing a pyramid, setting the neighborhood range (i.e., the average window size), calculate the sum of matching errors in the neighborhood range of each point to obtain the optical flow of each point in the top layer image; feed the optical flow calculation result of the top layer back to the next layer as an estimate of the initial optical flow value of that layer; finally, pass it down to the lower layers of the pyramid in sequence, and obtain the accurate optical flow value of each pixel through multiple iterations. The calculation of the optical flow of each point in each layer is based on minimizing the sum of matching errors of all points in the neighborhood; the function used to calculate the optical flow is formula (1):

[0020] flow=cv.calcOpticalFlowFarneback(prev,next,flow,pyr_scale,levels,winsize,iterations,poly_n,poly_sigma,flags) (1)

[0021] In the formula, flow is the dense optical flow obtained after calculation, a two-channel array with optical flow size and method, prev is the previous frame image, next is the next frame image, pyr_scale is the scale of the image used to construct the pyramid, levels is the number of pyramid layers in the initial image, winsize is the average window size, iterations is the number of iterations, poly_n is the size of the pixel neighborhood, poly_sigma represents the standard deviation of Gaussians, and flags is the initial flow approximation.

[0022] Step S32: Calculate the size and angle of the dense optical flow, and perform threshold segmentation on the size of the optical flow to achieve detection of the part of the target with large movement, that is, the size of the optical flow exceeds the set value: The calculation result of formula (1) is to obtain an optical flow vector, which is a two-channel array containing the size and direction of the optical flow. The size and direction of the optical flow can be obtained by formula (2). The size of the optical flow is thresholded and only the optical flow above the set threshold is retained. Pixels below the set value are discarded.

[0023] magnitude,angle=cv.cartToPolar(x,y,magnitude,angle,angleInDegrees)(2)

[0024] Where x and y are coordinate arrays that are perpendicular to each other and of the same size, magnitude is an array that outputs the number of values ​​of the same size and type as x, Angle is an array that outputs the angles of the same size and type as x, and angleInDegrees represents the unit measurement of the angle.

[0025] Step S33: Encode the optical flow values ​​after threshold segmentation with different colors to achieve motion visualization processing and obtain mouth movement change data of rumination activity; Encode the optical flow values ​​in the result with different colors to achieve visualization of moving pixels, and finally obtain the image of the mouth movement change data of the connected regions with different colors.

[0026] Furthermore, step S4 specifically includes:

[0027] Step S41: Based on the mouth movement change data, sheep rumination data can be extracted. First, the image of the acquired mouth movement change data is flooded to facilitate the calculation and comparison of the area of ​​the connected region using the floodFill function; second, grayscale processing is performed to improve the processing speed; filtering is performed to eliminate noise mixed in during image digitization; binary thresholding and morphological opening operations are performed to obtain a more ideal edge contour of the mouth movement region.

[0028] Step S42: For the obtained edge contour data of the mouth movement area, in order to realize multi-target rumination monitoring, the head position data obtained in step S2 can be combined to crop different targets in the same image to obtain the edge contour of the mouth movement area of ​​different target sheep; for targets with relatively small edge contours, that is, targets with small imaging size in the actual image, size transformation processing is performed to adjust the size of the edge contour of the target mouth movement area; finally, the preprocessing work of mouth movement change data image is completed.

[0029] Step S43: For the image of the target sheep's mouth movement change data obtained after preprocessing, the Canny algorithm is used to detect the contour edges of the image. At the same time, the contourArea function is used to iteratively calculate the area of ​​all contours within the range. For the largest area, the minAreaRect function is used to find the minimum bounding rectangle of the contour. The bounding rectangle is calculated, and the four vertices of the minimum bounding rectangle are drawn using the boxPoints function. The coordinates of the vertices in the image are output. The difference and sum operation is performed on different vertices to obtain the width and length of the data of the region.

[0030] Step S44: By extracting sheep rumination data from each frame of the video, the mouth width data corresponding to each frame of the image data can be obtained. The width data is processed, that is, the width data that is lower than the set width indicates that the data extraction is incorrect, and it is discarded and the standard mouth width, that is, the original closed mouth width, is added. Then, all the data are plotted into a line graph, which can clearly show the pattern of chewing activity, the number of ruminations and the intensity of rumination.

[0031] The present invention provides a computer vision-based system for monitoring individual rumination behavior in sheep flocks, comprising a video data acquisition module, a sheep mouth area movement data acquisition module, and a sheep rumination data extraction module.

[0032] The video data acquisition module preserves the data from the sheep's daily behavior video monitoring system as a video stream. Based on actual needs, the video monitoring layout with the largest coverage area is designed first. On this basis, a solution to the problem of optimizing the monitoring effect is proposed. This is achieved by adding four sets of single-gun video acquisition devices in a symmetrical distribution, and finally obtaining the optimized full coverage monitoring layout. At the same time, a multi-source video stream approach is adopted to acquire monitoring video data of rumination activities from various daily behaviors.

[0033] The module for acquiring sheep mouth movement change data first preprocesses the monitoring video data, including video processing and extraction of required location data. Secondly, it uses the Gunnar Farneback algorithm's optical flow method as the mouth region model construction algorithm. The `flow` function in OpenCV is used to calculate the change of each pixel in the previous frame in the second frame, obtaining the optical flow vector of each pixel and the change data of the magnitude and direction of the optical flow in these two adjacent frames. By calculating the optical flow vector change data for each pixel, the movement of the sheep's body parts is detected. Different colors are used to encode the optical flow vector so that the magnitude of the optical flow generated by the movement can be clearly represented on the image, resulting in a visualized image of the sheep's mouth movement change data.

[0034] The module for extracting sheep rumination data first preprocesses the images of sheep mouth movement changes to separate the target from the background. Next, the Canny algorithm is used to extract the contours of the motion data images, identifying the target's outline. Finally, a minimum bounding box is drawn using a function, and the width data is obtained after calculating the vertex coordinates. Based on the width data of a mouth region obtained in each frame, the relationship between mouth width and time is obtained through the combination of consecutive different frames. This relationship allows us to determine the rumination status of the target sheep and obtain rumination data.

[0035] Compared with existing technologies, this invention has the following advantages: The method of this invention addresses the problem of monitoring sheep rumination behavior by proposing an efficient monitoring layout scheme for acquiring rumination activity videos. By optimizing a fully covered monitoring layout system and employing multi-source video stream technology, it effectively solves the problem of sheep blocking each other's view within the sheepfold, making it impossible to obtain daily behavioral data of the target. This invention calculates the variation pattern of the sheep's mouth width during chewing activities using video data to obtain the target's rumination behavior patterns, resulting in more fundamental and accurate rumination data. Unlike traditional contact-based indirect detection methods for rumination behavior, this invention's image data extraction method based on computer vision technology can directly obtain the patterns of rumination behavior without causing stress to the sheep, making it more suitable for the requirements of animal welfare farming. Furthermore, the system required by the method provided by this invention is smaller in scale and simpler in structure, thus exhibiting higher stability and stronger anti-interference capabilities. The acquired rumination data patterns are simple and clear, providing strong evidence for determining whether sheep are in a healthy state. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating a method for monitoring individual rumination behavior in a flock of sheep based on computer vision, as described in an embodiment of the present invention.

[0037] Figure 2This is a floor plan of the target sheep farm in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the monitoring area of ​​a single-gun surveillance camera in an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the monitoring area of ​​the dual-gun surveillance camera in an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of the full-coverage monitoring layout in an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram illustrating the addition of a single-gun monitoring device in an embodiment of the present invention;

[0042] Figure 7 This is a schematic diagram of the monitoring layout under the optimal conditions in an embodiment of the present invention;

[0043] Figure 8 This is a schematic diagram of the optimal layout for full monitoring coverage in an embodiment of the present invention;

[0044] Figure 9 This is a schematic diagram of the monitoring layout for full coverage and optimization in area IV of this invention.

[0045] Figure 10 This is a schematic diagram illustrating the results of obtaining relevant data in an embodiment of the present invention;

[0046] Figure 11 In this embodiment of the invention, mouth motion data is calculated based on the Gunnar Farneback dense optical flow algorithm;

[0047] Figure 12 This is a technical roadmap for extracting sheep rumination data in an embodiment of the present invention;

[0048] Figure 13 This is a schematic diagram of the image preprocessing process in an embodiment of the present invention;

[0049] Figure 14 This is a schematic diagram of the data extraction process in an embodiment of the present invention;

[0050] Figure 15 This is a flowchart of the data filtering function in an embodiment of the present invention;

[0051] Figure 16 This is a table of average values ​​for video surveillance effect parameters described in the embodiments of the present invention;

[0052] Figure 17 This is a table describing the average values ​​of sheep rumination monitoring results in embodiments of the present invention. Detailed Implementation

[0053] This invention provides a method for monitoring the rumination behavior of individual sheep in a sheepfold based on computer vision. It adopts a non-contact identification method, which is low-cost, high-precision, safe and effective, thereby avoiding the problems of low monitoring accuracy, high monitoring system cost and stress caused by existing contact technologies.

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0055] Example 1

[0056] Step S1: Construct a monitoring system to efficiently acquire video data of rumination activities from daily behavior. Design the monitoring layout and optimize the monitoring effect to obtain the optimal video monitoring layout scheme. Use multi-source video stream technology to ensure that the activity range of sheep in the sheepfold is covered by the equipment, so as to accurately acquire video data of sheep rumination behavior.

[0057] Step S2: Preprocess the video data, including format conversion and cropping of the video stream data, video frame processing, and obtaining individual data of different target sheep in the video through a mouse interaction program;

[0058] Step S3: Process two adjacent frames of images to obtain mouth movement change data. Use Gunnar Farneback dense optical flow algorithm to detect the sheep's movement area. Calculate the optical flow vector generated by the movement of all pixels in the time domain between the two frames, obtain the magnitude and angle of the optical flow vector, and perform threshold segmentation on the optical flow vector to achieve detection of areas with large target movements. Finally, use different colors to encode the optical flow vector to make it visual, and obtain the mouth region model of rumination activity.

[0059] Step S4: Extract sheep rumination data. First, preprocess the obtained mouth region model image. Use the Canny algorithm to extract the contour of each frame model. Based on this, find the minimum bounding rectangle based on minAreaRect and output the corner point data of the bounding rectangle. The difference can be used to obtain the width data of each frame. By sorting the data of each frame, the mouth width change pattern of the target sheep during rumination can be obtained. Through this pattern, the chewing information of the sheep can be obtained. By marking the head position of each sheep, the rumination data of multi-target ruminating sheep can be obtained, and finally the sheep rumination data is obtained.

[0060] In one embodiment, step S above involves: constructing a monitoring system to obtain video stream data of individual sheep rumination activities from daily behavior;

[0061] Step S11: Select monitoring equipment according to the actual monitoring area;

[0062] In this example, the main activity area of ​​the sheep flock is 9m × 10m, such as Figure 2 As shown. First, based on the determined monitoring range, select a surveillance camera that meets the requirements. In this example, a bullet camera with a focal length of 6mm is selected based on the monitoring range. Its monitoring angle is 50°, and the optimal monitoring range distance is 5-8m.

[0063] Step S12: Construct an efficient video data monitoring system for acquiring rumination activities to obtain individual rumination behavior data of sheep, design the monitoring layout and optimize the monitoring effect layout to obtain a full-coverage monitoring model;

[0064] First, the layout of the monitoring equipment is designed according to requirements. Full coverage is a necessary objective to capture every target within the sheep flock; therefore, full coverage of the monitored area is a crucial requirement for the video surveillance layout system. This is based on the monitoring area of ​​a single-lens camera (e.g.,...). Figure 3 As shown: (The arc from MN to PQ in the diagram represents the optimal monitoring range) and the monitoring area of ​​the dual-gun surveillance camera (e.g. Figure 4 By comparing with the above (as shown), the area of ​​the maximum monitored rectangle can be calculated using the following formula:

[0065] or

[0066] Where R represents the radius of the monitoring sector of the monitoring device, and α represents the central angle corresponding to the sector.

[0067] After calculating the two monitoring methods, two sets of dual-gun monitoring devices were finally adopted to achieve full coverage of the required monitoring area. Figure 5 As shown in the figure. The actual area is divided in the middle, using two groups located at O ​​respectively. M and O N The dual-gun surveillance camera monitoring system calculated that the area that meets the requirements for full monitoring coverage and is at the optimal monitoring distance accounts for more than 90% of the total monitoring area, with the furthest point being 10.96m away from the camera.

[0068] Secondly, having solved the problem of maximizing coverage area, the remaining issue is the monitoring effectiveness. This invention proposes the concept of "optimization," and to achieve this, it uses a reasonable increase in monitoring equipment. The following is an example. Figure 6 As shown, the actual area evaluation is divided into four parts, and single-gun monitoring equipment is added. The final optimized monitoring layout is as follows. Figure 7 As shown. Combining the full-coverage monitoring area model, the optimal full-coverage monitoring layout model is finally obtained, as follows. Figure 8 As shown, the two sets of dual-gun monitoring devices are O M With ON The four single-gun monitoring devices O1, O2, O3, and O4 theoretically meet the needs of this study, achieving full monitoring coverage. Furthermore, the area within the optimal monitoring zone (5-10m) accounts for 100% of the total area. Simultaneously, the same point is under both full coverage and optimal layout conditions, or the area covered by dual monitoring devices accounts for 85.8% of the total area. Even if the target is obstructed by one device, the other device can still observe normally. Multi-source video streams effectively solve the problem of target loss.

[0069] Step S13: Accurately obtain daily behavior data of sheep.

[0070] According to the design monitoring layout model and the monitoring equipment, this embodiment only studies one monitoring unit, using two monitoring devices to monitor area IV, such as... Figure 9 As shown. The actual installation height is 1.2 meters above the ground. Adjust the angle between the monitor and the bracket based on the actual image on the monitor so that the center of area IV is in the middle of the monitor, thus completing the acquisition of ruminant video data.

[0071] In one embodiment, step S2 above involves preprocessing the video data in the acquired video stream data. This preprocessing includes cropping the video data, frame conversion of the video data, and acquiring individual data of different target sheep in the video. The individual data includes head position coordinates and mouth width data. This completes the pre-processing of the video, specifically including:

[0072] Step S21: Format conversion and cropping of video stream data;

[0073] First, the video data stored in the recorder is processed by converting the Hikvision Smart265 bitrate to standard MP4 format and setting the video frame rate to 30 frames per second. In this embodiment, Leawo Prof.Media software is used. Second, in order to carry out subsequent processing and algorithm research, the video editing tools of this software can be used to obtain a shorter video stream. Finally, the preprocessed shorter video data is saved.

[0074] Step S22: Video frame conversion processing and acquisition of individual data of different target sheep in the video;

[0075] To understand video data, we need to start with each frame of the image. First, we need to perform frame processing on the cut video data, splitting it into individual frames for further study. In this invention, we call the VideoCapture constructor in OpenCV and set it to extract each frame.

[0076] Secondly, to achieve multi-target rumination detection later, some data needs to be acquired. This invention uses the `setMouseCallback` mouse callback function and the `puttext` function in OpenCV. Marking can be done manually by clicking the corresponding edge with the mouse, and the coordinates of the clicked point on the graph are output, such as... Figure 10 As shown, the video preprocessing is complete.

[0077] In one embodiment, step S3 above involves processing two adjacent frames of images to obtain mouth movement change data. The Gunnar Farneback dense optical flow algorithm is used to detect the sheep's movement region. The optical flow vectors generated by the temporal motion of all pixels between the two frames are calculated, yielding the magnitude and angle of the optical flow vectors. Thresholding is then applied to the optical flow vectors to detect the target moving parts. Finally, different colors are used to encode the optical flow, connecting pixels with the same optical flow magnitude to obtain an optical flow map with multiple color blocks, thus completing the acquisition of mouth movement change data. Specifically, this includes:

[0078] Step S31: Motion detection is performed on the sheep's mouth region based on the Gunnar Farneback dense optical flow algorithm. This algorithm calculates the optical flow vector of all pixels in the image, which is the instantaneous velocity change rate at a specific coordinate point in the two-dimensional image, representing the instantaneous velocity of pixel motion in the image, indicating the motion of a spatial object. In OpenCV, this invention calls the following function:

[0079] `flow = cv.calcOpticalFlowFarneback(prev, next, flow, pyr_scale, levels, winsize, iterations, poly_n, poly_sigma, flags)`, where `pyr_scale` is the image scale used to build the pyramid for each image, set to 0.5; `levels` is the number of pyramid levels for the initial image, set to 3; `winsize` is the average window size, set to 15; `iterations` is the number of iterations, set to 3; `poly_n` is the polynomial expansion size of the pixel neighborhood, set to 5; `poly_sigma` is the standard deviation of the Gaussian flow, set to 1.2; and `flags` is set to select the input stream as the approximation of the initial flow.

[0080] This invention constructs a three-layer pyramid, setting an average window size of 15 pixels in the image, which is the pixel neighborhood; calculates the sum of matching errors within the neighborhood of each pixel to obtain the optical flow of each point in the top layer image; feeds the optical flow calculation result of the top layer back to the next layer as an estimate of the initial optical flow value of that layer; finally, it is passed down to the lower layers of the pyramid in sequence, and through two iterations, the accurate optical flow value of each pixel neighborhood is obtained. The calculation of the optical flow of each point in each layer is based on minimizing the sum of matching errors of all points in the neighborhood, and finally obtains an optical flow vector that can represent the motion of each pixel neighborhood.

[0081] Step S32: Calculate the magnitude and angle of the optical flow vector by calling the function `magnitude, angle = cv.cartToPolar(x, y, magnitude, angle, angleInDegrees)`, where `x` and `y` represent two coordinate arrays of the same size and type, and `magnitude` represents an array that outputs the number of coordinates of the same size and type as `x`. `angle` represents an array of angles with the same size and type as `x`. angleInDegrees indicates that the angle is measured in radians.

[0082] Finally, by calculating the input optical flow vector, the magnitude and direction of motion in each pixel region can be obtained. By filtering the optical flow values ​​using a set threshold, pixel regions with small motion changes are removed, thus completing the motion extraction.

[0083] Step S33: To facilitate subsequent data processing, improve program speed, and accelerate convergence, the optical flow vector is normalized. This involves normalizing multiple pixels within a certain range with similar optical flow magnitudes, making incomparable data comparable while maintaining the relative relationship between the compared data. By assigning different colors to pixels with different changes after normalization, optical flow maps with various color block distributions are obtained, completing the acquisition of mouth movement change data. Finally, morphological closing operations are performed to obtain larger connected regions, which are the motion regions, such as... Figure 11 As shown.

[0084] Finally, by outputting and saving the data, we obtain an image of the mouth movement changes.

[0085] In one embodiment, step S4 above, extracting sheep rumination data, firstly, preprocesses the obtained mouth movement change data images, uses the Canny algorithm to extract the contour of each frame image, and then finds the minimum bounding rectangle based on minAreaRect, outputting the corner point data of the bounding rectangle. The difference is then used to obtain the width data of each frame. Organizing the data of each frame yields the mouth width change pattern of the target sheep during rumination. This pattern allows for the acquisition of the sheep's chewing information. By marking the head position of each sheep, rumination data of multiple target ruminating sheep is obtained, ultimately yielding the rumination data of each individual sheep. Specifically, this includes:

[0086] Step S41: Based on the obtained visualized mouth movement change data images, establish a detection model for the relevant muzzle region during sheep rumination, thereby realizing the detection and extraction of sheep rumination data. The technical route for sheep rumination data extraction in this invention is proposed to adopt the following... Figure 12 As shown.

[0087] Image data containing large movement areas of one or more sheep are preprocessed, including flooding, grayscale processing, filtering, and morphological processing. First, the `floodFill` function is used, setting the upper and lower limits of the mask to be incremented by 2, and outputting an unsigned integer.

[0088] Secondly, to facilitate subsequent processing and improve speed, the BGR three-channel format was converted into a GRAY single-channel grayscale image.

[0089] The grayscale image then needs to be processed and transformed in the spatial domain, mainly through filtering (convolution). The principle is to calculate a function based on the neighborhood I for each pixel X(x,y) in the image, as shown in equation (4-28) below:

[0090]

[0091] Where (x, y) is the position of pixel X, (k, l) is the position in the convolution kernel, L[x, y] is the filtering result of pixel X, f[k, l] is the weight parameter of the filtering function on (k, l), and I[x+k, y+l] is the pixel neighborhood value corresponding to f[k, l]. This invention uses smooth Gaussian filtering, with the function being GaussianBlur, and the Gaussian kernel size is set to 17, with both the standard deviation and standard deviation of the Gaussian kernel being zero.

[0092] The next step uses the threshold function, setting the threshold to 177 and the threshold type to THRESH_BINARY. The thresholding process uses a binary threshold to further distinguish the candidate region from the background.

[0093] Finally, morphological processing is performed. To remove interference from small pixel patches, an opening operation is used to process the image, effectively optimizing the target region and making the extracted range more ideal. The function used is `morphologyEx`, and the structuring element is `MORPH_OPEN` (opening operation), achieving the best image visual effect and converting it into a form more suitable for computer analysis and processing, thereby improving the image's useful information and application value.

[0094] Step S42: In order to achieve multi-target monitoring of sheep rumination behavior and reduce background interference other than the target sheep, the image after opening operation is geometrically processed by combining the position coordinates of the target sheep's head obtained in the previous section. The image is cropped using the function img[y0:y1,x0:x1], where the parameters represent the start and end coordinates of the y-axis and x-axis, respectively. The target position parameters obtained in step 2 are input to obtain a binary image of the target sheep's head region.

[0095] For distant targets, their models in the image will be relatively small. Therefore, it is necessary to scale the image to adjust its size and then perform interpolation to enlarge it. This can be done using the function cv2.resize, with the interpolation method set to bicubic interpolation and the corresponding interpolation function being cv2.INTER_CUBIC.

[0096] Finally, a binary image containing only the target of interest is obtained. A schematic diagram of this processing step is shown below. Figure 13 As shown.

[0097] Step S43: Based on the binary image of the pre-processed target sheep's head area mouth movement change data, the sheep's rumination data can be extracted by establishing a width detection model for the sheep's rumination mouth area. The width detection model mainly includes edge detection of the image, finding edge contours, drawing the minimum bounding rectangle of the contour, and calculating the width data through vertices.

[0098] First, the image data of the head region is read, which requires an edge detection algorithm. The Canny algorithm is adopted as the edge detection algorithm for this system, with the first hysteresis threshold set to 20 and the second hysteresis threshold set to 150.

[0099] Next, we begin finding the edge contour using the function `cv2.contourArea()`. Areas smaller than a certain value are discarded. For the contour with the largest area, we calculate the coordinates of the four vertices of its rotated bounding box using the function `box = np.int0(cv2.boxPoints())`. We then connect the vertex coordinates and draw them in the graph, forming the minimum bounding rectangle that encloses the largest contour, thus maximizing the extraction of data relative to the actual mouth.

[0100] Finally, the width and height of the outline can be obtained by calculating the distance between the four vertices. The output value represents the distance in the image, in pixels. Lines of different colors are then drawn, and the calculated data is written onto the original image. This completes the data extraction process. A schematic diagram of the data extraction process in this example is shown below. Figure 14 As shown.

[0101] Step S43: By extracting images of mouth movement changes, a width data point corresponding to each optical flow image is obtained. By calculating each frame of continuous video data, a series of mouth width data points can be obtained. The frequency of rumination in sheep can be determined by observing the pattern of width changes. Combined with the temporal information of the video stream, the rumination pattern of sheep can be obtained, providing a basis for judging sheep health.

[0102] Secondly, during the chewing interval, i.e., the interval between two consecutive rumination chewing activities, the muzzle is in a closed state, resulting in temporary stillness and causing abnormal data extraction. Additionally, movement of smaller areas such as the ears may occur during this time, potentially leading to monitoring errors. To address these two situations, for data with an extraction value of 0 or less than the normal muzzle width, this invention sets a data filtering function. The actual muzzle width of the target sheep is taken as the minimum value. When the detected data is lower than this initial value, the data is discarded, and the initial value is output; otherwise, the detected data is output. Figure 15 As shown.

[0103] In this embodiment of the invention, data was collected over several consecutive days from area IV of the daily activities of 32 local small-tailed Han sheep in a breeding base to test the monitoring layout. The results are as follows: Figure 16 As shown, the maximum monitoring rate (M, monitoring rate) and the maximum rumination monitoring rate (RM, rumination monitoring rate) are 100%, the minimum are 50% and 33.3% respectively, and the average monitoring rate and average rumination monitoring rate are 68.3% and 76.1% respectively. The feasibility and accuracy of the rumination behavior detection system are evaluated in this embodiment of the invention, and the results are as follows: Figure 17 As shown, the average detection accuracy (AR) is 92.2%, the average false positive rate (MR) and false negative rate (LR) are 9.1% and 10.5% respectively, and the overall average recognition rate (RR) is 74.8%, demonstrating the feasibility of this method in obtaining sheep rumination behavior.

[0104] This invention discloses a method for monitoring individual rumination behavior in sheep flocks based on computer vision. By optimizing a comprehensive monitoring layout system, it effectively solves the problem of sheep occlusion within the sheepfold during the acquisition of daily behavior data. Since the method provided by this invention is a non-contact detection method based on computer vision technology, it does not cause stress to the sheep. Unlike traditional contact-based rumination behavior detection methods, the image data extraction method provided by this invention does not require indirect acquisition of rumination behavior patterns. Instead, it obtains the rumination behavior patterns by acquiring changes in the width of the sheep's mouth during chewing, resulting in more essential and accurate rumination characteristics. Furthermore, the system required by this invention is relatively small in scale and simple in structure, thus exhibiting high stability and strong anti-interference capabilities. The acquired rumination data patterns are simple and clear, providing strong evidence for determining whether sheep are in a healthy state.

[0105] The above embodiments are provided merely for the purpose of describing the present invention and are not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications made without departing from the spirit and principles of the invention should be covered within the scope of the invention.

Claims

1. A computer vision-based method for monitoring individual ruminant behavior in a flock, characterized in that, The problem that sheep in the sheep house cannot obtain target daily behavior data due to mutual shielding can be effectively solved, the change rule of the mouth width of the sheep during chewing in the rumination activity is calculated through the video data, the target rumination behavior rule is obtained, more essential and more accurate rumination data are obtained, including: Step S1: constructing a monitoring system to obtain video stream data of individual rumination activities of the sheep group from daily behaviors; Step S2: pre-processing video data in the video stream data obtained in step S1, the pre-processing includes cropping of the video data, frame processing of the video data and obtaining individual data of different target sheep in the video; the individual data includes head position coordinates and mouth width data; Step S3: using computer vision analysis technology to process two adjacent images in the video data to obtain mouth movement change data; using Gunnar Farneback dense optical flow algorithm to detect the head movement area of the sheep, calculating the optical flow vector generated by the movement of all pixel points in the time domain between two frames, and then obtaining the size and angle of the optical flow vector, then performing threshold segmentation on the optical flow vector to realize detection of the target movement part, and finally using different colors to encode the optical flow to connect the pixel points with the same optical flow size, obtaining a light flow map with multiple color block distributions, and completing the acquisition of the mouth movement change data; Step S4: extracting sheep rumination data according to the mouth movement change data, first pre-processing the obtained light flow image, using Canny algorithm to extract the contour of each frame model, and based on this, finding the minimum circumscribed rectangle based on the minAreaRect algorithm and outputting the corner data of the circumscribed rectangle, and the difference between the two is the width data of each frame, which determines the change rule of the mouth width of the target sheep during the rumination activity, and through the change rule, the chewing information of the sheep is obtained, and the head positions of each sheep are marked to realize the rumination data acquisition of multiple target rumination sheep, and finally the individual rumination data of the sheep group is obtained.

2. The method according to claim 1, wherein, The step S1 specifically includes: Step S11: designing a monitoring layout to obtain video data of existing rumination activities from daily behaviors: by dividing the actual sheep house, the most clear monitoring unit or the combination of two adjacent monitoring units covered by the monitoring device is obtained, so that the monitoring coverage area is maximized, and the monitoring device is one or two combinations, finally achieving the monitoring layout with the maximum overall monitoring coverage area, so that the activity range of the sheep in the sheep house is covered by the device. Step S12: optimizing the largest coverage monitoring layout to efficiently obtain the video data of the existence of the rumination activity from the daily behavior: in order to achieve the optimization of the monitoring effect, another set of monitoring layouts perpendicular to the largest coverage monitoring layout is designed in the other direction of each monitoring unit, so that each position on the monitoring unit has a monitoring visual angle perpendicular to each other, through the superposition of the two parts of the monitoring device, the data of the front of the sheep is obtained regardless of the movement of the sheep, the largest coverage video monitoring layout is optimized, and the problem of inefficient acquisition of target daily behavior data due to mutual shielding between the sheep in the sheep house is effectively solved; the multi-source video stream is used to realize the acquisition of the individual rumination behavior video data in the sheep house.

3. The method of computer vision based monitoring of individual ruminant behavior in a flock according to claim 2, wherein, The step S2 specifically comprises: Step S21: format conversion is performed on the video data derived from the monitoring system to MP4 format video, and the video is cropped to obtain the required length; Step S22: the video is converted into continuous images; meanwhile, the first frame image is read, and the head position data and mouth width data of all the sheep in the image are obtained by using a mouse interaction program, thereby laying a foundation for subsequent multi-target sheep rumination monitoring.

4. The method for computer vision based monitoring of individual ruminant behavior in a flock as claimed in claim 1, wherein, The step S3 specifically comprises: Step S31: the mouth movement change data is obtained by using the OpenCV platform, and the optical flow vector generated by the time domain motion of all pixel points between two frames is calculated based on the Gunnar Farneback dense optical flow algorithm: by constructing a pyramid, setting the neighborhood range, i.e., the average window size, the matching error sum in the neighborhood range of each point is calculated to obtain the optical flow of each point in the top layer image; the calculation result of the optical flow of the top layer is fed back to the next layer as the estimation of the initial optical flow value of the layer; finally, the calculation is sequentially passed to the lower layer of the pyramid, and the accurate optical flow value of each pixel point is obtained through multiple iterations, and the calculation of the optical flow of each point of each layer is based on the minimization of the matching error sum of all points in the neighborhood; the function used to calculate the optical flow is formula (1): （1） In the formula, flow is the dense optical flow obtained after calculation, which is a double-channel array with the size and method of the optical flow, prev is the previous frame image, next is the next frame image, pyr_scale is the image ratio of the constructed pyramid, levels is the number of pyramid layers of the initial image, winsize is the average window size, iterations is the number of iterations, poly_n is the size of the pixel neighborhood, poly_sigma represents the standard deviation of the Gaussian, and flags is the initial flow approximation value; Step S32: the size and angle of the dense optical flow are calculated, and the size of the optical flow is threshold segmented to detect the part with large target motion, i.e., the optical flow size exceeding the set value: the calculation result of formula (1) is a dense optical flow F, which is a double-channel array containing the size and direction of the optical flow, the size and direction of the optical flow are calculated by formula (2), the size of the optical flow is threshold segmented, only the optical flow above the set threshold is retained, and the pixel points below the set value are discarded. (2) Wherein, x, y are the direction perpendicular, the same size of the coordinate array, magnitude is the output of the same size and type of the number of arrays, Angle is the output of the same size and type of angle array, angleInDegrees represents the unit measurement of angle; Step S33: using different colors to encode the threshold segmented optical flow value to realize the visualization processing of the motion, and obtaining the mouth movement change data of the rumination activity; using different colors to encode the optical flow value in the result, realizing the visualization of the motion pixel points, and finally obtaining the connected regions with different colors as the image of the obtained mouth movement change data.

5. The method for computer vision based monitoring of individual ruminant behavior in a flock as claimed in claim 1, wherein, The step S4 specifically comprises: Step S41: According to the mouth movement change data, the sheep's rumination data can be extracted. First, the acquired mouth movement change data image is subjected to flood processing, which facilitates the calculation and comparison of the connected region area, and the function is used. Secondly, the gray scale processing is carried out to improve the processing speed; the filter processing is carried out to eliminate the noise mixed in the image digitization; the binary threshold and the morphological open operation processing are carried out to obtain the edge profile of the relatively ideal mouth movement region. Step S42: for the obtained edge contour data of the mouth movement region, in order to realize the multi-target rumination monitoring, the head position data obtained in step S2 can be combined to crop different targets in the same image, so as to obtain the edge contour of the mouth movement region of different target sheep; for the edge contour which is relatively small, i.e. the target which is relatively small in the actual image, size transformation processing is performed to adjust the size of the edge contour of the target mouth movement region; finally, the preprocessing work of the mouth movement change data image is completed; Step S43: using Canny algorithm to detect the contour edge of the image of the target sheep mouth movement change data obtained after preprocessing, and using The function loop calculates the area of all contours in the range, and uses the function minAreaRect to find the minimum bounding rectangle of the contour with the largest area. Calculate the bounding rectangle, and use the function Draw the four vertices of the minimum bounding rectangle, and output the coordinates of the vertices in the graph. The difference and operation of different vertices obtains the width and length of the region data. Step S44: by extracting the sheep rumination data from each frame of data of the video, the mouth width data corresponding to each frame of image data can be obtained, the width data is processed, the width data lower than the set width is removed and the standard mouth width is supplemented, i.e. the original closed width, and then all the data are drawn into a broken line graph, so that the rules of chewing activity, the number of rumination and the strength of rumination can be clearly obtained.

6. A computer vision based system for monitoring individual ruminant behavior in a flock implementing the method of any one of claims 1-5, characterized in that, The method comprises a video data acquisition module, a sheep mouth region movement data acquisition module and a sheep rumination data extraction module; The video data acquisition module retains the data in the form of video stream from the sheep daily behavior video monitoring system, according to the actual needs, first designs the video monitoring layout with the largest coverage area, on this basis, proposes a solution to the optimization problem of monitoring effect, realizes by increasing four groups of single gun type video acquisition devices in a symmetrical distribution, and finally obtains the monitoring full coverage layout under the optimization; at the same time, the multi-source video stream is used to realize the acquisition of the monitoring video data of the existing rumination activity from various daily behaviors; The sheep mouth movement change data acquisition module first needs to pre-process the monitoring video data, including video processing and extracting the required position data, and secondly uses the optical flow method based on the Gunnar Farneback algorithm as a mouth region model construction algorithm, uses the The function calculates the change of each pixel point in the previous frame image on the two adjacent frames on the second frame, can obtain the optical flow vector of each pixel point, obtains the change data of the optical flow size and direction of the pixel point on the two adjacent frames, and obtains the optical flow vector change data of the frame through the calculation of each pixel point, realizes the detection of the movement part of the sheep, encodes the optical flow vector using different colors to make the optical flow size generated by the movement can be clearly represented on the image, and obtains the visualized sheep mouth movement change data image. The sheep rumination data extraction module first preprocesses the sheep mouth movement change data image to separate the target from the background; secondly, the contour extraction is performed on the movement data image through the Canny algorithm to identify the contour line of the target; finally, the minimum rectangular frame is drawn through the function to obtain the vertex coordinate calculation and obtain the width data; according to the width data of one mouth region obtained from each frame, the relationship between the mouth width data and the time is obtained through the combination of continuous different frames, the rumination data is obtained, and the rumination condition of the target sheep is obtained, and the rumination data is obtained.

Images