A sheep behavior recognition method and system based on machine vision

Abstract

The application belongs to the technical field of livestock behavior recognition, and provides a sheep behavior recognition method and system based on machine vision, which comprises the following steps: data acquisition, image extraction, region division, long-time sequence position data construction, region type division, node recognition, node correction, long-time sequence action data construction and behavior recognition; the application improves the recognition response speed and efficiency of the model by using image extraction-based data for model analysis; the quality of the model input data and the recognition accuracy of the model are improved by dividing the collected images into regions of interest, matrix mapping and node data extraction; the reliability of the model and the dimension of the data are improved by setting multiple cameras.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of livestock behavior recognition, in particular to a sheep behavior recognition method and system based on machine vision. BACKGROUND

[0002] Common sheep behavior recognition methods include single camera visual monitoring, wearable sensor monitoring, and manual observation recording. Single camera visual monitoring relies on a single fixed camera to collect video data, and extracts the target area through background subtraction or simple threshold segmentation; wearable sensor monitoring uses GPS collars, accelerometers or RFID tags and other devices to indirectly infer behavior through sensor data; manual observation recording relies on pasture workers to regularly patrol and manually record sheep behavior.

[0003] The existing technology has high cost and time consumption. The current sheep behavior recognition model needs to face a large amount of image input and needs to use a large amount of computing power for recognition, resulting in high cost and low efficiency. The existing technology often does not preprocess the data, often directly inputs the model, resulting in redundant useless data and poor model training effect. In addition, the existing technology does not consider multi-camera collaborative correction, resulting in large positioning error, does not intelligently filter video frames, and is susceptible to interference such as blur, uneven illumination, or motion artifacts. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a sheep behavior recognition method and system based on machine vision, which solves the problems of low efficiency and high cost of the prior art.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] A sheep behavior recognition method based on machine vision, comprising:

[0007] Based on a preset height and an inclination angle, a plurality of cameras are fixed at the edge position of a target monitoring area, and the cameras are used to monitor target sheep in the target monitoring area to obtain video stream data;

[0008] According to a fixed frequency, frame image filtering and extraction are performed on the video stream data in each time window to obtain a set of to-be-detected images;

[0009] Contour segmentation is performed on the set of to-be-detected images to obtain a plurality of candidate regions, similarity calculation is performed on each candidate region to obtain a region similarity, and the candidate region whose region similarity meets a similarity standard is determined as a region of interest;

[0010] The target monitoring area is divided into grid images according to a preset resolution. The region of interest is mapped using a preset transformation matrix to obtain sheep location data. The sheep location data and frame image time data are fused in frame time order to obtain long-time location data.

[0011] The grid image is divided into regions to obtain the region type data of the sheep corresponding to the sheep location data;

[0012] The target body nodes within the candidate region are identified using a pre-built part recognition model to obtain sheep part images and node location data.

[0013] Determine the torso angle of the sheep in the candidate region, and use the torso angle to correct and fuse all the node position data at the same time to obtain node correction data;

[0014] By fusing the node correction data and the frame image time data in frame time order, long-time action data is obtained.

[0015] The long-term location data, the node location data, and the long-term action data are input into a sheep behavior recognition model pre-trained based on a gated recurrent unit network for detection, and sheep behavior recognition results are obtained. The sheep behavior recognition results include: feeding behavior, rumination behavior, wandering behavior, resting behavior, drinking behavior, excretion behavior, and mating behavior.

[0016] Preferably, it further includes:

[0017] Extract the sheep's face image and abdomen image from the sheep's body parts image;

[0018] Using a pre-constructed image library of sick sheep faces and abdomens, state matching is performed on the sheep's face images and the sheep's abdomen images to obtain matching degree data. If the maximum value of the matching degree data exceeds a preset matching threshold, the state label corresponding to the maximum value is determined as the state label of the sheep's part image.

[0019] Based on the sheep behavior recognition results, statistics on sheep feeding, rumination, wandering, resting, drinking, and excretion are obtained. The sheep behavior statistics include: feeding time data, feeding frequency data, rumination frequency data, wandering frequency data, resting frequency data, drinking frequency data, and excretion frequency data.

[0020] Based on the pre-collected behavioral data of sick sheep, the k-nearest neighbor technique is used to classify the statistical data of sheep behavior, and the results of sheep disease detection and sheep disease type detection are obtained.

[0021] Preferably, the video stream data within each time window is subjected to frame image filtering and extraction at a fixed frequency to obtain a set of images to be detected, including:

[0022] The video stream data is segmented according to the fixed frequency to obtain multiple video sub-segments;

[0023] The preset time of the beginning of each video sub-segment is determined as the time window;

[0024] The frame quality scoring formula is used to calculate the frame quality score for each frame image within the time window, resulting in a frame quality score dataset; the frame quality score formula is:

[0025]

[0026] Where, f1=∑ (x,y) |G(x,y)|;

[0027] Q is the calculated value of the frame quality scoring formula; f1, f2, f3, and f4 are the sharpness factor, illumination factor, motion blur factor, and content integrity factor, respectively; W1, W2, W3, and W4 are the sharpness weight, illumination weight, motion blur weight, and content integrity weight, respectively; ∈ is the smoothing coefficient; Γ(C) is the constraint function, which outputs 0 if the current frame image meets the preset rejection condition, and 1 otherwise; G(x,y) represents the convolution of the Laplacian operator at pixel (x,y); μ L This represents the average value of the image's brightness channel. H represents the brightness variance; F(I) is the result of the Fast Fourier Transform of the image; H represents the brightness variance. high For ideal high-pass filters;

[0028] The frame image corresponding to the highest score in the frame scoring dataset is used as the representative image of the video sub-segment. The representative images of all the video sub-segments are integrated to obtain the image set to be detected.

[0029] Preferably, the image set to be detected is segmented to obtain several candidate regions. A similarity calculation is performed on each candidate region to obtain a region similarity. Candidate regions whose region similarity meets the similarity criteria are determined as regions of interest, including:

[0030] The Canny algorithm is used to perform edge extraction and image segmentation on the set of images to be detected to obtain the candidate regions.

[0031] After the candidate region is size-normalized, the similarity value between the candidate region and each contour image in the preset livestock contour library is calculated using the cosine similarity formula, and the largest similarity value is determined as the region similarity of the candidate region.

[0032] Preferably, the target monitoring area is divided into grid images according to a preset resolution, and the region of interest is mapped using a preset transformation matrix to obtain sheep location data. The sheep location data and frame image time data are then fused in frame time order to obtain long-term time-series location data, including:

[0033] Based on the fixed position of the camera, the target monitoring area is expanded into a rectangular area, and the rectangular area is divided into grid areas of the same size according to the preset resolution to obtain the grid image;

[0034] The image centroid coordinates of the region of interest are converted into grid coordinates of the grid image using the transformation matrix to obtain the mapped coordinates; the expression of the transformation matrix is:

[0035]

[0036] Where M is the transformation matrix; h is the preset height; θ is the tilt angle; f x f y These are the equivalent pixel values ​​of the camera's focal length along the X and Y axes of the image coordinate system, respectively; c x c y These are the principal points of the image along the X and Y axes of the image coordinate system, respectively.

[0037] Calculate the minimum circumcircle of the mapped coordinates of all the cameras, and use the grid number of the center of the minimum circumcircle as the sheep location data of the region of interest.

[0038] Preferably, the area type data for sheep includes: drinking area, feeding area, resting area, and exercise area.

[0039] Preferably, the training process of the sheep behavior recognition model includes:

[0040] The pre-collected labeled long-time position data, node position data, and long-time action data are subjected to noise removal, normalization, and temporal alignment to obtain standardized input data.

[0041] The standardized input data is fed into a pre-built bidirectional GRU network for computation to obtain the network output;

[0042] The total loss value of the network output is calculated using weighted cross-entropy loss.

[0043] Based on the total loss value, the bidirectional GRU network is iterated using the Adam optimizer and chord annealing strategy to obtain the trained sheep behavior recognition model.

[0044] Preferably, the matching degree data is SSIM data calculated using ffmpeg.

[0045] Preferably, the torso angle of the sheep in the candidate region is determined, and the torso angle is used to correct and fuse all the node position data at the same time to obtain node correction data, including:

[0046] Principal component analysis was used to determine the torso angle of the candidate region.

[0047] A correction matrix is ​​constructed based on the torso angle; the expression for the correction matrix is:

[0048]

[0049] Wherein, C is the correction matrix; α is the torso angle;

[0050] The node position data is corrected using the correction matrix to obtain corrected node position data;

[0051] The node correction position data corresponding to the same node are integrated into a group, and the corresponding node correction position data corresponding to the same node are merged in chronological order to obtain the node correction data.

[0052] Preferably, a machine vision-based sheep behavior recognition system includes:

[0053] The data acquisition module is used to monitor the target sheep using the camera and obtain the video stream data;

[0054] The image extraction module is used to filter and extract frame images of the video stream data within each time window according to a fixed frequency to obtain the set of images to be detected;

[0055] The region segmentation module is used to perform contour segmentation on the image set to be detected to obtain a number of candidate regions, calculate the similarity of each candidate region to obtain the region similarity, and determine the candidate regions whose region similarity meets the similarity criteria as the region of interest.

[0056] The location fusion module is used to divide the target monitoring area into the grid image according to the preset resolution, map the region of interest using the transformation matrix to obtain sheep location data, and fuse the sheep location data and the frame image time data in frame time order to obtain the long time series location data.

[0057] The region type division module is used to divide the grid image into region types to obtain the region type data where the sheep are located;

[0058] The node recognition module is used to identify target body nodes within the candidate region using the part recognition model, thereby obtaining the sheep part image and the node location data.

[0059] The node correction module is used to determine the torso angle of the sheep in the candidate region, and to correct and fuse all the node position data at the same time using the torso angle to obtain the node correction data.

[0060] The motion fusion module is used to fuse the node correction data and the frame image time data in frame time order to obtain the long time sequence motion data;

[0061] The behavior recognition module is used to input the long-time position data, the node position data, and the long-time action data into the sheep behavior recognition model for detection, and obtain the sheep behavior recognition result.

[0062] The present invention discloses the following technical effects:

[0063] This invention provides a machine vision-based sheep behavior recognition method and system. By utilizing image-extracted data for model analysis, it solves the problems of increased computing power and redundant invalid data caused by directly inputting images and other data in existing technologies, achieving feature extraction and data volume reduction from image data. By performing region of interest division, matrix mapping, and node data extraction on the acquired images, it solves the problem of low model recognition accuracy caused by directly inputting acquired data in existing technologies, achieving the filtering and optimization of image-extracted data. By setting up multiple cameras, it overcomes the shortcomings of single cameras being prone to errors, achieving comprehensive processing of multi-dimensional data. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 A schematic diagram of a machine vision-based sheep behavior recognition process provided in an embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of the sick sheep identification process provided in an embodiment of the present invention;

[0067] Figure 3 This is a schematic diagram of the process for acquiring the set of images to be detected provided in an embodiment of the present invention;

[0068] Figure 4 This is a schematic diagram of the region of interest delineation process provided in an embodiment of the present invention;

[0069] Figure 5 This is a schematic diagram of the long-time-series location data construction process provided in an embodiment of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] The purpose of this invention is to provide a machine vision-based method and system for sheep behavior recognition, which solves the problems of low efficiency and high cost in existing technologies.

[0072] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Figure 1 This is a schematic diagram of a machine vision-based sheep behavior recognition process provided in an embodiment of the present invention, as shown below. Figure 1 As shown, this invention provides a machine vision-based method for sheep behavior recognition, comprising:

[0074] Step 100: Fix multiple cameras at the edge of the target monitoring area based on preset height and tilt angle, and use the cameras to monitor the target sheep in the target monitoring area to obtain video stream data;

[0075] Step 200: Filter and extract frame images from the video stream data within each time window according to a fixed frequency to obtain the set of images to be detected;

[0076] Step 300: Perform contour segmentation on the image set to be detected to obtain several candidate regions. Calculate the similarity of each candidate region to obtain the region similarity. Then, determine the candidate regions whose region similarity meets the similarity criteria as regions of interest.

[0077] Step 400: Divide the target monitoring area into grid images according to the preset resolution, map the region of interest using the preset transformation matrix to obtain sheep location data, and fuse the sheep location data and frame image time data in frame time order to obtain long time series location data.

[0078] Step 500: Divide the grid image into regions to obtain the region type data of the sheep corresponding to the sheep location data;

[0079] Step 600: Use the pre-built part recognition model to identify the target body nodes in the candidate area to obtain sheep part images and node location data;

[0080] Step 700: Determine the trunk angle of the sheep in the candidate region, and use the trunk angle to correct and fuse all node position data at the same time to obtain node correction data;

[0081] Step 800: Fuse node correction data and frame image time data in frame time order to obtain long-time sequence motion data;

[0082] Step 900: Input long-time location data, node location data, and long-time action data into a sheep behavior recognition model pre-trained based on a gated recurrent unit network for detection, and obtain sheep behavior recognition results; the sheep behavior recognition results include: feeding behavior, rumination behavior, wandering behavior, resting behavior, drinking behavior, excretion behavior, and mating behavior.

[0083] refer to Figure 2 It also includes:

[0084] Step 1000: Extract the sheep's face image and abdomen image from the sheep body parts image;

[0085] Step 1100: Use the pre-constructed facial image set and abdominal image library of sick sheep to perform state matching on the sheep's facial images and sheep's abdominal images to obtain matching degree data. If the maximum value of the matching degree data exceeds the preset matching threshold, the state label corresponding to the maximum value is determined as the state label of the sheep part image.

[0086] Step 1200: Based on the sheep behavior recognition results, statistical data on sheep feeding, rumination, wandering, resting, drinking, and excretion are obtained. The sheep behavior statistical data includes: feeding time data, feeding frequency data, rumination frequency data, wandering frequency data, resting frequency data, drinking frequency data, and excretion frequency data.

[0087] Step 1300: Based on the pre-collected behavioral data of sick sheep, the k-nearest neighbor technique is used to classify the statistical data of sheep behavior to obtain the detection results of sheep diseases and the detection results of sheep disease types.

[0088] refer to Figure 3 Frame images are filtered and extracted from video stream data within each time window at a fixed frequency to obtain a set of images to be detected, including:

[0089] Step 201: Segment the video stream data at a fixed frequency to obtain multiple video sub-segments;

[0090] Step 202: Determine the preset time of the beginning of each video segment as the time window;

[0091] Step 203: Calculate the frame quality score for each frame within the time window using the pre-constructed frame quality scoring formula to obtain the frame quality score dataset; the frame quality scoring formula is:

[0092]

[0093] Where, f1=∑ (x,y) |G(x,y)|;

[0094] Q is the calculated value of the frame quality scoring formula; f1, f2, f3, and f4 are the sharpness factor, illumination factor, motion blur factor, and content integrity factor, respectively; W1, W2, W3, and W4 are the sharpness weight, illumination weight, motion blur weight, and content integrity weight, respectively; ∈ is the smoothing coefficient; Γ(C) is the constraint function, which outputs 0 if the current frame image meets the preset rejection condition, and 1 otherwise; G(x,y) represents the convolution of the Laplacian operator at pixel (x,y); μ L This represents the average value of the image's brightness channel. H represents the brightness variance; F(I) is the result of the Fast Fourier Transform of the image; H represents the brightness variance. high For ideal high-pass filters;

[0095] Step 204: Take the frame image corresponding to the highest score in the frame scoring dataset as the representative image of the video sub-segment, and integrate the representative images of all video sub-segments to obtain the image set to be detected.

[0096] refer toFigure 4 Contour segmentation is performed on the image set to be detected to obtain several candidate regions. Similarity is calculated for each candidate region to obtain region similarity. Candidate regions whose region similarity meets the similarity criteria are identified as regions of interest, including:

[0097] Step 301: Use the Canny algorithm to perform edge extraction and image segmentation on the set of images to be detected to obtain candidate regions;

[0098] Step 302: After standardizing the size of the candidate region, the similarity value between the candidate region and each contour image in the preset livestock contour library is calculated using the cosine similarity formula, and the largest similarity value is determined as the region similarity of the candidate region.

[0099] refer to Figure 5 The target monitoring area is divided into grid images according to a preset resolution. A preset transformation matrix is ​​used to map the region of interest to obtain sheep location data. The sheep location data and frame image time data are then fused in frame time order to obtain long-term time-series location data, including:

[0100] Step 401: Expand the target monitoring area into a rectangular area based on the fixed position of the camera, and divide the rectangular area into grid areas of the same size according to the preset resolution to obtain a grid image;

[0101] Step 402: Use a transformation matrix to convert the image centroid coordinates of the region of interest into grid coordinates of the grid image, obtaining the mapped coordinates; the expression for the transformation matrix is:

[0102]

[0103] Where M is the transformation matrix; h is the preset height; θ is the tilt angle; f x f y These are the equivalent pixel values ​​of the camera's focal length along the X and Y axes of the image coordinate system, respectively; c x c y These are the principal points of the image along the X and Y axes of the image coordinate system, respectively.

[0104] Step 403: Calculate the minimum circumcircle of the mapped coordinates of all cameras, and use the grid number of the center of the minimum circumcircle as the sheep location data of the region of interest.

[0105] Preferably, the area type data for sheep includes: drinking area, feeding area, resting area, and exercise area.

[0106] Furthermore, the training process for the sheep behavior recognition model includes:

[0107] Noise removal, normalization, and temporal alignment are performed on the pre-collected labeled long-term location data, node location data, and long-term action data to obtain standardized input data.

[0108] Standardized input data is fed into a pre-built bidirectional GRU network for computation, and the network output is obtained.

[0109] The total loss value of the network output is calculated using weighted cross-entropy loss.

[0110] Based on the total loss value, the bidirectional GRU network is iterated using the Adam optimizer and chord annealing strategy to obtain the trained sheep behavior recognition model.

[0111] Preferably, the matching degree data is SSIM data calculated using ffmpeg.

[0112] Specifically, the torso angles of sheep in the candidate region are determined, and the torso angles are used to correct and fuse all node position data at the same time to obtain node correction data, including:

[0113] Principal component analysis was used to determine the torso angle of the candidate region.

[0114] A correction matrix is ​​constructed based on the torso angle; the expression for the correction matrix is:

[0115]

[0116] Where C is the correction matrix; α is the torso angle;

[0117] The node position data is corrected using a correction matrix to obtain the corrected node position data;

[0118] The node correction position data corresponding to the same node are integrated into a group, and the corresponding node correction position data of the same node are merged in chronological order to obtain node correction data.

[0119] Furthermore, a machine vision-based sheep behavior recognition system includes:

[0120] The data acquisition module is used to monitor the target sheep using a camera and obtain video stream data;

[0121] The image extraction module is used to filter and extract frame images from video stream data within each time window at a fixed frequency to obtain a set of images to be detected.

[0122] The region segmentation module is used to perform contour segmentation on the image set to be detected, obtain several candidate regions, calculate the similarity of each candidate region, obtain the region similarity, and determine the candidate regions whose region similarity meets the similarity criteria as regions of interest.

[0123] The location fusion module is used to divide the target monitoring area into grid images according to a preset resolution, map the region of interest using a transformation matrix to obtain sheep location data, and fuse the sheep location data and frame image time data in frame time order to obtain long-time location data.

[0124] The region type segmentation module is used to segment the grid image into regions to obtain the region type data where the sheep are located.

[0125] The node recognition module is used to identify target body nodes within the candidate area using a part recognition model, thereby obtaining images of sheep parts and node location data.

[0126] The node correction module is used to determine the torso angle of sheep in the candidate region, and to correct and fuse all node position data at the same time using the torso angle to obtain node correction data.

[0127] The motion fusion module is used to fuse node correction data and frame image time data in frame time order to obtain long-time motion data.

[0128] The behavior recognition module is used to input long-term location data, node location data, and long-term action data into the sheep behavior recognition model for detection, and obtain sheep behavior recognition results.

[0129] Specifically, this embodiment provides a method for implementing a part recognition model. EfficientNet-B4 is used as the backbone network for recognizing various parts of a sheep, HRNet-W48 is used for key node recognition, and finally, Mask2Former is used for image segmentation to output pixel-level part masks. To capture the movement data of the target sheep, this embodiment focuses on the positions of the sheep's face and torso, specifically including nodes such as the upper lip, lower jaw, neck root, thoracic vertebrae, lumbar vertebrae, anus, leg root, and leg joints. During training, images with node labels are used as input data. During training, some node labels are randomly deleted, and the model is used for prediction to improve the training effect.

[0130] Furthermore, common sheep diseases and their symptoms are shown in Table 1. Referring to Table 1, common sheep diseases often manifest on the face and abdomen. However, judging solely by the face and abdomen is unreliable. Therefore, in this embodiment, after matching the sheep's facial and abdominal images, statistical analysis of the sheep's feeding, rumination, wandering, resting, drinking, and excretion was performed. The two sets of data were then comprehensively processed, and k-nearest neighbor technology was used to determine the disease status and type.

[0131] Table 1

[0132]

[0133]

[0134] Preferably, the Canny algorithm is a multi-stage edge detection method. In this embodiment, the specific implementation process includes: smoothing the image using a Gaussian filter to eliminate noise interference; calculating the horizontal and vertical gradients of the image using the Sobel operator to obtain the gradient magnitude and direction; retaining only pixels with the largest local gradient along the gradient direction, and refining the edges to a single pixel width. Before implementation, high and low thresholds need to be set. Strong edges are directly retained, while weak edges are only retained when connected to strong edges to reduce false edges caused by noise and to connect broken contours. In pasture environments, there may be weeds, shadows, or other animal interference. The Canny algorithm, through Gaussian filtering and a dual-threshold mechanism, can effectively distinguish sheep contours from complex backgrounds; the sheep's head, torso, and limbs usually have obvious edge features in the image. The Canny algorithm, through non-maximum suppression and edge connection, can extract complete candidate region contours; the Canny algorithm has high computational efficiency and is suitable for real-time processing of video stream data.

[0135] Furthermore, this embodiment designs the area type data parameter for the sheep's location. The sheep's activity behavior has strong regional characteristics, such as drinking water and resting in fixed locations. Therefore, this embodiment sets area type labels in fixed areas when constructing the grid to adapt to the behavior recognition of the subsequent model.

[0136] Preferably, this embodiment selects a bidirectional GRU network as the base network for the sheep behavior recognition model. It is suitable for processing long-term temporal data. In this embodiment, the long-term temporal location data, node location data, and long-term temporal action data are all temporal data, satisfying the training input requirements of the model. The fully connected layers of the bidirectional GRU network are used for result classification, mapping the hidden states output by the intermediate network layers to behavior categories (7 categories: grazing, rumination, wandering, etc.). During model training, random occlusion is performed to simulate sheep being occluded, and the time steps of the action sequences are slightly adjusted to improve model robustness and enhance temporal generalization ability. This operation is performed after a preset number of iterations to ensure that the initial network training is not disturbed.

[0137] The beneficial effects of this invention are as follows:

[0138] This invention improves the model's recognition response speed and efficiency by utilizing image-based data for model analysis; it enhances the quality of the model's input data and the model's recognition accuracy by performing region of interest segmentation, matrix mapping, and node data extraction on the acquired images; and it improves the model's reliability and data dimensionality by setting up multiple cameras.

[0139] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0140] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for sheep behavior recognition based on machine vision, characterized in that, The method comprises the following steps: fixing multiple cameras at the edge positions of a target monitoring area based on preset height and inclination angle, and monitoring target sheep in the target monitoring area by using the cameras to obtain video stream data; frame image screening and extraction of the video stream data in each time window according to a fixed frequency to obtain a set of to-be-detected images; contour segmentation of the set of to-be-detected images to obtain a plurality of candidate regions, similarity calculation of each candidate region to obtain region similarity, and determination of the candidate region whose region similarity meets a similarity standard as a region of interest; dividing the target monitoring area into grid images according to a preset resolution, mapping the region of interest by using a preset conversion matrix to obtain sheep position data, and fusing the sheep position data and frame image time data in frame time sequence to obtain long-time sequence position data; region type division of the grid images to obtain sheep region type data corresponding to the sheep position data; identification of target body nodes in the candidate region by using a pre-constructed part recognition model to obtain sheep part images and node position data; determination of the torso angle of the sheep in the candidate region, correction and fusion of all the node position data at the same time by using the torso angle to obtain node correction data; fusion of the node correction data and the frame image time data in frame time sequence to obtain long-time sequence action data; input of the long-time sequence position data, the node position data, and the long-time sequence action data into a sheep behavior recognition model pre-trained based on a gated recurrent unit network for detection to obtain a sheep behavior recognition result; The sheep behavior recognition result includes feeding behavior, rumination behavior, walking behavior, resting behavior, drinking water behavior, excretion behavior, and mating behavior.

2. The sheep behavior recognition method based on machine vision according to claim 1, characterized in that, Further comprising: extracting sheep face images and sheep abdomen images from the sheep part images; state matching of the sheep face images and the sheep abdomen images by using a pre-constructed sick sheep face image set and abdomen image library to obtain matching degree data, and determination of a state label corresponding to a maximum value of the matching degree data as a state label of the sheep part image if the maximum value exceeds a preset matching threshold; statistical analysis of sheep feeding conditions, rumination conditions, walking conditions, resting conditions, drinking water conditions, and excretion conditions according to the sheep behavior recognition result to obtain sheep behavior statistical data; the sheep behavior statistical data includes feeding time data, feeding frequency data, rumination frequency data, walking frequency data, resting frequency data, drinking water frequency data, and excretion frequency data; classification of the sheep behavior statistical data by using k-nearest neighbor technology according to pre-acquired sick sheep behavior data to obtain sheep disease detection results and sheep disease type detection results. 3.The sheep behavior recognition method based on machine vision of claim 1, wherein, Frame image screening and extraction of the video stream data in each time window according to a fixed frequency to obtain a set of to-be-detected images, comprising: segmenting the video stream data according to the fixed frequency to obtain a plurality of video sub-clips; determination of a preset time at the head of each video sub-clip as the time window; Calculating each frame image in the time window by using a pre-constructed frame quality scoring formula to obtain a frame score dataset; the frame quality scoring formula is: wherein f1=∑ (x,y) |G(x,y)|; Q is the calculated value of the frame quality score formula; f1, f2, f3, f4 are respectively the definition factor, the illumination factor, the motion blur factor, and the content completeness factor; W1, W2, W3, W4 are respectively the definition weight, the illumination weight, the motion blur weight, and the content completeness weight; ∈ is the smoothing coefficient; Γ(C) is the constraint function, outputting 0 if the current frame image meets the preset rejection condition, and otherwise outputting 1; G(x, y) represents the convolution of the Laplacian operator at the pixel point (x, y); μ L is the average of the image brightness channel; is the brightness variance; F(I) is the image fast Fourier transform result; H high is the ideal high-pass filter; Taking the frame image corresponding to the highest score in the frame score dataset as the representative image of the video sub-fragment, and integrating the representative images of all the video sub-fragments to obtain the set of images to be detected.

4. The sheep behavior recognition method based on machine vision according to claim 1, characterized in that, Segmenting the set of images to be detected to obtain a plurality of candidate regions, calculating the similarity of each candidate region to obtain a region similarity, and determining the candidate region as a region of interest if the region similarity meets the similarity standard, including: Extracting edges and segmenting images of the set of images to be detected by using the Canny algorithm to obtain the candidate regions; After size normalization processing of the candidate regions, calculating the similarity value of each contour image in the preset livestock contour library with the candidate region by using the cosine similarity formula, and determining the maximum similarity value as the region similarity of the candidate region.

5. The sheep behavior recognition method based on machine vision according to claim 1, characterized in that, Dividing the target monitoring area into grid images according to a preset resolution, mapping the region of interest by using a preset conversion matrix to obtain sheep position data, and fusing the sheep position data and frame image time data in frame time order to obtain long-time sequence position data, including: According to the fixed position of the camera, the target monitoring area is expanded into a rectangular area, and the rectangular area is divided into grid areas of the same size according to the preset resolution to obtain the grid images; Converting the image barycentric coordinates of the region of interest into grid coordinates of the grid images by using the conversion matrix to obtain mapping coordinates; the expression of the conversion matrix is: Wherein, M is the conversion matrix; h is the preset height; θ is the inclination angle; f x , f y are the equivalent pixel values of the focal length of the camera in the X and Y axis directions of the image coordinate system, respectively; c x , c y are the principal points of the image in the X and Y axis directions of the image coordinate system, respectively. Calculating the minimum circumscribed circle of all the mapping coordinates mapped by the camera, and taking the grid number where the center of the minimum circumscribed circle is located as the sheep position data of the region of interest.

6. The sheep behavior recognition method based on machine vision according to claim 1, characterized in that, The sheep area type data includes drinking water area, feeding area, resting area and exercise area.

7. The sheep behavior recognition method based on machine vision according to claim 1, characterized in that, The training process of the sheep behavior recognition model includes: Removing noise, normalizing and time aligning the pre-collected labeled long-time sequence position data, node position data and long-time sequence action data to obtain standardized input data; Inputting the standardized input data into a pre-constructed bidirectional GRU network for calculation to obtain network output; Calculating the total loss value of the network output by using weighted cross-entropy loss; Iterating the bidirectional GRU network according to the total loss value by using the adam optimizer and the chord annealing strategy to obtain the trained sheep behavior recognition model. 8.The sheep behavior recognition method based on machine vision of claim 2, wherein, The matching degree data is SSIM data calculated by ffmpeg. 9.The sheep behavior recognition method based on machine vision of claim 5, wherein, Determining the torso angle of the sheep in the candidate region, correcting and fusing all the node position data at the same time by using the torso angle to obtain node correction data, including: Determining the torso angle of the candidate region by using principal component analysis; Constructing a correction matrix according to the torso angle; the expression of the correction matrix is: Wherein, C is the correction matrix; Alpha is the trunk angle; Correct the node position data by using the correction matrix to obtain node correction position data; Integrate the node correction position data corresponding to the same node into a group, and fuse the node correction position data corresponding to the same node in time sequence to obtain the node correction data.

10. A machine vision-based sheep behavior recognition system, characterized in that, The system is applied to the sheep behavior recognition method based on machine vision of claim 1, and the system comprises: A data acquisition module is configured to monitor the target sheep by using the camera to obtain the video stream data; An image extraction module is configured to extract frame images from the video stream data in each time window according to a fixed frequency to obtain a set of to-be-detected images; A region division module is configured to perform contour segmentation on the set of to-be-detected images to obtain a plurality of candidate regions, perform similarity calculation on each candidate region to obtain region similarity, and determine the candidate region with the region similarity meeting a similarity standard as the region of interest; A position fusion module is configured to divide the target monitoring region into grid images according to the preset resolution, map the region of interest by using the conversion matrix to obtain sheep position data, and fuse the sheep position data and frame image time data in frame time sequence to obtain long-time sequence position data; A region type division module is configured to divide the grid images into region type data of the region where the sheep is located; A node recognition module is configured to recognize target body nodes in the candidate region by using the part recognition model to obtain sheep part images and node position data; A node correction module is configured to determine the trunk angle of the sheep in the candidate region, correct and fuse all the node position data at the same time by using the trunk angle to obtain the node correction data; An action fusion module is configured to fuse the node correction data and the frame image time data in frame time sequence to obtain long-time sequence action data; A behavior recognition module is configured to input the long-time sequence position data, the node position data and the long-time sequence action data into the sheep behavior recognition model for detection to obtain a sheep behavior recognition result.

Images