Sheep behavior identification method and system based on machine vision

Abstract

The invention belongs to the technical field of livestock behavior recognition, and provides a sheep behavior recognition method and system based on machine vision, and the method comprises the steps: data collection, image extraction, region division, long-time-sequence position data construction, region type division, node recognition, node correction, long-time-sequence action data construction and behavior recognition. According to the invention, model analysis is carried out by using data extracted based on images, so that the recognition response speed and efficiency of the model are improved; by performing region-of-interest division, matrix mapping and node data extraction on the acquired image, the quality of model input data and the recognition accuracy of the model are improved; and by arranging a plurality of cameras, the reliability of the model and the dimension of data are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of livestock behavior recognition, and particularly to a sheep behavior recognition method and system based on machine vision. Background Art

[0002] Common sheep behavior recognition methods include: single camera vision monitoring, wearable sensor monitoring, and manual observation and recording. Single camera vision monitoring relies on a single fixed camera to collect video data, and extracts the target area through background subtraction or simple threshold segmentation; wearable sensor monitoring uses devices such as GPS collars, accelerometers, or RFID tags, and indirectly infers behaviors through sensor data; manual observation and recording relies on ranch staff to regularly patrol and manually record sheep behaviors.

[0003] The costs and time consumption of the prior art are relatively high. The current sheep behavior recognition model needs to face a large number of image inputs and requires a large amount of computing power for recognition, resulting in high costs and low efficiency; the prior art often does not perform preprocessing on data and often directly inputs it into the model, resulting in redundant useless data and poor model training effects; in addition, the prior art also has problems such as not considering multi-camera collaborative correction, resulting in large positioning errors, and not performing intelligent screening on video frames, being easily interfered by blurring, uneven illumination, or motion artifacts. Summary of the Invention

[0004] In order to overcome the deficiencies of the prior art, the purpose of the present invention is to provide a sheep behavior recognition method and system based on machine vision, and solve the problems of low efficiency and high cost of the prior art.

[0005] To achieve the above purpose, the present invention provides the following solutions:

[0006] A sheep behavior recognition method based on machine vision, comprising:

[0007] Fix a plurality of cameras at the edge positions of the target monitoring area based on a preset height and tilt angle, and use the cameras to monitor the target sheep in the target monitoring area to obtain video stream data;

[0008] Perform frame image screening and extraction on the video stream data within each time window according to a fixed frequency to obtain a set of images to be detected;

[0009] Perform contour segmentation on the set of images to be detected to obtain a number of candidate regions, calculate the similarity of each candidate region to obtain the region similarity, and determine the candidate regions whose region similarity meets the similarity standard as regions of interest;

[0010] Divide the target monitoring area into grid images according to a preset resolution, map the region of interest using a preset transformation matrix to obtain sheep position data, and fuse the sheep position data and frame image time data in chronological order of frames to obtain long-term position data;

[0011] Perform regional type division on the grid images to obtain sheep region type data corresponding to the sheep position data;

[0012] Use a pre-constructed body part recognition model to recognize target body nodes in the candidate region to obtain sheep body part images and node position data;

[0013] Determine the torso angle of the sheep in the candidate region, and use the torso angle to correct and fuse all the node position data at the same moment to obtain node correction data;

[0014] Fuse the node correction data and the frame image time data in chronological order of frames to obtain long-term action data;

[0015] Input the long-term position data, the node position data, and the long-term action data into a sheep behavior recognition model pre-trained based on a gated recurrent unit network for detection to obtain a sheep behavior recognition result; the sheep behavior recognition result includes: feeding behavior, rumination behavior, walking behavior, lying behavior, drinking behavior, excretion behavior, and mating behavior.

[0016] Preferably, it further includes:

[0017] Extract the sheep face image and the sheep abdomen image from the sheep body part image;

[0018] Use a pre-constructed diseased sheep face atlas and abdomen atlas to perform state matching on the sheep face image and the sheep abdomen image to obtain matching degree data. If the maximum value of the matching degree data exceeds a preset matching threshold, determine the state label corresponding to the maximum value as the state label of the sheep body part image;

[0019] Statistically analyze the feeding situation, rumination situation, walking situation, lying situation, drinking situation, and excretion situation of the sheep according to the sheep behavior recognition result to obtain sheep behavior statistical data; the sheep behavior statistical data includes: feeding time data, feeding frequency data, rumination frequency data, walking frequency data, lying frequency data, drinking frequency data, excretion frequency data;

[0020] Classify the sheep behavior statistical data using the k-nearest neighbor technique based on pre-collected diseased sheep behavior data to obtain a sheep disease detection result and a sheep disease type detection result.

[0021] Preferably, the video stream data within each time window is screened and extracted according to a fixed frequency to obtain a set of images to be detected, including:

[0022] The video stream data is segmented according to the fixed frequency to obtain a plurality of video sub - segments;

[0023] The preset time at the head of each video sub - segment is determined as the time window;

[0024] Each frame image within the time window is calculated using a pre - constructed frame quality scoring formula to obtain a frame score dataset; the frame quality scoring formula is:

[0025]

[0026] where, f1 = ∑ (x,y) |G(x,y)|;

[0027] Q is the calculated value of the frame quality scoring formula; f1, f2, f3, f4 are the clarity factor, illumination factor, motion blur factor, and content integrity factor respectively; W1, W2, W3, W4 are the clarity weight, illumination weight, motion blur weight, and content integrity weight respectively; ∈ is the smoothing coefficient; Γ(C) is a constraint function, which outputs 0 if the current frame image meets the preset veto condition, otherwise outputs 1; G(x,y) represents the convolution of the Laplacian operator at the pixel point (x,y); μ L is the mean of the image luminance channel; is the luminance variance; F(I) is the result of the fast Fourier transform of the image; H high is the ideal high - pass filter;

[0028] The frame image corresponding to the highest score in the frame score dataset is used as the representative image of the video sub - segment, and the representative images of all the video sub - segments are integrated to obtain the set of images to be detected.

[0029] Preferably, the set of images to be detected is subjected to contour segmentation to obtain a number of candidate regions, the similarity of each candidate region is calculated to obtain the region similarity, and the candidate regions whose region similarity meets the similarity standard are determined as the regions of interest, including:

[0030] The Canny algorithm is used to perform edge extraction and image segmentation on the set of images to be detected to obtain the candidate regions;

[0031] After performing size normalization on the candidate region, use the cosine similarity formula to calculate the similarity value between the candidate region and each contour image in the preset livestock contour library, and determine the maximum similarity value as the region similarity of the candidate region.

[0032] Preferably, divide the target monitoring area into grid images according to a preset resolution, map the region of interest using a preset transformation matrix to obtain sheep position data, and fuse the sheep position data and frame image time data in chronological order of frames to obtain long-term position data, including:

[0033] Expand the target monitoring area into a rectangular area according to the fixed position of the camera, and divide the rectangular area into grid areas of the same size according to the preset resolution to obtain the grid images;

[0034] Use the transformation matrix to convert the image center-of-gravity coordinates of the region of interest into grid coordinates of the grid image to obtain mapped coordinates; the expression of the transformation matrix is:

[0035]

[0036] where M is the transformation matrix; h is the preset height; θ is the tilt angle; f x 、f y are the equivalent pixel values of the focal length of the camera in the X and Y axis directions of the image coordinate system respectively; c x 、c y are the principal points of the image in the X and Y axis directions of the image coordinate system respectively;

[0037] Calculate the minimum circumscribed circle of the mapped coordinates mapped by all the cameras, and use the grid number where the center of the minimum circumscribed circle is located as the sheep position data of the region of interest.

[0038] Preferably, the area type data where the sheep are located includes: drinking area, feeding area, resting area, and movement area.

[0039] Preferably, the training process of the sheep behavior recognition model includes:

[0040] Perform noise removal, normalization processing, and time series alignment processing on the pre-collected long-term position data, node position data, and long-term action data with labels to obtain standardized input data;

[0041] Input the standardized input data into a pre-constructed bidirectional GRU network for calculation to obtain network output;

[0042] Use weighted cross-entropy loss to calculate the total loss value of the network output;

[0043] Iterate the bidirectional GRU network according to the total loss value using the adam optimizer and the chord annealing strategy to obtain the trained sheep behavior recognition model.

[0044] Preferably, the matching degree data is SSIM data calculated using ffmpeg.

[0045] Preferably, determine the torso angle of the sheep in the candidate region, and use the torso angle to correct and fuse all the node position data at the same moment to obtain node correction data, including:

[0046] Use principal component analysis to determine the torso angle of the candidate region;

[0047] Construct a correction matrix according to the torso angle; the expression of the correction matrix is:

[0048]

[0049] where C is the correction matrix; α is the torso angle;

[0050] Use the correction matrix to correct the node position data to obtain node corrected position data;

[0051] Integrate the node corrected position data corresponding to the same node into a group, and fuse the node corrected position data corresponding to the same node in chronological order to obtain the node correction data.

[0052] Preferably, a sheep behavior recognition system based on machine vision includes:

[0053] A data acquisition module for monitoring the target sheep using the camera to obtain the video stream data;

[0054] An image extraction module for screening and extracting frame images from the video stream data in each time window according to a fixed frequency to obtain the set of images to be detected;

[0055] A region division module for performing contour segmentation on the set of images to be detected to obtain a number of candidate regions, calculating the similarity of each candidate region to obtain the region similarity, and determining the candidate regions whose region similarity meets the similarity standard as the regions of interest;

[0056] A position fusion module, which is used to divide the target monitoring area into the grid images according to the preset resolution, map the region of interest by using the transformation matrix to obtain sheep position data, and fuse the sheep position data and the frame image time data in chronological order of frames to obtain the long-term position data;

[0057] A region type division module, which is used to divide the region type of the grid images to obtain the region type data where the sheep are located;

[0058] A node recognition module, which is used to recognize the target body nodes in the candidate region by using the part recognition model to obtain the sheep part images and the node position data;

[0059] A node correction module, which is used to determine the torso angle of the sheep in the candidate region, and correct and fuse all the node position data at the same moment by using the torso angle to obtain the node correction data;

[0060] An action fusion module, which is used to fuse the node correction data and the frame image time data in chronological order of frames to obtain the long-term action data;

[0061] A behavior recognition module, which is used to input the long-term position data, the node position data, and the long-term action data into the sheep behavior recognition model for detection to obtain the sheep behavior recognition result.

[0062] The present invention discloses the following technical effects:

[0063] The present invention provides a sheep behavior recognition method and system based on machine vision. By using the data extracted from images for model analysis, it solves the problems of increased computing power and redundant invalid data caused by directly inputting data such as images in the prior art, and realizes the feature extraction and data volume reduction of image data; by dividing the region of interest, matrix mapping, and node data extraction of the collected images, it solves the problem of low model recognition accuracy caused by directly inputting the collected data in the prior art, and realizes the screening and optimization of the data extracted from images; by setting multiple cameras, it solves the defect that a single camera is prone to errors, and realizes the comprehensive processing of multi-dimensional data. Description of the Drawings

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0065] Figure 1 Schematic diagram of the sheep behavior recognition process based on machine vision provided by the embodiment of the present invention;

[0066] Figure 2 Schematic diagram of the sick sheep recognition process provided by the embodiment of the present invention;

[0067] Figure 3 Schematic diagram of the process for obtaining the image set to be detected provided by the embodiment of the present invention;

[0068] Figure 4 Schematic diagram of the process for delineating the region of interest provided by the embodiment of the present invention;

[0069] Figure 5 Schematic diagram of the process for constructing long - term position data provided by the embodiment of the present invention. Detailed implementation manners

[0070] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0071] The purpose of the present invention is to provide a sheep behavior recognition method and system based on machine vision, so as to solve the problems of low efficiency and high cost in the prior art.

[0072] To make the above - mentioned objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0073] Figure 1 Schematic diagram of the sheep behavior recognition process based on machine vision provided by the embodiment of the present invention, as Figure 1 shown, the present invention provides a sheep behavior recognition method based on machine vision, including:

[0074] Step 100: Fix a plurality of cameras at the edge positions of the target monitoring area based on a preset height and tilt angle, and use the cameras to monitor the target sheep in the target monitoring area to obtain video stream data;

[0075] Step 200: Screen and extract frame images from the video stream data in each time window according to a fixed frequency to obtain an image set to be detected;

[0076] Step 300: Perform contour segmentation on the image set to be detected to obtain a number of candidate regions. Calculate the similarity for each candidate region to obtain the region similarity, and determine the candidate regions whose region similarity meets the similarity criterion as the regions of interest;

[0077] Step 400: Divide the target monitoring area into grid images according to a preset resolution. Use the preset transformation matrix to map the regions of interest to obtain sheep position data. Integrate the sheep position data and the frame image time data in chronological order of frames to obtain long-term position data;

[0078] Step 500: Perform region type division on the grid images to obtain the region type data of the areas where the sheep are located corresponding to the sheep position data;

[0079] Step 600: Use a pre-built body part recognition model to recognize the target body nodes in the candidate regions to obtain sheep part images and node position data;

[0080] Step 700: Determine the torso angle of the sheep in the candidate regions, and use the torso angle to correct and integrate all node position data at the same moment to obtain node correction data;

[0081] Step 800: Integrate the node correction data and the frame image time data in chronological order of frames to obtain long-term action data;

[0082] Step 900: Input the long-term position data, node position data, and long-term action data into a sheep behavior recognition model pre-trained based on a gated recurrent unit network for detection to obtain sheep behavior recognition results; The sheep behavior recognition results include: feeding behavior, rumination behavior, walking behavior, lying behavior, drinking behavior, excretion behavior, and mating behavior.

[0083] Reference Figure 2 , further including:

[0084] Step 1000: Extract the sheep face image and the sheep abdomen image from the sheep part image;

[0085] Step 1100: Use the pre-built diseased sheep face atlas and abdomen atlas to perform state matching on the sheep face image and the sheep abdomen image to obtain matching degree data. If the maximum value of the matching degree data exceeds the preset matching threshold, then determine the state label corresponding to the maximum value as the state label of the sheep part image;

[0086] Step 1200: According to the sheep behavior recognition results, count the feeding situation, rumination situation, walking situation, lying and resting situation, drinking situation, and excretion situation of the sheep to obtain sheep behavior statistical data; the sheep behavior statistical data includes: feeding time data, feeding frequency data, rumination frequency data, walking frequency data, lying and resting frequency data, drinking frequency data, and excretion frequency data;

[0087] Step 1300: Classify the sheep behavior statistical data using the k-nearest neighbor technique based on the pre-collected diseased sheep behavior data to obtain the sheep disease detection result and the sheep disease type detection result.

[0088] Reference Figure 3 , screen and extract frame images from the video stream data in each time window according to a fixed frequency to obtain a set of images to be detected, including:

[0089] Step 201: Segment the video stream data at a fixed frequency to obtain multiple video sub-fragments;

[0090] Step 202: Determine the preset time at the head of each video sub-fragment as the time window;

[0091] Step 203: Calculate each frame image in the time window using a pre-constructed frame quality scoring formula to obtain a frame score data set; the frame quality scoring formula is:

[0092]

[0093] where, f1 = ∑ (x,y) |G(x,y)|;

[0094] Q is the calculated value of the frame quality scoring formula; f1, f2, f3, f4 are the clarity factor, illumination factor, motion blur factor, and content integrity factor respectively; W1, W2, W3, W4 are the clarity weight, illumination weight, motion blur weight, and content integrity weight respectively; ∈ is the smoothing coefficient; Γ(C) is a constraint function, which outputs 0 if the current frame image meets the preset veto condition, otherwise outputs 1; G(x,y) represents the convolution of the Laplacian operator at the pixel point (x,y); μ L is the mean of the image luminance channel; is the luminance variance; F(I) is the result of the fast Fourier transform of the image; H high is the ideal high-pass filter;

[0095] Step 204: Use the frame image corresponding to the highest score in the frame score data set as the representative image of the video sub-fragment, and integrate the representative images of all video sub-fragments to obtain the set of images to be detected.

[0096] ReferenceFigure 4 Perform contour segmentation on the image set to be detected to obtain a number of candidate regions, calculate the similarity of each candidate region to obtain the region similarity, and determine the candidate regions whose region similarity meets the similarity standard as the regions of interest, including:

[0097] Step 301: Use the Canny algorithm to perform edge extraction and image segmentation on the image set to be detected to obtain candidate regions;

[0098] Step 302: After performing size normalization processing on the candidate regions, use the cosine similarity formula to calculate the similarity value between each candidate region and each contour image in the preset livestock contour library, and determine the maximum similarity value as the region similarity of the candidate region.

[0099] Reference Figure 5 Divide the target monitoring area into grid images according to the preset resolution, map the regions of interest using the preset transformation matrix to obtain sheep position data, and fuse the sheep position data and frame image time data in chronological order of frames to obtain long-term position data, including:

[0100] Step 401: Expand the target monitoring area into a rectangular area according to the fixed position of the camera, and divide the rectangular area into grid areas of the same size according to the preset resolution to obtain grid images;

[0101] Step 402: Use the transformation matrix to convert the image center of gravity coordinates of the region of interest into grid coordinates of the grid image to obtain mapping coordinates; the expression of the transformation matrix is:

[0102]

[0103] where M is the transformation matrix; h is the preset height; θ is the tilt angle; f x 、f y are the equivalent pixel values of the focal length of the camera in the X and Y axis directions of the image coordinate system respectively; c x 、c y are the principal points of the image in the X and Y axis directions of the image coordinate system respectively;

[0104] Step 403: Calculate the minimum circumscribed circle of the mapping coordinates mapped by all cameras, and use the grid number where the center of the minimum circumscribed circle is located as the sheep position data of the region of interest.

[0105] Preferably, the data on the type of area where the sheep are located includes: drinking area, feeding area, resting area, and exercise area.

[0106] Furthermore, the training process of the sheep behavior recognition model includes:

[0107] Perform noise removal, normalization, and temporal alignment processing on pre-collected long-term position data with tags, node position data, and long-term action data to obtain standardized input data;

[0108] Input the standardized input data into a pre-constructed bidirectional GRU network for calculation to obtain network output;

[0109] Calculate the total loss value of the network output using weighted cross-entropy loss;

[0110] Iterate the bidirectional GRU network according to the total loss value using the adam optimizer and chord annealing strategy to obtain a trained sheep behavior recognition model.

[0111] Preferably, the matching degree data is SSIM data calculated using ffmpeg.

[0112] Specifically, determine the torso angle of the sheep in the candidate region, and use the torso angle to correct and fuse all node position data at the same moment to obtain node correction data, including:

[0113] Use principal component analysis to determine the torso angle of the candidate region;

[0114] Construct a correction matrix according to the torso angle; the expression of the correction matrix is:

[0115]

[0116] where C is the correction matrix; α is the torso angle;

[0117] Use the correction matrix to correct the node position data to obtain node corrected position data;

[0118] Integrate the node corrected position data corresponding to the same node into a group, and fuse the node corrected position data corresponding to the same node in chronological order to obtain node correction data.

[0119] Furthermore, a sheep behavior recognition system based on machine vision includes:

[0120] A data acquisition module for monitoring the target sheep using a camera to obtain video stream data;

[0121] An image extraction module for screening and extracting frame images from the video stream data in each time window according to a fixed frequency to obtain a set of images to be detected;

[0122] A region division module for performing contour segmentation on the set of images to be detected to obtain several candidate regions, calculating the similarity of each candidate region to obtain region similarity, and determining the candidate regions whose region similarity meets the similarity standard as regions of interest;

[0123] A position fusion module, which is used to divide the target monitoring area into grid images according to a preset resolution, map the region of interest by using a transformation matrix to obtain sheep position data, and fuse the sheep position data and the frame image time data in the order of frame time to obtain long-term sequence position data;

[0124] A region type division module, which is used to divide the region type of the grid image to obtain the region type data where the sheep are located;

[0125] A node recognition module, which is used to recognize the target body nodes in the candidate region by using a part recognition model to obtain the sheep part image and the node position data;

[0126] A node correction module, which is used to determine the torso angle of the sheep in the candidate region, and use the torso angle to correct and fuse all the node position data at the same moment to obtain the node correction data;

[0127] An action fusion module, which is used to fuse the node correction data and the frame image time data in the order of frame time to obtain long-term sequence action data;

[0128] A behavior recognition module, which is used to input the long-term sequence position data, the node position data, and the long-term sequence action data into a sheep behavior recognition model for detection to obtain the sheep behavior recognition result.

[0129] Specifically, this embodiment provides a method for implementing a part recognition model. EfficientNet-B4 is used as the backbone network for recognizing each part of the sheep, the HRNet-W48 network is used for key node recognition, and finally image segmentation is performed through Mask2Former and a pixel-level part mask is output. In order to capture the motion data of the target sheep in this embodiment, the positions of the sheep's face and torso are focused on. The specific nodes include: upper lip, lower jaw, root of the neck, thoracic vertebra, lumbar vertebra, anus, root of the leg, and leg joint, etc. During training, images with node labels are used as input data. During the training process, some node labels are randomly deleted and the model is used for prediction to improve the training effect of the model.

[0130] Furthermore, a reference table for common sheep diseases and their symptoms is shown in Table 1. Referring to Table 1, common sheep diseases often manifest in the face and abdomen, but it is not reliable to judge only through the face and abdomen. Therefore, in this embodiment, after matching the states through the sheep face image and the sheep abdomen image, the feeding situation, rumination situation, wandering situation, lying and resting situation, drinking situation, and excretion situation of the sheep are still counted, and the two kinds of data are comprehensively processed, and the k-nearest neighbor technology is used to realize the judgment of the diseased condition and the disease type.

[0131] Table 1

[0132]

[0133]

[0134] Preferably, the Canny algorithm is a multi-stage edge detection method. In this embodiment, the specific implementation process of this algorithm includes: using a Gaussian filter to smooth the image to eliminate noise interference; calculating the horizontal and vertical gradients of the image through a Sobel operator to obtain the gradient magnitude and direction; in the gradient direction, only retaining the pixels with the largest local gradient, and refining the edge to a single-pixel width. Before implementation, high and low thresholds need to be set. For strong edges, they are directly retained, and weak edges are only retained when connected to strong edges to reduce false edges caused by noise, and broken contours are connected. In the pasture environment, there may be weeds, shadows, or other animal interferences. The Canny algorithm can effectively distinguish the sheep's contour from the complex background through Gaussian filtering and a double-threshold mechanism; the head, torso, and limbs of the sheep usually have obvious edge features in the image. The Canny algorithm can extract the complete contour of the candidate region through non-maximum suppression and edge connection; the Canny algorithm has high computational efficiency and is suitable for real-time processing of video stream data.

[0135] Furthermore, in this embodiment, data parameters of the region type where the sheep is located are designed. The activity behaviors of the sheep have strong regional characteristics. For example, they will drink water and rest at fixed positions. Therefore, in this embodiment, when constructing the grid, region type labels will be set in fixed regions to adapt to the subsequent behavior recognition of the model.

[0136] Preferably, this embodiment selects a bidirectional GRU network as the basic network of the sheep behavior recognition model. It is suitable for processing long-time series data. The long-time series position data, node position data, and long-time series action data in this embodiment are all time series data, meeting the training input requirements of the model; the fully connected layer of the bidirectional GRU network is used for result classification, mapping the hidden state output by the intermediate network layer to behavior categories (7 categories: eating, ruminating, wandering, etc.). During the model training process, random occlusion is performed to simulate the scenario where the sheep is occluded and the time steps of the action sequence are slightly adjusted to improve the robustness of the model and enhance the time series generalization ability. This operation is performed after a preset number of iteration rounds to ensure that the initial network training is not interfered.

[0137] The beneficial effects of the present invention are as follows:

[0138] The present invention improves the recognition response speed and efficiency of the model by using the data extracted from the image for model analysis; improves the quality of the model input data and the recognition accuracy of the model by performing region of interest division, matrix mapping, and node data extraction on the collected images; and improves the reliability of the model and the dimension of the data by setting multiple cameras.

[0139] In the present specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other.

[0140] In this article, specific examples are used to elaborate on the principles and implementation manners of the present invention. The descriptions of the above embodiments are only used to help understand the method of the present invention and its core idea. At the same time, for those of ordinary skill in the art, based on the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A method for identifying sheep behavior based on machine vision, characterized in that, Including: Fixing a plurality of cameras at the edge positions of the target monitoring area based on a preset height and inclination angle, and using the cameras to monitor the target sheep in the target monitoring area to obtain video stream data; Filtering and extracting frame images from the video stream data within each time window according to a fixed frequency to obtain a set of images to be detected; Performing contour segmentation on the set of images to be detected to obtain a number of candidate regions, calculating the similarity of each candidate region to obtain the region similarity, and determining the candidate regions whose region similarity meets the similarity standard as regions of interest; Dividing the target monitoring area into grid images according to a preset resolution, mapping the regions of interest using a preset transformation matrix to obtain sheep position data, and fusing the sheep position data and the frame image time data in the order of frame time to obtain long-term sequential position data; Dividing the grid images into region types to obtain region type data of the regions where the sheep are located corresponding to the sheep position data; Using a pre-built body part recognition model to recognize the target body nodes in the candidate regions to obtain sheep body part images and node position data; Determining the torso angle of the sheep in the candidate regions, and using the torso angle to correct and fuse all the node position data at the same moment to obtain node correction data; Fusing the node correction data and the frame image time data in the order of frame time to obtain long-term sequential action data; Inputting the long-term sequential position data, the node position data, and the long-term sequential action data into a sheep behavior recognition model pre-trained based on a gated recurrent unit network for detection to obtain a sheep behavior recognition result; The sheep behavior recognition result includes: feeding behavior, rumination behavior, walking behavior, lying behavior, drinking behavior, excretion behavior, and mating behavior.

2. The method for identifying sheep behavior based on machine vision according to claim 1, wherein, It also includes: Extracting the sheep face image and the sheep abdomen image from the sheep body part image; Using a pre-built diseased sheep face atlas and abdomen image library to perform state matching on the sheep face image and the sheep abdomen image to obtain matching degree data. If the maximum value of the matching degree data exceeds a preset matching threshold, determining the state label corresponding to the maximum value as the state label of the sheep body part image; Statistically analyzing the feeding, rumination, walking, lying, drinking, and excretion situations of the sheep according to the sheep behavior recognition result to obtain sheep behavior statistical data; the sheep behavior statistical data includes: feeding time data, feeding frequency data, rumination frequency data, walking frequency data, lying frequency data, drinking frequency data, and excretion frequency data; Classifying the sheep behavior statistical data using the k-nearest neighbor technique based on pre-collected diseased sheep behavior data to obtain a sheep disease detection result and a sheep disease type detection result.

3. The method for identifying sheep behavior based on machine vision according to claim 1, characterized in that Filtering and extracting frame images from the video stream data within each time window according to a fixed frequency to obtain a set of images to be detected, including: Dividing the video stream data according to the fixed frequency to obtain a plurality of video sub-fragments; Determining a preset time at the head of each video sub-fragment as the time window; Calculate each frame image within the time window using a pre - constructed frame quality scoring formula to obtain a frame score dataset; the frame quality scoring formula is: where f1 = ∑ (x,y) |G(x, y)|; Q is the calculated value of the frame quality scoring formula; f1, f2, f3, and f4 are the clarity factor, lighting factor, motion blur factor, and content integrity factor respectively; W1, W2, W3, and W4 are the clarity weight, lighting weight, motion blur weight, and content integrity weight respectively; ∈ is the smoothing coefficient; Γ(C) is a constraint function that outputs 0 if the current frame image meets the preset rejection condition, otherwise outputs 1; G(x, y) represents the convolution of the Laplacian operator at the pixel point (x, y); μ L is the mean value of the image luminance channel; is the luminance variance; F(I) is the result of the fast Fourier transform of the image; H high is an ideal high-pass filter; Take the frame image corresponding to the highest score in the frame score dataset as the representative image of the video sub - segment, and integrate the representative images of all the video sub - segments to obtain the image set to be detected.

4. A method for identifying sheep behavior based on machine vision according to claim 1, characterized in that, Perform contour segmentation on the image set to be detected to obtain a number of candidate regions, calculate the similarity of each candidate region to obtain the region similarity, and determine the candidate regions whose region similarity meets the similarity standard as the regions of interest, including: Use the Canny algorithm to perform edge extraction and image segmentation on the image set to be detected to obtain the candidate regions; After performing size normalization processing on the candidate regions, use the cosine similarity formula to calculate the similarity value between each candidate region and each contour image in the preset livestock contour library, and determine the maximum similarity value as the region similarity of the candidate region.

5. A method for recognizing sheep behavior based on machine vision according to claim 1, characterized in that, Divide the target monitoring area into grid images according to a preset resolution, use a preset transformation matrix to map the regions of interest to obtain sheep position data, and fuse the sheep position data and frame image time data in chronological order of frames to obtain long - time - series position data, including: Expand the target monitoring area into a rectangular area according to the fixed position of the camera, and divide the rectangular area into grid areas of the same size according to the preset resolution to obtain the grid images; Use the transformation matrix to convert the image centroid coordinates of the region of interest into grid coordinates of the grid image to obtain the mapped coordinates; the expression of the transformation matrix is: where M is the conversion matrix; h is the preset height; θ is the tilt angle; f x and f y are respectively the equivalent pixel values of the focal length of the camera in the X and Y axis directions of the image coordinate system; c x and c y are respectively the principal points of the image in the X and Y axis directions of the image coordinate system; Calculate the minimum circumscribed circle of all the mapped coordinates mapped by the cameras, and use the grid number where the center of the minimum circumscribed circle is located as the sheep position data of the region of interest.

6. The method for identifying sheep behavior based on machine vision according to claim 1, wherein The data of the area type where the sheep is located includes: drinking area, feeding area, resting area, and movement area.

7. A method for identifying sheep behavior based on machine vision according to claim 1, characterized in that The training process of the sheep behavior recognition model includes: Perform noise removal, normalization processing, and time - series alignment processing on the pre - collected long - time - series position data, node position data, and long - time - series action data with labels to obtain standardized input data; Input the standardized input data into a pre - constructed bidirectional GRU network for calculation to obtain the network output; Use weighted cross - entropy loss to calculate the total loss value of the network output; Iterate the bidirectional GRU network according to the total loss value using the adam optimizer and the chord annealing strategy to obtain the trained sheep behavior recognition model.

8. The method for identifying sheep behavior based on machine vision according to claim 2, wherein The matching degree data is SSIM data calculated using ffmpeg.

9. A method for identifying sheep behavior based on machine vision according to claim 5, characterized in that Determine the torso angle of the sheep in the candidate region, and use the torso angle to correct and fuse all the node position data at the same moment to obtain node correction data, including: Use principal component analysis to determine the torso angle of the candidate region; Construct a correction matrix according to the torso angle; the expression of the correction matrix is: where C is the correction matrix; α is the torso angle; The node position data is corrected using the correction matrix to obtain corrected node position data; The corrected node position data corresponding to the same node is integrated into a group, and the corrected node position data corresponding to the same node is fused in chronological order to obtain the node correction data.

10. A sheep behavior recognition system based on machine vision, characterized in that, Applied to the method for sheep behavior recognition based on machine vision according to claim 1, the system includes: A data acquisition module for monitoring the target sheep using the camera to obtain the video stream data; An image extraction module for screening and extracting frame images from the video stream data within each time window according to a fixed frequency to obtain the set of images to be detected; A region division module for performing contour segmentation on the set of images to be detected to obtain a number of candidate regions, calculating the similarity of each candidate region to obtain the region similarity, and determining the candidate regions whose region similarity meets the similarity standard as the regions of interest; A position fusion module for dividing the target monitoring region into the grid images according to the preset resolution, mapping the regions of interest using the transformation matrix to obtain sheep position data, and fusing the sheep position data and the frame image time data in chronological order of frames to obtain the long-time series position data; A region type division module for dividing the grid images into region type data where the sheep is located; A node recognition module for using the part recognition model to recognize the target body nodes within the candidate regions to obtain the sheep part images and the node position data; A node correction module for determining the torso angle of the sheep in the candidate regions, and using the torso angle to correct and fuse all the node position data at the same moment to obtain the node correction data; An action fusion module for fusing the node correction data and the frame image time data in chronological order of frames to obtain the long-time series action data; A behavior recognition module for inputting the long-time series position data, the node position data, and the long-time series action data into the sheep behavior recognition model for detection to obtain the sheep behavior recognition result.

Images