Livestock behavior characterization method based on computer vision technology and related equipment

Abstract

The invention provides a livestock behavior characterization method based on a computer vision technology and related equipment, and the method comprises the steps: sequentially obtaining video stream queues; the video stream queue comprises video clips collected by a plurality of video collection devices in the livestock house, and the video clips are arranged according to a corresponding spatial sequence and a time sequence; in each video stream queue, sequentially performing animal detection, animal head and tail detection and image segmentation on each video clip to obtain a first detection result of animal detection, a second detection result of animal head and tail detection and an image segmentation result of image segmentation; and according to the first detection results, the second detection results and the image segmentation results, determining an activeness index and a typical ingestion index, and according to the activeness index and the typical ingestion index, analyzing daily behaviors of the livestock. According to the activity index and the typical ingestion index, the health and feeding conditions of animals can be accurately reflected.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a livestock behavior characterization method based on computer vision technology and related equipment. Background Art

[0002] With the development of large-scale and intelligent livestock farms, video surveillance systems have become widely used in livestock and poultry production enterprises. In addition to remote monitoring, security measures, and enterprise management functions similar to those in other industries, intelligent surveillance systems in livestock enterprises, powered by computer vision algorithms, can automatically analyze, understand, and process farming videos. This enables real-time capture of footage from animal living areas, enabling identification, tracking, and early warning of livestock and poultry. This provides a reliable and effective technical means for monitoring abnormal conditions in barns, understanding animal behavior and health, and ensuring the safety of animals and facilities.

[0003] Video monitoring of animal behavior is crucial for understanding livestock production. In-depth analysis of the correlation between animal feeding behavior, activity levels, and health status provides a more comprehensive understanding of animal feeding habits and activity patterns. This not only helps promptly identify and address potential health issues, but also provides a scientific basis for more informed adjustments to feed supply and optimized feeding strategies. Therefore, a solution for accurately analyzing livestock behavior is urgently needed. Summary of the Invention

[0004] The present invention provides a livestock behavior characterization method and related equipment based on computer vision technology, which are used to solve the defects in the existing technology and realize accurate analysis of livestock behavior.

[0005] The present invention provides a method for characterizing livestock behavior based on computer vision technology, comprising: Sequentially acquiring a video stream queue; wherein the video stream queue includes video clips captured by a plurality of video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and a temporal order; In each of the video stream queues, performing animal detection, animal head and tail detection, and image segmentation on each of the video clips in sequence to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; According to each of the first detection results, the second detection results and the image segmentation result, the activity index and the typical feeding index corresponding to each of the video stream queues are determined, and the behavior of the livestock is analyzed based on the activity index and the typical feeding index.

[0006] According to a livestock behavior characterization method based on computer vision technology provided by the present invention, determining the activity index and feeding typicality index corresponding to each video stream queue based on each of the first detection results, the second detection results, and the image segmentation result, includes: In each of the video stream queues, a plurality of video pairs are determined based on the first detection result, the second detection result, and the image segmentation result; wherein the video pairs include two adjacent video frames in which livestock are present; Determining an absolute difference map of the region of interest in each of the video pairs by an inter-frame difference method, and calculating the activity index based on the absolute difference map; Based on the first detection result and the second detection result, it is detected whether the livestock has eating behavior, and the typical eating index is determined based on the eating behavior.

[0007] According to a livestock behavior characterization method based on computer vision technology provided by the present invention, the activity index is calculated based on the absolute difference map, comprising: performing summing processing on the pixel values ​​in the absolute difference map, and performing normalization processing on the summed absolute difference map to obtain a normalized activity index of each of the regions of interest; Based on the standardized activity index, the activity index of the time period corresponding to each of the regions of interest is calculated.

[0008] According to a livestock behavior characterization method based on computer vision technology provided by the present invention, detecting whether the livestock has feeding behavior based on the first detection result and the second detection result, and determining the feeding typicality index based on the feeding behavior, includes: Determining a head detection frame of the livestock based on the first detection result and the second detection result, and calculating an interaction ratio between the head detection frame and the feeding frame, determining whether each livestock has engaged in feeding behavior, and determining the number of feeding livestock that have engaged in feeding behavior, and calculating the feeding typicality index based on the number of feeding livestock; determining the total number of livestock based on the first detection result, and calculating an average feed intake rate according to the number of livestock eating and the total number of livestock; Based on the average feed intake rate and feed time, each hour is divided into a typical feed intake period, an atypical feed intake period, or a no feed intake period by the third quartile; According to the typical feeding time period, the atypical feeding time period or the non-feeding time period, the feeding typicality index is plotted as a feeding typicality index graph.

[0009] According to a livestock behavior characterization method based on computer vision technology provided by the present invention, before determining the activity index and feeding typicality index corresponding to each video stream queue based on each of the first detection results, the second detection results, and the image segmentation result, the method further includes: Performing image quality detection on each of the video clips to obtain a third detection result; If the third detection result indicates that the image quality does not meet the preset visual condition, the first detection result, the second detection result, and the image segmentation result of the video segment whose image quality does not meet the preset visual condition are deleted.

[0010] According to a livestock behavior characterization method based on computer vision technology provided by the present invention, the image quality detection of each video clip is performed to obtain a third detection result, including: At least one image quality detection of strong light detection, brightness and darkness detection, rain and fog detection, and occlusion detection is performed on each of the video clips to obtain the third detection result.

[0011] According to a livestock behavior characterization method based on computer vision technology provided by the present invention, before obtaining the video stream queue, the method further includes: Obtaining video clips collected by various video collection devices in a livestock house according to a preset sampling rule; wherein each of the video clips corresponds to a different time period; Determining the spatial order of the video clips according to the spatial positions of the video capture devices of the video clips, and determining the temporal order of the video clips according to the capture time of the video clips; The video segments are sorted according to the spatial order and the temporal order corresponding to the video segments to obtain the video stream queue.

[0012] The present invention also provides a livestock behavior characterization device based on computer vision technology, comprising: A first acquisition module is configured to sequentially acquire a video stream queue; wherein the video stream queue includes video clips captured by multiple video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and a temporal order; a detection module configured to perform animal detection, animal head and tail detection, and image segmentation on each of the video clips in each video stream queue, to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; The behavior analysis module is configured to determine the activity index and feeding typicality index corresponding to each video stream queue based on the first detection results, the second detection results and the image segmentation results, and perform behavior analysis on the livestock based on the activity index and the feeding typicality index.

[0013] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for characterizing livestock behavior based on computer vision technology as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for characterizing livestock behavior based on computer vision technology.

[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned methods for characterizing livestock behavior based on computer vision technology.

[0016] The present invention provides a method and related equipment for characterizing livestock behavior based on computer vision technology. The video stream queue is composed of video clips collected by multiple video acquisition devices in a livestock barn. Each video clip is arranged in a corresponding spatial and temporal order. Therefore, the video clips in the video stream queue can cover most of the area within the livestock barn. In each video stream queue, animal detection, animal head and tail detection, and image segmentation are performed on each video clip in sequence to obtain a first detection result of animal detection, a second detection result of animal head and tail detection, and an image segmentation result. Based on the first detection result, the second detection result, and the image segmentation result, the activity index and feeding typicality index corresponding to each video stream queue are obtained, and then behavioral analysis is performed. The activity index characterizes the activity level of livestock and can provide valuable reference information for livestock barn environmental management. The feeding typicality index effectively captures the temporal distribution characteristics of the feeding behavior of a livestock group. Therefore, based on the activity index and feeding typicality index, accurate livestock behavior analysis can be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1It is a flow chart of the livestock behavior characterization method based on computer vision technology provided by the present invention.

[0019] Figure 2 It is a schematic diagram of a typical feeding index diagram provided by the present invention.

[0020] Figure 3 This is a schematic diagram of detecting a video stream queue provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the time required for detecting a video stream queue provided by the present invention.

[0022] Figure 5 It is a schematic diagram of collecting images of a sheep house in an embodiment provided by the present invention.

[0023] Figure 6 Schematic diagram of the activity index of the sheep house in the embodiment provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of the livestock behavior characterization device based on computer vision technology provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0027] The following combination Figures 1-8 The present invention describes a livestock behavior characterization method based on computer vision technology and related equipment.

[0028] Livestock farm surveillance video content is characterized by high similarity, strong repetitiveness, and minimal content differentiation. Traditional full-time, full-frame-rate, and full-bitrate video recording methods significantly waste data transmission bandwidth, video storage space, system computing power, and energy consumption. In livestock applications, roving recording across multiple cameras is a way to significantly reduce redundant video content and lower data acquisition and usage costs. Current video codec technologies, such as H.264 / AVC, HEVC (H.265), VP9, ​​and the more recent AV1, achieve compression by reducing spatial and temporal redundancy, using techniques such as block partitioning, motion estimation, transform coding, and entropy coding to reduce data size. For applications requiring long-term monitoring of livestock and poultry houses and intelligent video analysis, the visual analysis technologies of traditional cameras and general-purpose smart cameras cannot meet the specific requirements of roving recording, image quality assessment, and behavioral analysis in livestock scenarios.

[0029] For example, feeding behavior is the basis for animal survival and growth. It is not only directly related to the animal's energy intake and nutritional balance, but also a core element for maintaining the normal operation of its physiological functions. Rest and activity levels are important indicators reflecting the health status and behavior patterns of animals. Many animals spend half or more of their time resting every day. Activity behavior refers to the physical movements exhibited by animals for purposes such as exploring the environment, finding food, and engaging in social interactions. The rest and activity levels of animals are often affected by biological rhythms and will show a certain periodicity, such as circadian rhythms, seasonal changes, etc. In-depth analysis of the correlation between animal feeding behavior, activity level and health status can provide a more comprehensive understanding of the animal's feeding habits and activity patterns. It not only helps to promptly discover and deal with possible health problems, but also provides a scientific basis for more scientifically adjusting feed supply and optimizing feeding strategies. Therefore, the present invention provides a livestock behavior characterization method and related equipment based on computer vision technology.

[0030] Figure 1 FIG. 1 is a flow chart showing a method for characterizing livestock behavior based on computer vision technology according to an exemplary embodiment. Figure 1 As shown, in an exemplary embodiment, the livestock behavior characterization method based on computer vision technology includes steps 110 to 150, which are described in detail as follows.

[0031] Step 110 , sequentially obtaining a video stream queue; wherein the video stream queue includes video clips captured by multiple video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and time order.

[0032] In an embodiment of the present invention, the video stream queue includes multiple video clips captured by different video capture devices in a livestock house, each video clip is arranged in a corresponding time sequence and spatial sequence, and the video clips in the video stream queue can cover most of the range in the livestock house.

[0033] Step 120, in each of the video stream queues, perform animal detection, animal head and tail detection, and image segmentation on each of the video clips in turn to obtain a first detection result of animal detection, a second detection result of animal head and tail detection, and an image segmentation result of image segmentation.

[0034] In the embodiment of the present invention, in the video stream queue generated by the video stream sampling program, each video clip will be subjected to animal detection, animal head and tail detection, and image segmentation.

[0035] Animal detection is to perform animal detection on video frames extracted from video clips using a pre-trained animal detection model to determine whether there is an animal in the video frame. The first detection result obtained includes the detection frame of the identified animal and the confidence of the prediction result.

[0036] Animal head and tail detection uses a pre-trained animal head and tail detection model to identify the head and tail of an animal within a detection frame in a video frame. The second detection result obtained contains two pairs of key points of the head and tail.

[0037] Image segmentation is to segment the outline of the animal in the animal detection frame in the video frame through a pre-trained image segmentation model. The obtained image segmentation result contains the mask and outline list of the animal area in the video frame.

[0038] Step 130: Determine the activity index and feeding typicality index corresponding to each video stream queue based on the first detection results, the second detection results, and the image segmentation results, and perform behavioral analysis on the livestock based on the activity index and the feeding typicality index.

[0039] In an embodiment of the present invention, based on animal detection, head and tail detection, and image segmentation results in consecutive frames of a video clip, a calculation scheme for evaluating livestock behavioral characteristics—an activity index and a typical feeding index—is derived. Assessing livestock activity levels using the activity index can provide valuable reference information for livestock house environmental management. The typical feeding index can effectively capture the temporal distribution of feeding behavior in a group of livestock. Therefore, accurate livestock behavioral analysis can be performed based on the activity index and the typical feeding index.

[0040] In an exemplary embodiment of the present invention, determining the activity index and feeding typicality index corresponding to each of the video stream queues based on the first detection results, the second detection results, and the image segmentation results includes: In each of the video stream queues, a plurality of video pairs are determined based on the first detection result, the second detection result, and the image segmentation result; wherein the video pairs include two adjacent video frames in which livestock are present; Determining an absolute difference map of the region of interest in each of the video pairs by an inter-frame difference method, and calculating the activity index based on the absolute difference map; Based on the first detection result and the second detection result, it is detected whether the livestock has eating behavior, and the typical eating index is determined based on the eating behavior.

[0041] In an embodiment of the present invention, in each video stream queue, multiple video pairs are determined based on the first detection result, the second detection result, and the image segmentation result. Each video pair includes two consecutive video frames, in which livestock exist, and a region of interest (ROI), i.e., the area where the livestock is located, is marked. The ROI defines the position coordinate points of the livestock in the video screen.

[0042] For each pair of video frames, an absolute difference map is calculated in each region of interest. The absolute difference map represents the change between the latter frame and the previous frame in each pair of video frames, and the activity index is calculated based on the absolute difference map.

[0043] Based on the first detection result and the second detection result, it is detected whether the livestock has feeding behavior, and a typical feeding index is determined based on the feeding behavior.

[0044] In an exemplary embodiment of the present invention, calculating the activity index based on the absolute difference graph includes: performing summing processing on the pixel values ​​in the absolute difference map, and performing normalization processing on the summed absolute difference map to obtain a normalized activity index of each of the regions of interest; Based on the standardized activity index, the activity index of the time period corresponding to each of the regions of interest is calculated.

[0045] In the embodiment of the present invention, the pixel values ​​in the absolute difference map are summed and normalized by the total number of pixels and the maximum pixel value (such as 255) in the region of interest to calculate the standardized activity index of the region ( ). Take the average of the standardized activity index of all areas of interest in each time period to obtain the activity index of the current time period ( Then, taking the dormitory as the unit, the columns corresponding to all video acquisition devices in the dormitory are averaged to obtain the daily average activity index ( ), as an indicator of the overall activity level of livestock throughout the day.

[0046] Specifically, the normalized activity index is calculated using the following formula: ; Among them, for the Region of interest, set Indicates the Frame is the video frame in the region of interest (t=1,2,…,T, T is the total number of video pairs determined in the video clip), and Indicates the coordinates of the upper left corner and lower right corner of the region of interest, represents the pixel position, Indicates the current frame and the previous frame In position The absolute value of the pixel difference, Indicates the sum of all pixels within the region of interest.

[0047] The activity index for the current time period is calculated using the following formula: ; in, is the number of regions of interest.

[0048] The daily average activity index is calculated using the following formula: ; in, The number of video capture devices.

[0049] In an exemplary embodiment of the present invention, detecting whether the livestock has feeding behavior based on the first detection result and the second detection result, and determining the feeding typicality index based on the feeding behavior, includes: Determining a head detection frame of the livestock based on the first detection result and the second detection result, and calculating an interaction ratio between the head detection frame and the feeding frame, determining whether each livestock has engaged in feeding behavior, and determining the number of feeding livestock that have engaged in feeding behavior, and calculating the feeding typicality index based on the number of feeding livestock; determining the total number of livestock based on the first detection result, and calculating an average feed intake rate according to the number of livestock eating and the total number of livestock; Based on the average feed intake rate and feed time, each hour is divided into a typical feed intake period, an atypical feed intake period, or a no feed intake period by the third quartile; According to the typical feeding time period, the atypical feeding time period or the non-feeding time period, the feeding typicality index is plotted as a feeding typicality index graph.

[0050] In this embodiment of the present invention, a head detection frame of the livestock is determined based on the animal detection frame in the first detection result and the identified animal head in the second detection result. A feeding frame is pre-set, representing the location where the livestock feeds. The Intersection of Union (IoU) between the head detection frame and the feeding frame is calculated. When the IoU is greater than or equal to a preset threshold, the livestock is considered to have fed. The number of feeding livestock that have engaged in feeding is calculated using the following formula: ; Among them, B i is the detection frame of the i-th animal, i∈[1, N f (k)]; F is the feeding frame; is the detection box B of the i-th animal i Intersection-over-union ratio with the feeding frame F; It is an indicator function that takes the value 1 when the condition in the brackets is met, otherwise it takes the value 0.

[0051] Identify the total number of livestock N in the livestock barn t (k) Using a one-hour statistical window, calculate the average feeding rate (AFR), total feeding duration (TFD), and normalized cumulative feeding duration (NCFD) for each hour of each day. These indicators can be used as typical feeding indices.

[0052] The average feed intake rate was calculated using the following formula: ; Among them, h is the time window of the current hour; is the number of animals feeding in the kth frame, in units of heads; is the total number of livestock identified in the kth frame, in heads; The unit is %.

[0053] The total feeding time was calculated using the following formula: ; in, The unit is seconds.

[0054] Normalized cumulative feeding time was calculated using the following formula: ; in, The unit is seconds; , For the current hour The maximum value of , in units of heads.

[0055] Using the third quartile (Q3) of the average feeding rate as the classification standard, each hour was classified as a typical feeding period, an atypical feeding period, or a non-feeding period, and then a feeding typicality index chart was drawn.

[0056] Figure 2 This is a typical index chart of feed intake in a beef cattle herd in a day, such as Figure 2 As shown, the average feed intake rate of livestock reached its highest point at 13:29, which corresponds to the daily feeding schedule of the farm. After feeding, the first continuous typical feeding period of the day occurs, followed by subsequent typical feeding periods approximately 6 and 8 hours later. This pattern may be closely related to rumen emptying and the onset of hunger in beef cattle. A single-day feeding schedule results in a cluster of feeding behaviors after feeding, while the intervals between feeding periods may reflect the rumen emptying time and digestion cycle. The combined variation in AFR and NCFD reflects both the heterogeneity and regularity of individual feeding behaviors within the herd. During atypical feeding periods, when the AFR is low but the NCFD is high, it indicates that even with a low overall feed intake rate, some cattle continue to feed for extended periods. When the AFR is high but the NCFD is high, the feeding cattle exhibit prolonged feeding periods, averaging over 15 minutes. During typical feeding periods, especially during feeding times, when feed intake peaks, the NCFD is also high, indicating that most cattle are engaged in concentrated feeding. When the feed intake rate dropped to around 30%, the NCFD remained high, lasting an average of more than 30 minutes. This phenomenon may be related to differences in satiety between individuals. Some cows may have met their energy needs while others are still supplementing feed, resulting in a prolonged feeding period.

[0057] In an exemplary embodiment of the present invention, before determining the activity index and feeding typicality index corresponding to each of the video stream queues based on the first detection results, the second detection results, and the image segmentation results, the method further includes: Performing image quality detection on each of the video clips to obtain a third detection result; If the third detection result indicates that the image quality does not meet the preset visual condition, the first detection result, the second detection result, and the image segmentation result of the video segment whose image quality does not meet the preset visual condition are deleted.

[0058] In an embodiment of the present invention, a video clip is subjected to image quality detection, and the visual conditions of the video clip are scored. Based on the score, it is determined whether the image quality of the video clip meets the preset visual conditions. If so, the first detection result, the second detection result, and the image segmentation result of the video clip are considered valid. If the preset visual conditions are not met, the first detection result, the second detection result, and the image segmentation result of the video clip may have large errors, thereby affecting the subsequent activity index and feeding typicality index, and therefore should be discarded.

[0059] In another embodiment of the present invention, after obtaining the video stream queue, the video clips in the video stream queue can be firstly subjected to image quality detection. If the video clips do not meet the preset visual conditions, there is no need to perform animal detection, animal head and tail detection, and image segmentation on the video clips.

[0060] In an exemplary embodiment of the present invention, performing image quality detection on each of the video clips to obtain a third detection result includes: At least one image quality detection of strong light detection, brightness and darkness detection, rain and fog detection, and occlusion detection is performed on each of the video clips to obtain the third detection result.

[0061] In the embodiment of the present invention, image quality detection is performed on the first video frame of the video clip, including but not limited to strong light detection, brightness and darkness detection, rain and fog detection, occlusion detection, etc.

[0062] In an exemplary embodiment of the present invention, before obtaining the video stream queue, the method further includes: Obtaining video clips collected by various video collection devices in a livestock house according to a preset sampling rule; wherein each of the video clips corresponds to a different time period; Determining the spatial order of the video clips according to the spatial positions of the video capture devices of the video clips, and determining the temporal order of the video clips according to the capture time of the video clips; The video segments are sorted according to the spatial order and the temporal order corresponding to the video segments to obtain the video stream queue.

[0063] In this embodiment of the present invention, the spatial location of video capture devices within a livestock barn can be determined by the site, which can be categorized as field, barn, pen, or station. The video capture devices deployed on the farm are first organized into a patrol queue based on predefined sampling rules. These sampling rules include the sampling interval, the duration of a single sampling, and the sampling frame rate for the same video capture device. Video streams from different video capture devices are organized into a queue in the order of pen, barn, and field, with the temporal attributes of video streams from cameras belonging to the same parent level aligned.

[0064] like Figure 3 As shown, the video acquisition devices in the livestock house can be sorted in advance according to their spatial positions. For example, the livestock houses are sorted to obtain a livestock house order, and then multiple video acquisition devices located in the same livestock house are sorted. According to the obtained livestock house order and video acquisition device order, the corresponding video acquisition devices are controlled in turn to capture video clips. Figure 3 As shown, after camera 1 in barn 1 captures a video clip, camera 2 in barn 1 then captures the second video clip, and so on until all cameras in barn 1 have captured the corresponding video clips. Then camera 1 in barn 2 captures the corresponding video clip, and so on until all cameras in all barns have captured the corresponding video clips.

[0065] The video clips in the video stream queue are controlled by the inspection program to enter the algorithm detection module, which optimizes the distribution and detection order of video clips based on the current computing power and memory resources. Figure 4 As shown, the atomic algorithms in the algorithm detection module are executed sequentially. Video capture devices in the same barn form the smallest inspection unit. The time consumed for a single inspection is T, and the interval between two consecutive inspections of the same video capture device is the round interval, INTER. INTER must be greater than the time it takes to complete all inferences. Based on the algorithm's inference efficiency and the inspection timeliness requirement (INTER), the number of inferences per video segment is optimized to achieve more stable inference results.

[0066] This method uses a specific sampling frequency to determine the activity index and typical feeding index. Compared to traditional full-scale calculations, it can significantly reduce the frequency of data collection and processing, significantly lowering computing power requirements and storage costs. While ensuring the accuracy of livestock behavior analysis, it optimizes computing efficiency, reduces hardware investment and operating costs. This feature is particularly suitable for edge devices with limited computing power or remote monitoring scenarios, thereby improving the cost-effectiveness and practicality of the monitoring system.

[0067] In an exemplary embodiment of the present invention, six binocular cameras and four POE hemispherical cameras are set in two sheep sheds. The frame rate of the binocular cameras is 25 fps and the resolution is 1920*1080. The frame rate of the POE hemispherical cameras is 15 fps and the resolution is 1920*1080. These six cameras are used to continuously monitor a total of 16 stalls in the two sheep sheds. The monitoring range of the cameras is as follows: Figure 5 As shown, Figure 5 The image on the left is captured by a binocular camera, and the image on the right is captured by a POE hemispherical camera. The technical solution provided by this invention calculates the activity index of a flock, providing a deeper understanding of its behavioral characteristics and enabling accurate assessment and monitoring of its overall health and comfort.

[0068] like Figure 6 The figure shows the difference in the average activity index of the sheep barn from September 4, 2024, to December 4, 2024. Comparison and analysis with temperature data and event records show that in September and November, there was a significant difference in the number of sheep in the two bars, and the sheep activity levels were high, resulting in a significant difference in the average activity index between the two bars. In early November, due to adjustments to the curtain control inside the barn to achieve a heat-retaining effect due to cooling, the barn temperature was higher than before, and the overall activity level of the sheep increased. This is consistent with the behavior of sheep in cold environments, which reduce their activity to conserve heat. Therefore, this activity index can be used as an effective tool to assess animal activity levels and provide valuable reference information for livestock barn environmental management.

[0069] The present invention uses a scientific and reasonable calculation method to quantify feeding and resting behaviors, revealing the differences and regularities in individual animal behaviors. Subsequently, accurate analysis and breeding management decisions can be made based on behavioral data, providing a scientific basis for optimizing feeding time and frequency in actual production, and having high research value and practical significance. Therefore, the present invention not only achieves major innovations at the technical level, but also brings significant economic and quality advantages in practical applications. Especially in the field of precision animal husbandry, through this method, animal husbandry-related scientific researchers can accelerate the mining and utilization of data, and promote scientific research and technological progress in related fields.

[0070] The following describes the livestock behavior characterization device based on computer vision technology provided by the present invention. The livestock behavior characterization device based on computer vision technology described below can be referenced in conjunction with the livestock behavior characterization method based on computer vision technology described above. It should be noted that the device provided in the following embodiments and the method provided in the above embodiments share the same concept. The specific manner in which each module and unit performs its operations has been described in detail in the method embodiments and will not be repeated here.

[0071] In an exemplary embodiment of the present invention, see Figure 7 , Figure 7 A livestock behavior characterization device based on computer vision technology is shown according to an exemplary embodiment, including the following modules.

[0072] A first acquisition module 710 is configured to sequentially acquire a video stream queue; wherein the video stream queue includes video clips captured by multiple video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and a temporal order; a detection module 720 configured to perform animal detection, animal head and tail detection, and image segmentation on each of the video clips in each video stream queue, to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; The behavior analysis module 730 is configured to determine the activity index and feeding typicality index corresponding to each video stream queue based on the first detection results, the second detection results and the image segmentation results, and perform behavior analysis on the livestock based on the activity index and the feeding typicality index.

[0073] In an exemplary embodiment of the present invention, the behavior analysis module 730 includes: a first determining submodule configured to determine, in each of the video stream queues, a plurality of video pairs based on the first detection result, the second detection result, and the image segmentation result; wherein the video pairs include two adjacent video frames in which livestock are present; a calculation submodule configured to determine an absolute difference map of the region of interest in each of the video pairs by an inter-frame difference method, and calculate the activity index based on the absolute difference map; The detection submodule is configured to detect whether the livestock has eating behavior based on the first detection result and the second detection result, and determine the typical eating index based on the eating behavior.

[0074] In an exemplary embodiment of the present invention, the calculation submodule includes: a processing unit configured to sum the pixel values ​​in the absolute difference map and normalize the summed absolute difference map to obtain a normalized activity index for each of the regions of interest; The first calculation unit is configured to calculate the activity index of the time period corresponding to each of the regions of interest based on the standardized activity index.

[0075] In an exemplary embodiment of the present invention, the detection submodule includes: a second calculation unit configured to determine a head detection frame of the livestock based on the first detection result and the second detection result, calculate an interaction ratio between the head detection frame and the feeding frame, determine whether each livestock has engaged in feeding behavior, determine the number of feeding livestock that have engaged in feeding behavior, and calculate the feeding typicality index based on the number of feeding livestock; a third calculating unit configured to determine the total number of livestock based on the first detection result, and calculate an average feeding rate according to the number of feeding livestock and the total number of livestock; a dividing unit configured to divide each hour into a typical feeding time period, an atypical feeding time period or a non-feeding time period by using a third quartile based on the average feeding rate and feeding time; The drawing unit is configured to draw the typical feeding index into a typical feeding index graph according to the typical feeding time period, the atypical feeding time period or the non-feeding time period.

[0076] In an exemplary embodiment of the present invention, the livestock behavior characterization method based on computer vision technology further includes: an image quality detection module configured to perform image quality detection on each of the video clips to obtain a third detection result; The deletion module is configured to delete the first detection result, the second detection result and the image segmentation result of the video clip whose image quality does not meet the preset visual condition if the third detection result indicates that the image quality does not meet the preset visual condition.

[0077] In an exemplary embodiment of the present invention, the image quality detection module includes: The image quality detection submodule is configured to perform at least one image quality detection of strong light detection, brightness and darkness detection, rain and fog detection, and occlusion detection on each of the video clips to obtain the third detection result.

[0078] In an exemplary embodiment of the present invention, the livestock behavior characterization method based on computer vision technology further includes: The second acquisition module is configured to acquire video clips captured by various video acquisition devices in the livestock barn according to a preset sampling rule; wherein each of the video clips corresponds to a different time period; a determining module configured to determine a spatial order of each of the video segments according to a spatial position of a video capture device of each of the video segments, and to determine a temporal order of each of the video segments according to a capture time of each of the video segments; The video stream queue module is configured to sort the video segments according to the spatial order and time order corresponding to the video segments to obtain the video stream queue.

[0079] Figure 8 An example of a physical structure diagram of an electronic device is shown below. Figure 8As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communications bus 840. The processor 810 may call logic instructions in the memory 830 to execute a livestock behavior characterization method based on computer vision technology, the method comprising: sequentially acquiring a video stream queue; wherein the video stream queue includes video clips captured by multiple video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and a temporal order; In each of the video stream queues, performing animal detection, animal head and tail detection, and image segmentation on each of the video clips in sequence to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; According to each of the first detection results, the second detection results and the image segmentation result, the activity index and the typical feeding index corresponding to each of the video stream queues are determined, and the behavior of the livestock is analyzed based on the activity index and the typical feeding index.

[0080] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0081] On the other hand, the present invention further provides a computer program product, comprising a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the livestock behavior characterization method based on computer vision technology provided by the above methods, the method comprising: sequentially acquiring a video stream queue; wherein the video stream queue comprises video clips captured by multiple video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and temporal order; In each of the video stream queues, performing animal detection, animal head and tail detection, and image segmentation on each of the video clips in sequence to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; According to each of the first detection results, the second detection results and the image segmentation result, the activity index and the typical feeding index corresponding to each of the video stream queues are determined, and the behavior of the livestock is analyzed based on the activity index and the typical feeding index.

[0082] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the livestock behavior characterization method based on computer vision technology provided by the above methods, the method comprising: sequentially acquiring a video stream queue; wherein the video stream queue comprises video clips captured by multiple video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and temporal order; In each of the video stream queues, performing animal detection, animal head and tail detection, and image segmentation on each of the video clips in sequence to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; According to each of the first detection results, the second detection results and the image segmentation result, the activity index and the typical feeding index corresponding to each of the video stream queues are determined, and the behavior of the livestock is analyzed based on the activity index and the typical feeding index.

[0083] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0084] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for characterizing livestock behavior based on computer vision technology, characterized in that: include: Sequentially acquiring a video stream queue; wherein the video stream queue includes video clips captured by a plurality of video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and a temporal order; In each of the video stream queues, performing animal detection, animal head and tail detection, and image segmentation on each of the video clips in sequence to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; According to each of the first detection results, the second detection results and the image segmentation result, the activity index and the typical feeding index corresponding to each of the video stream queues are determined, and the behavior of the livestock is analyzed based on the activity index and the typical feeding index.

2. The livestock behavior characterization method based on computer vision technology according to claim 1, characterized in that: Determining the activity index and feeding typicality index corresponding to each of the video stream queues according to the first detection results, the second detection results, and the image segmentation results includes: In each of the video stream queues, a plurality of video pairs are determined based on the first detection result, the second detection result, and the image segmentation result; wherein the video pairs include two adjacent video frames in which livestock are present; determining an absolute difference map of the region of interest in each of the video pairs by an inter-frame difference method, and calculating the activity index based on the absolute difference map; Based on the first detection result and the second detection result, it is detected whether the livestock has eating behavior, and the typical eating index is determined based on the eating behavior.

3. The livestock behavior characterization method based on computer vision technology according to claim 2, characterized in that: Calculating the activity index based on the absolute difference graph includes: performing summing processing on the pixel values ​​in the absolute difference map, and performing normalization processing on the summed absolute difference map to obtain a normalized activity index of each of the regions of interest; Based on the standardized activity index, the activity index of the time period corresponding to each of the regions of interest is calculated.

4. The livestock behavior characterization method based on computer vision technology according to claim 2, characterized in that: The detecting whether the livestock has a feeding behavior based on the first detection result and the second detection result, and determining the feeding typicality index based on the feeding behavior, includes: Determining a head detection frame of the livestock based on the first detection result and the second detection result, and calculating an interaction ratio between the head detection frame and the feeding frame, determining whether each livestock has engaged in feeding behavior, and determining the number of feeding livestock that have engaged in feeding behavior, and calculating the feeding typicality index based on the number of feeding livestock; determining the total number of livestock based on the first detection result, and calculating an average feed intake rate according to the number of livestock eating and the total number of livestock; Based on the average feed intake rate and feed time, each hour is divided into a typical feed intake period, an atypical feed intake period, or a no feed intake period by the third quartile; According to the typical feeding time period, the atypical feeding time period or the non-feeding time period, the feeding typicality index is plotted as a feeding typicality index graph.

5. The livestock behavior characterization method based on computer vision technology according to any one of claims 1 to 4, characterized in that: Before determining the activity index and feeding typicality index corresponding to each of the video stream queues based on the first detection results, the second detection results, and the image segmentation results, the method further includes: Performing image quality detection on each of the video clips to obtain a third detection result; If the third detection result indicates that the image quality does not meet the preset visual condition, the first detection result, the second detection result, and the image segmentation result of the video segment whose image quality does not meet the preset visual condition are deleted.

6. The livestock behavior characterization method based on computer vision technology according to claim 5, characterized in that: The performing of image quality detection on each of the video clips to obtain a third detection result includes: At least one image quality detection of strong light detection, brightness and darkness detection, rain and fog detection, and occlusion detection is performed on each of the video clips to obtain the third detection result.

7. The livestock behavior characterization method based on computer vision technology according to any one of claims 1 to 4, characterized in that: Before sequentially acquiring the video stream queues, the method further includes: Obtaining video clips collected by various video collection devices in a livestock house according to a preset sampling rule; wherein each of the video clips corresponds to a different time period; Determining the spatial order of the video clips according to the spatial positions of the video capture devices of the video clips, and determining the temporal order of the video clips according to the capture time of the video clips; The video segments are sorted according to the spatial order and the temporal order corresponding to the video segments to obtain the video stream queue.

8. A livestock behavior characterization device based on computer vision technology, characterized in that: include: A first acquisition module is configured to sequentially acquire a video stream queue; wherein the video stream queue includes video clips captured by multiple video capture devices in a livestock barn, and each of the video clips is arranged in a corresponding spatial order and a temporal order; a detection module configured to perform animal detection, animal head and tail detection, and image segmentation on each of the video clips in each video stream queue, to obtain a first detection result of the animal detection, a second detection result of the animal head and tail detection, and an image segmentation result of the image segmentation; The behavior analysis module is configured to determine the activity index and feeding typicality index corresponding to each video stream queue based on the first detection results, the second detection results and the image segmentation results, and perform behavior analysis on the livestock based on the activity index and the feeding typicality index.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the livestock behavior characterization method based on computer vision technology as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the livestock behavior characterization method based on computer vision technology as described in any one of claims 1 to 7 is implemented.

Images