Cow behavior recognition method based on interaction area, program product and device

Abstract

The application discloses a kind of based on interactive area's cow behavior identification method, program product and equipment, method includes: obtaining multiple interactive area images of multiple behavior types of cow, the interactive area mask of each behavior type is extracted;The image of the cow to be identified is input into cow behavior detection model, the predicted target frame of cow and the Logits value of the behavior of each behavior type of cow are obtained;The overlapping proportion of the predicted target frame of cow and the interactive area mask of each behavior type is calculated, and the spatial confidence is calculated with the overlapping proportion as coefficient;The exponential function of the Logits value of the behavior of each behavior type of cow is calculated, and the proportion in the sum of the exponential function of the Logits value of all behavior types is taken as visual confidence;According to spatial confidence and visual confidence, the total confidence of each behavior type is calculated;Select the behavior type with highest total confidence as the predicted cow behavior type output.The application has high identification accuracy and high robustness.

Description

Technical Field

[0001] This invention relates to the field of machine learning, and in particular to a method, program product, and device for recognizing cow behavior based on interactive regions. Background Technology

[0002] With the development of modern animal husbandry, higher demands have been placed on the feeding management and health monitoring of dairy cows. Behavioral recognition of dairy cows is crucial for assessing their physical condition and feeding environment, allowing for the timely detection of abnormalities. Traditional dairy cow behavioral recognition primarily relies on manual observation, which is inefficient and has a low recognition rate.

[0003] With the development of artificial intelligence, some AI and machine learning technologies have been applied to dairy cow behavior recognition. Currently, AI-based dairy cow behavior recognition methods mainly rely on the animal's posture and body features in images for identification. However, in actual farming environments, dairy cows are densely distributed, and their activity areas are often obscured by structures such as fences and enclosures. Frequent mutual occlusion between individuals makes it difficult to fully present key parts of the cow, resulting in incomplete behavioral feature information and affecting recognition accuracy. Furthermore, the frequency of different dairy cow behaviors varies significantly, forming a clear long-tail distribution of data. Behavior recognition algorithms perform poorly on behavior categories with scarce samples, struggling to maintain good generalization performance and robustness, thus limiting their reliability and practicality in real-world applications. Summary of the Invention

[0004] To address the problems existing in the prior art, the purpose of this invention is to provide a method, program product, and device for recognizing dairy cow behavior based on interactive areas, which has high recognition accuracy and robustness.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0006] A method for recognizing cow behavior based on interactive regions, the method comprising the following steps:

[0007] (1) Obtain images of multiple interaction regions where various behaviors of cows are located, and extract the interaction region mask for each behavior type from them;

[0008] (2) Input the cow image to be identified into the trained cow behavior detection model to obtain the predicted target box of the cow and the Logits value of the cow behavior belonging to each behavior type;

[0009] (3) Calculate the overlap ratio between the predicted target box of the cow and the interaction region mask of each behavior type, and use the overlap ratio as a coefficient to calculate the spatial confidence of the cow behavior belonging to each behavior type;

[0010] (4) Calculate the exponential function of the Logits value of each behavior type and the proportion of the exponential function of the Logits value of each behavior type in the sum of all behavior types. Use this as the visual confidence that the behavior belongs to the corresponding behavior type.

[0011] (5) Calculate the total confidence score of the cow's behavior belonging to each behavior type based on spatial confidence score and visual confidence score;

[0012] (6) Select the behavior type with the highest total confidence as the predicted cow behavior type output.

[0013] Furthermore, step (1) specifically includes:

[0014] (1.1) Establish a set of all behavioral types of dairy cows. ;

[0015] (1.2) Obtaining the set Extract the interaction region of each behavior type from the interaction region image;

[0016] (1.3) Assign the value of 1 to the pixel at the location of the interaction area in the interaction area image of each behavior type, and assign the value of 0 to the other pixels to obtain the interaction area mask of each behavior type.

[0017] Furthermore, the cow behavior detection model mentioned in step (2) is specifically YOLOv5, which is used to predict the predicted target box of the cow and the Logits value of the cow behavior belonging to each behavior type based on the cow image.

[0018] Furthermore, the formula for calculating the spatial confidence level is as follows:

[0019]

[0020]

[0021] ,

[0022] in, This indicates that the cow's behavior belongs to the i-th cow behavior type. Spatial confidence, Represents the predicted target box Behavioral types of dairy cows Interactive region mask The overlap ratio, For intersection operations, To calculate the area of ​​the region, As the attenuation factor, To predict the target box Center to Interactive Area Mask The Euclidean distance at the center It is the stability constant. This represents the set of all behavior types, where N represents the number of behavior types.

[0023] Furthermore, the formula for calculating visual confidence in step (4) is as follows:

[0024]

[0025] in, This indicates that the cow's behavior belongs to the i-th cow behavior type. Visual confidence, , The behavior of dairy cows belongs to , Logits value, , for , temperature coefficient, This represents the set of all behavior types.

[0026] Furthermore, the formula for calculating the temperature coefficient is as follows:

[0027]

[0028] In the formula, The number of times each of the cow's behavioral types occurred. For the i-th cow behavior type The number of times it occurs.

[0029] Furthermore, the formula for calculating the total confidence level in step (5) is as follows:

[0030]

[0031] in, , , The cow behaviors belong to the i-th cow behavior type. Total confidence, spatial confidence, and visual confidence. , The cow behaviors belong to the j-th cow behavior type. Spatial confidence, visual confidence, This represents the set of all behavior types.

[0032] Furthermore, in step (6), the following formula is used for calculation:

[0033]

[0034] In the formula, This represents the behavior type number with the highest overall confidence level. The cow's behavior belongs to the i-th cow behavior type. Total confidence level, select the first Individual cow behavior types The output is the predicted type of cow behavior.

[0035] A computer program product includes a computer program / instructions that, when executed by a processor, implement the above-described method.

[0036] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.

[0037] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention provides a method for recognizing cow behavior based on interactive regions. The method first establishes a mask for the cow's interactive regions; then, it obtains the target bounding box and preliminary behavior classification results of the cow through a target detection algorithm; finally, it combines the relationship between the target bounding box of the cow in the image and the interactive region mask to correct the behavior results, obtaining more accurate recognition results. This invention comprehensively considers the spatial distribution characteristics of cow behavior in its living environment, and utilizes the correlation between functional regions and behavior to effectively improve the robustness and accuracy of behavior recognition. Furthermore, this invention optimizes the original recognition results through post-processing without changing the existing detection model structure, exhibiting good compatibility and adaptability, and is applicable to various behavior recognition scenarios. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the cow behavior recognition method based on interactive regions provided by the present invention.

[0039] Figure 2 This is a structural diagram of the computer device provided by the present invention. Detailed Implementation

[0040] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0041] Example 1

[0042] This invention provides a method for recognizing cow behavior based on interactive regions, such as... Figure 1 As shown, the method includes the following steps:

[0043] S101. Obtain multiple interaction area images of various cow behavior types, and extract the interaction area mask for each behavior type.

[0044] Specifically, the steps include:

[0045] S1011. Establish a set of all behavioral types of dairy cows. .

[0046] In the context of dairy cow behavior recognition, dairy cow behaviors can be divided into four categories: (1) Standing: The cow stands upright on all fours, possibly in a resting or moving state. (2) Lying: The cow lies close to the ground with its limbs bent, usually in a resting or ruminating stage. (3) Feeding: The cow approaches the feeding trough and exhibits feeding behaviors, such as opening its mouth to chew or lowering its head to eat. (4) Drinking: The cow approaches the water trough and exhibits the behavior of lowering its head to touch the water surface. Therefore, a set is established. i=1,2,3,4 represent standing, lying down, eating, and drinking, respectively.

[0047] S1012, Obtain the set The interaction area of ​​each behavior type is extracted from the interaction area image.

[0048] The dairy cow's living environment can be divided into different interaction zones (the spatial range in which dairy cow behaviors occur) based on the characteristics of behavioral distribution. Behavior occurrence is strongly correlated with these interaction zones. For example, in the water trough area, dairy cows almost exclusively stand and drink, rarely lying down; in the feed trough area, dairy cows typically stand or eat; while inside the barn, lying down and standing are more common, while eating and drinking are almost impossible. Therefore, extracting interaction zones can improve the accuracy of behavior recognition.

[0049] S1013. Assign a value of 1 to the pixel at the location of the interaction area in the interaction area image of each behavior type, and assign a value of 0 to other pixels to obtain the interaction area mask of each behavior type.

[0050] In terms of behavior For example, collecting images of the interactive areas of dairy cows. ,Sure In the image Interactive area in Acquisition behavior Interactive region mask The interaction region mask In the interactive area The pixels within the mask are assigned a value of 1. With the image The number of pixels is consistent.

[0051] S102. Input the cow image to be identified into the trained cow behavior detection model to obtain the predicted target box of the cow and the Logits value of the cow behavior belonging to each behavior type.

[0052] Specifically, the cow behavior detection model is YOLOv5, but other types of neural networks can also be used to predict the bounding boxes of cows based on cow images. And the Logits values ​​for each behavior type of the cow. Logits are the raw, unprocessed scores or ratings of the model's output layer. These predictions are not processed by any activation function and are therefore raw, unnormalized scores. These values ​​can be viewed as a measure of the model's probability for each category, but they do not directly represent probabilities.

[0053] S103. Calculate the overlap ratio between the predicted target box of the cow and the interaction region mask of each behavior type, and use the overlap ratio as a coefficient to calculate the spatial confidence of the cow's behavior belonging to each behavior type.

[0054] This step first constructs dynamic spatial prior information to quantify the correlation between the target and the interaction region, and establishes a spatial confidence distribution. The degree of matching is measured by calculating the coverage ratio between the target bounding box and the interaction region mask for each behavior type. The calculation method is as follows:

[0055]

[0056] , ,

[0057] in To calculate the area of ​​the region, This is for intersection operations. Represents the target box Completely within this interaction area, Represents the target box It is not in that area at all.

[0058] Then, the confidence score of the spatial prior is calculated to measure the behavior. The reasonableness of the area where the target box is located is calculated as follows:

[0059]

[0060] in As the attenuation factor, Let Euclidean distance be the distance from the center of the cow's target bounding box to the geometric center of the interaction area. It is the stability constant.

[0061] S104. Calculate the exponential function of the Logits value of each behavior type and the proportion of the exponential function of the Logits value of each behavior type in the sum of all behavior types. Use this as the visual confidence that the cow behavior belongs to the corresponding behavior type.

[0062] To address the issue of uneven behavior distribution, the behavior prediction results of the YOLOv5 model are adjusted by temperature scaling to obtain visual confidence.

[0063] The formula for calculating the visual confidence level is as follows:

[0064]

[0065] in, This indicates that the cow's behavior belongs to the i-th cow behavior type. Visual confidence, , The behavior of dairy cows belongs to , Logits value, , for , The temperature coefficient is used to expand the original Logits output for low-frequency behavior, and the specific calculation formula is as follows:

[0066]

[0067] In the formula, The number of times each of the cow's behavioral types occurred. For the i-th cow behavior type The number of times it occurs.

[0068] S105. Calculate the total confidence score of each behavior type of dairy cow based on spatial confidence score and visual confidence score.

[0069] To integrate spatial prior constraints and visual confidence calibration information, the formula for calculating the total confidence is as follows:

[0070]

[0071] in, , , The cow behaviors belong to the i-th cow behavior type. Total confidence, spatial confidence, and visual confidence. , The cow behaviors belong to the j-th cow behavior type. Spatial confidence and visual confidence.

[0072] S106. Select the behavior type with the highest total confidence as the predicted cow behavior type output.

[0073] The specific calculation is as follows:

[0074]

[0075] In the formula, This represents the behavior type number with the highest overall confidence level. The cow's behavior belongs to the i-th cow behavior type. Total confidence level, select the first Individual cow behavior types The output is the predicted type of cow behavior.

[0076] Example 2

[0077] This invention also provides a computer product, such as an app on a mobile phone or tablet, or an installer on a computer. This product includes a computer program / instructions that, when executed by a processor, implement the method described in Embodiment 1. The code for a computer-executable program that performs the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0078] Example 3

[0079] Figure 2 This is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. The embodiment of the present invention provides services for the implementation of the method of the first embodiment of the present invention. Figure 2 As shown, the device may include: a memory 301 storing a computer-executable program; a processor 302 coupled to the memory 301; the processor 302 calls the computer-executable program stored in the memory 301 to perform the steps in the method described in Embodiment 1.

[0080] Memory 301 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, memory 301 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). A program / utility having a set (at least one) of program modules may be stored, for example, in memory 301. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The computer-executable program of the program modules typically performs the functions and / or methods described in the embodiments of the present invention.

[0081] The processor 302 executes various functional applications and data processing by running programs stored in the memory 301, such as implementing the method provided in Embodiment 1 of the present invention.

[0082] The code of a computer executable program can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

[0083] It should be understood that the embodiments and descriptions above are only the principles, main features and advantages of the present invention. Various changes and modifications can be made to the present invention without departing from the spirit and scope of the invention, and all such changes and modifications fall within the protection scope of the present invention.

Claims

1. A method for recognizing cow behavior based on interactive regions, characterized in that, Includes the following steps: (1) Obtain images of multiple interaction regions where various behaviors of cows are located, and extract the interaction region mask for each behavior type from them; (2) Input the cow image to be identified into the trained cow behavior detection model to obtain the predicted target box of the cow and the Logits value of the cow behavior belonging to each behavior type; (3) Calculate the overlap ratio between the predicted target box of the cow and the interaction region mask of each behavior type, and use the overlap ratio as a coefficient to calculate the spatial confidence of the cow behavior belonging to each behavior type; (4) Calculate the exponential function of the Logits value of each behavior type and the proportion of the exponential function of the Logits value of each behavior type in the sum of all behavior types. Use this as the visual confidence that the behavior belongs to the corresponding behavior type. (5) Calculate the total confidence score of the cow's behavior belonging to each behavior type based on spatial confidence score and visual confidence score; (6) Select the behavior type with the highest total confidence as the predicted cow behavior type output; The formula for calculating the spatial confidence level in step (3) is as follows: , , ， , in, This indicates that the cow's behavior belongs to the i-th cow behavior type. Spatial confidence, Represents the predicted target box Behavioral types of dairy cows Interactive region mask The overlap ratio, For intersection operations, To calculate the area of ​​the region, As the attenuation factor, To predict the target box Center to Interactive Area Mask The Euclidean distance at the center It is the stability constant. N represents the set of all behavior types, where N represents the number of behavior types. The formula for calculating visual confidence in step (4) is as follows: , in, This indicates that the cow's behavior belongs to the i-th cow behavior type. Visual confidence, , The behavior of dairy cows belongs to , Logits value, , for , temperature coefficient, This represents the set of all behavior types.

2. The method for recognizing cow behavior based on interactive regions according to claim 1, characterized in that, Step (1) specifically includes: (1.1) Establish a set of all behavioral types of dairy cows. ; (1.2) Obtaining the set Extract the interaction region of each behavior type from the interaction region image; (1.3) Assign the value of 1 to the pixel at the location of the interaction area in the interaction area image of each behavior type, and assign the value of 0 to the other pixels to obtain the interaction area mask of each behavior type.

3. The method for recognizing cow behavior based on interactive regions according to claim 1, characterized in that, The cow behavior detection model mentioned in step (2) is YOLOv5, which is used to predict the target box of the cow and the Logits value of the cow behavior belonging to each behavior type based on the cow image.

4. The method for recognizing cow behavior based on interactive regions according to claim 1, characterized in that, The formula for calculating the temperature coefficient is: , In the formula, The number of times each of the cow's behavioral types occurred. For the i-th cow behavior type The number of times it occurs.

5. The method for recognizing cow behavior based on interactive regions according to claim 1, characterized in that, The formula for calculating the total confidence level in step (5) is as follows: , in, , , The cow behaviors belong to the i-th cow behavior type. Total confidence, spatial confidence, and visual confidence. , The cow behaviors belong to the j-th cow behavior type. Spatial confidence, visual confidence, This represents the set of all behavior types.

6. The method for recognizing cow behavior based on interactive regions according to claim 1, characterized in that, In step (6), calculate according to the following formula: , In the formula, This represents the behavior type number with the highest overall confidence level. The cow's behavior belongs to the i-th cow behavior type. Total confidence level, select the first Individual cow behavior types The output is the predicted type of cow behavior.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method of any one of claims 1-6.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the method as described in any one of claims 1-6.

Images