Multi-view cattle identity recognition method fusing positioning information in grazing scene

Abstract

The invention discloses a multi-view cattle identity recognition method fusing positioning information in a grazing scene, and relates to the technical field of intelligent breeding. Comprising the steps of obtaining cattle position data and image data; the method comprises the following steps: constructing a cattle identity recognition model, deploying the final cattle identity recognition model into a core controller of a patrol robot, traversing each cattle in a pasture through a patrol function of the robot, capturing images by using a binocular camera, calculating a relative position relationship among the images, extracting a cattle contour, and carrying out the detection of the cattle contour in the patrol process of the robot. And constructing a cattle three-dimensional space, combining cattle necklace positioning information and a cattle identity recognition model recognition result, and realizing multi-view cattle identity recognition fused with the positioning information by using a dynamic weight. According to the invention, unmanned grazing and real-time monitoring and early warning are realized, the grazing efficiency is improved, the economic benefit of the industry is improved, the increasing demand of the society on high-quality beef is met, and the modernization development of animal husbandry is promoted.

Description

Technical Field

[0001] The present invention relates to the field of smart farming technology, and in particular to a multi-perspective cattle identification method that integrates positioning information in a grazing scenario. Background Art

[0002] As my country promotes the modernization of animal husbandry, it is particularly important to use the Internet, big data, artificial intelligence and other technologies to establish smart ranches for refined management. Compared with traditional manual grazing, robot grazing can achieve all-weather patrols and inspections. Combined with computer vision and artificial intelligence and other technologies, it can achieve non-contact cattle weight prediction, non-contact cattle body measurement, estrus, lameness, delivery and other abnormal behavior monitoring of cattle. However, these all require the ability to identify the identity of the cattle, so that the corresponding information can be entered into the cattle files, and the corresponding cattle numbers can be provided to the workers so that the workers know the abnormal cattle. Therefore, cattle identity recognition is a key link in realizing smart ranches.

[0003] Existing cattle identification methods mainly include contact and non-contact methods. Contact methods mainly include ear incisions, ear prints, hot iron branding, frozen branding, ear tags, collars, and radio frequency identification (RFID). Non-contact methods use cattle retina, iris, nose pattern, face, body, etc. Ear incisions, ear prints, branding and other marks do not meet the requirements of animal welfare. Simple contact labels are often very small and require close proximity to the cattle to see clearly. The RFID radio frequency range is relatively limited, making it difficult to perform long-distance multi-target identification. Although some collars have positioning and identification functions, they cannot currently be combined with vision. When using robots for grazing, it is difficult for the robot to match multiple cattle in the field of vision, the location of cattle, and the identity of cattle. Using biological features such as cattle retina, iris, nose pattern, face, and body, it is difficult to obtain ideal image data in grazing scenes due to factors such as the influence of light and the distance limit between the robot and the cattle. The recognition accuracy is not high and the recognition ability is limited.

[0004] Therefore, it is an urgent problem for technical personnel in this field to propose a multi-perspective cattle identification method that integrates positioning information in grazing scenarios to solve the difficulties existing in the prior art. Summary of the invention

[0005] In view of this, the present invention provides a multi-perspective cattle identification method that integrates positioning information in a grazing scenario. In a grazing scenario, the use of cattle identification technology helps to achieve unmanned grazing, real-time monitoring and early warning, improve grazing efficiency, enhance the economic benefits of the industry, meet the society's growing demand for high-quality beef, and promote the development of modernization in animal husbandry.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A multi-view cattle identification method integrating positioning information in a grazing scene comprises the following steps:

[0008] Determine the cattle herds for which data are to be collected, put collars on the cattle herds for which data are to be collected, and obtain cattle position data and image data;

[0009] Annotate the acquired cattle image data;

[0010] Construct a cattle image dataset and divide it into training set, validation set and test set;

[0011] Preprocess the training set, validation set, and test set;

[0012] Construct a cattle identification model, input the training set into the cattle identification model for training, update the model weight parameters through the loss function, and obtain a trained cattle identification model after several trainings;

[0013] The best cattle identification model is selected using the validation set, and the test set is input into the trained cattle identification model to obtain the identification results. The identification results are integrated using the maximum likelihood method to evaluate the cattle identification model and obtain the final cattle identification model.

[0014] The final cattle identification model is deployed to the core controller of the inspection robot, and the robot's patrol function is used to traverse each cattle in the pasture. During the patrol process, the robot uses a binocular camera to capture images, calculates the relative position relationship between each image, extracts the cattle outline, constructs the cattle's three-dimensional space, combines the cattle collar positioning information and the cattle identification model recognition results, and uses dynamic weights to realize multi-perspective cattle identification that integrates positioning information.

[0015] Optionally, the Huaxi cattle herd is selected as the data collection target, and collars are fitted on the Huaxi cattle herd whose data is to be collected. The collars include dual-mode receivers of the Beidou satellite navigation system and the global positioning system to obtain cattle location data and image data.

[0016] Optionally, the collected location data includes cattle location information and cattle identification number, distance to the target location output by the binocular camera, and location information of the inspection robot carrying the binocular camera;

[0017] The collected image data include multi-target cattle detection images, single-target cattle identification images and corresponding cattle identification numbers.

[0018] Optionally, use LabelMe to select cattle, the label is the cattle, and the collected single-target cattle images are classified according to the cattle identification number. The label is the cattle identification number and the shooting angle number, so as to annotate the obtained cattle image data.

[0019] Optionally, construct a cattle image dataset and divide it into training set, validation set and test set in a ratio of 8:1:1;

[0020] The cattle image dataset includes a cattle detection image dataset and a cattle identification image dataset; the cattle detection image dataset includes multi-target cattle images, the location information of each cattle and cattle labels; the cattle identification image dataset includes images of a cattle taken from multiple angles, the identification number and the angle sequence number.

[0021] Optionally, the training set is preprocessed and the validation set and test set are standardized and normalized using data augmentation methods, including cropping, flipping, rotation, and image transformation.

[0022] Optionally, the constructed cattle identification model is a structure of a deep learning model nested model, wherein a machine learning model is nested inside the deep learning model, the deep learning model adopts YOLOv8, the machine learning model uses metric learning, and uses a triplet loss function to update the model weight parameters.

[0023] Optionally, the evaluation indicators of the cattle identification model are the first accuracy and the average precision. The accuracy is calculated by dividing the number of correctly classified samples by the number of all samples. The first accuracy is calculated by the accuracy of the output classification with the highest probability. The formula is as follows:

[0024]

[0025] Among them, Accuracy is the first accuracy, mAP is the average precision, P i is the average precision of the ith category, k is the total number of categories, TP is the number of cases where the model correctly identifies positive examples as positive examples, TN is the number of cases where the model correctly identifies negative examples as negative examples, FP is the number of cases where the model incorrectly identifies negative examples as positive examples, and FN is the number of cases where the model incorrectly identifies positive examples as negative examples.

[0026] Optionally, the final cattle identification model is deployed to the core controller of the inspection robot, and the patrol function of the robot is used to traverse each cattle in the pasture. The specific content of cattle identification is as follows:

[0027] Scheduled and fixed-point patrol: The inspection robot patrols the ranch at pre-set time intervals and locations;

[0028] Collecting image data: When patrolling the pasture, the inspection robot will capture images of cattle and complete the data collection of cattle in the grazing scene;

[0029] Predict cattle identity: Call the cattle identification model that integrates positioning information deployed on the inspection robot to determine the cattle identification number and perform cattle identification.

[0030] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a multi-perspective cattle identification method integrating positioning information in a grazing scene, which has the following beneficial effects:

[0031] (1) The present invention provides a method for jointly identifying cattle from multiple perspectives. When multiple inspection robots jointly graze, they will capture images of the same cattle from different angles at the same time. The multiple images from different angles are input into a cattle identification model, and the maximum likelihood method is used to jointly identify the cattle's identity, thereby improving the accuracy of cattle identification by computer vision.

[0032] (2) The present invention integrates the positioning information of cattle, uses a binocular camera to measure the distance between the robot and the cattle, calculates the longitude, latitude and altitude of the cattle through the robot's accuracy, latitude, altitude and distance data, and compares it with the positioning information of the cattle collar to obtain the cattle's identity;

[0033] (3) The present invention combines the advantages of contact recognition and non-contact recognition to jointly identify the identity of cattle by combining the location information of cattle and images of cattle from multiple angles, and can effectively improve the accuracy of cattle identification in grazing scenarios;

[0034] (4) When using computer vision to identify cattle, it is time-consuming and laborious to build a database containing images of cattle from different angles. However, by using the positioning information of cattle and combining it with visual ranging, a database can be automatically established to train a cattle recognition model. This method has a high degree of automation and strong practicality, and can effectively reduce manpower and material resources, thereby improving economic and social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 A flow chart of a multi-view cattle identification method integrating positioning information in a grazing scene provided by the present invention;

[0037] Figure 2 A schematic diagram of the cattle identification model training process provided by the present invention;

[0038] Figure 3A schematic diagram of a four-legged walking inspection robot provided by the present invention;

[0039] Figure 4 A schematic diagram of the process of predicting the identity of cattle by the inspection robot provided by the present invention;

[0040] Figure 5 This is a specific flow chart of cattle identification provided by the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Reference Figure 1 As shown, the present invention discloses a multi-view cattle identification method integrating positioning information in a grazing scene, comprising the following steps:

[0043] Determine the cattle herds for which data are to be collected, put collars on the cattle herds for which data are to be collected, and obtain cattle position data and image data;

[0044] Annotate the acquired cattle image data;

[0045] Construct a cattle image dataset and divide it into training set, validation set and test set;

[0046] Preprocess the training set, validation set, and test set;

[0047] Construct a cattle identification model, input the training set into the cattle identification model for training, update the model weight parameters through the loss function, and obtain a trained cattle identification model after several trainings;

[0048] The best cattle identification model is selected using the validation set, and the test set is input into the trained cattle identification model to obtain the identification results. The identification results are integrated using the maximum likelihood method to evaluate the cattle identification model and obtain the final cattle identification model.

[0049] The final cattle identification model is deployed to the core controller of the inspection robot, and the robot's patrol function is used to traverse each cattle in the pasture. During the patrol process, the robot uses a binocular camera to capture images, calculates the relative position relationship between each image, extracts the cattle outline, constructs the cattle's three-dimensional space, combines the cattle collar positioning information and the cattle identification model recognition results, and uses dynamic weights to realize multi-perspective cattle identification that integrates positioning information.

[0050] Furthermore, the Huaxi cattle herd was selected as the data collection target, and collars were fitted on the Huaxi cattle herd whose data was to be collected. The collars included dual-mode receivers of the Beidou satellite navigation system and the global positioning system to obtain cattle location data and image data.

[0051] Specifically, the collar includes a dual-mode receiver for the BeiDou Navigation Satellite System (BDS) and the Global Positioning System (GPS), which can transmit back the cattle's identification number and location data.

[0052] Huaxi cattle have a unique pattern of yellow and white on their bodies. The pattern is determined by genetic genes, and different individuals have different gene combinations, resulting in the diversity and uniqueness of the pattern. The shape, size, and distribution of the pattern of each cow are different. The color depth, edge clarity, etc. of the pattern may also be different. These subtle differences can be used to distinguish individuals. The patterns of cattle are relatively stable during their life cycle and will not change significantly. Although some patterns may look similar, careful observation will reveal subtle differences. Although these differences are difficult for the human eye to capture, trained artificial intelligence can distinguish them well.

[0053] Furthermore, the collected location data includes cattle location information and cattle identification number, distance to the target location output by the binocular camera, and location information of the inspection robot carrying the binocular camera;

[0054] The collected image data include multi-target cattle detection images, single-target cattle identification images and corresponding cattle identification numbers.

[0055] Specifically, location data: use the BDS and GPS on the cattle collars to collect the longitude, latitude and altitude data of the designated cattle herd; use the image information captured by the binocular camera to collect the distance data between the camera and the shooting target; use the BDS and GPS on the robot to collect the longitude, latitude and altitude data of the robot, with positioning accuracy and distance error at the meter level;

[0056] Image data: Use a binocular camera to photograph a designated herd of cattle from multiple angles and lighting conditions at multiple time periods. The multi-target cattle detection images ensure that each image contains 3 or more targets, and the single-target cattle identification images ensure that each photo captures the central and main position of the target image.

[0057] Furthermore, LabelMe is used to select cattle, and the label is the cattle. The collected single-target cattle images are classified according to the cattle identification number. The label is the cattle identification number and the shooting angle number, so as to annotate the obtained cattle image data.

[0058] Furthermore, a cattle image dataset was constructed and divided into training set, validation set, and test set in a ratio of 8:1:1;

[0059] The cattle image dataset includes a cattle detection image dataset and a cattle identification image dataset; the cattle detection image dataset includes multi-target cattle images, the location information of each cattle and cattle labels; the cattle identification image dataset includes images of a cattle taken from multiple angles, the identification number and the angle sequence number.

[0060] Specifically, the training set is used for model training, the validation set is used to detect overfitting or underfitting and make adjustments, and the test set is used to evaluate the performance of the model on new data.

[0061] Since a herd may contain hundreds of cows, in order to build an identification image dataset for each cow, it is necessary to shoot each cow from different angles. If manual shooting is time-consuming and laborious, the cows can be driven through the channel, and multiple binocular cameras can be placed in different positions. The pre-trained cattle detection model can be combined with the positioning information to automatically enter the cattle identity and automatically build the corresponding cattle identification image dataset.

[0062] The evaluation indicators of the cattle detection model include mean average precision (mAP), precision, recall, and intersection over union (IoU). mAP is one of the most commonly used evaluation indicators in the field of target detection. It measures the average accuracy of the model on all target categories. Precision refers to the proportion of samples correctly predicted as positive. A high precision means that the model rarely makes mistakes when predicting positive samples. Recall refers to the proportion of all positive samples that are correctly predicted as positive. A high recall means that the model can find most of the positive samples. IoU measures the overlap between the predicted box and the true box. The higher the IoU, the higher the overlap between the predicted box and the true box.

[0063]

[0064] The Area of ​​overlap is the area covered by two areas, and the Area of ​​Union is the total area after the two areas are combined.

[0065] Furthermore, data augmentation methods are used to preprocess the training set, and to standardize and normalize the validation set and the test set. The data augmentation methods include cropping, flipping, rotation, and image transformation.

[0066] Further, such as Figure 2 As shown in the figure, the constructed cattle identification model is a structure of a deep learning model nested model, in which a machine learning model is nested inside the deep learning model. The deep learning model adopts YOLOv8, and the machine learning model uses metric learning and utilizes the triplet loss function to update the model weight parameters.

[0067] Furthermore, the evaluation indicators of the cattle identification model use the first accuracy and the average precision. The accuracy is calculated by dividing the number of correctly classified samples by the number of all samples. The first accuracy is calculated by the accuracy of the output classification with the highest probability. The formula is as follows:

[0068]

[0069] Among them, Accuracy is the first accuracy, mAP is the average precision, P i is the average precision of the ith category, k is the total number of categories, TP is the number of cases where the model correctly identifies positive examples as positive examples, TN is the number of cases where the model correctly identifies negative examples as negative examples, FP is the number of cases where the model incorrectly identifies negative examples as positive examples, and FN is the number of cases where the model incorrectly identifies positive examples as negative examples.

[0070] Further, such as Figure 4 As shown in the figure, the final cattle identification model is deployed to the core controller of the inspection robot. The patrol function of the robot is used to traverse each cattle in the pasture. The specific content of cattle identification is as follows:

[0071] Scheduled and fixed-point patrol: The inspection robot patrols the ranch at pre-set time intervals and locations;

[0072] Collecting image data: When patrolling the pasture, the inspection robot will capture images of cattle and complete the data collection of cattle in the grazing scene;

[0073] Predict cattle identity: Call the cattle identification model that integrates positioning information deployed on the inspection robot to determine the cattle identification number and perform cattle identification.

[0074] Specifically, Figure 3 As shown in the figure, the choice of inspection robot: In order to collect cattle images in grazing scenarios, in addition to using its own sensors, the inspection robot also needs an external visible light binocular camera. The collected data can be stored in local memory or uploaded to a cloud database via the network. Based on this requirement, Yushu Technology's four-legged walking robot can be selected.

[0075] In a specific embodiment, Figure 5As shown, in order to realize cattle identification, the cattle identification model is deployed to the core controller of the inspection robot. The patrol function of the inspection robot is used to traverse each cattle in the pasture and perform corresponding identification. The following are the detailed steps for cattle identification:

[0076] Fixed-time and fixed-point patrol: The inspection robot patrols the ranch at pre-set time intervals and locations. The time intervals and location ranges can be flexibly adjusted according to actual production needs.

[0077] Collecting image data: When patrolling the pasture, the inspection robot will capture images of cattle and complete the data collection of cattle in the grazing scene;

[0078] Predict cattle identity: Call the cattle identity recognition model deployed on the inspection robot, take the cattle image in the grazing scene as input, and use the deep learning model YOLOv8 to extract each target in the image containing multiple cattle to obtain multiple cattle images; compare each cattle image with the cattle images in the database, use metric learning, and the loss function is a triplet loss function to obtain the probability of predicting the cattle identity for each image, and then use maximum likelihood estimation to predict the cattle identity and probability using multiple images. The cattle identity with the largest comprehensive probability is the predicted cattle identity number. The binocular camera toolkit is used to extract the distance between each cow and the robot obtained by target detection, and then the relative position relationship between each photo is obtained based on the robot's position information. The Canny edge detection and OpenCV toolkit are used to obtain the outline of the cow in each picture and filter out background noise. Each outline is projected into three-dimensional space based on the relative position relationship to form a space surrounded by multiple outlines. The distance between each circle positioning point and the center of this space is calculated, and the cow identification number of the collar corresponding to the positioning point with the smallest distance is taken. At this time, the cow identification number is determined based on the dynamic weight combined with the image identification result.

[0079] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0080] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-view cattle identification method integrating positioning information in a grazing scene, characterized in that: The following steps are involved: Determine the cattle herds for which data are to be collected, put collars on the cattle herds for which data are to be collected, and obtain cattle position data and image data; Annotate the acquired cattle image data; Construct a cattle image dataset and divide it into training set, validation set and test set; Preprocess the training set, validation set, and test set; Construct a cattle identification model, input the training set into the cattle identification model for training, update the model weight parameters through the loss function, and obtain a trained cattle identification model after several trainings; The best cattle identification model is selected using the validation set, and the test set is input into the trained cattle identification model to obtain the identification results. The identification results are integrated using the maximum likelihood method to evaluate the cattle identification model and obtain the final cattle identification model. The final cattle identification model is deployed to the core controller of the inspection robot, and the robot's patrol function is used to traverse each cattle in the pasture. During the patrol process, the robot uses a binocular camera to capture images, calculates the relative position relationship between each image, extracts the cattle outline, constructs the cattle's three-dimensional space, combines the cattle collar positioning information and the cattle identification model recognition results, and uses dynamic weights to realize multi-perspective cattle identification that integrates positioning information.

2. According to claim 1, a multi-view cattle identification method integrating positioning information in a grazing scene is characterized in that: The Huaxi cattle herd was selected as the data collection target, and collars were worn on the Huaxi cattle herd to be collected. The collars included dual-mode receivers of the Beidou satellite navigation system and the global positioning system to obtain cattle location data and image data.

3. According to claim 2, a multi-view cattle identification method integrating positioning information in a grazing scene is characterized in that: The collected location data includes the location information and identification number of the cattle, the distance to the target location output by the binocular camera, and the location information of the inspection robot carrying the binocular camera; The collected image data include multi-target cattle detection images, single-target cattle identification images and corresponding cattle identification numbers.

4. According to claim 1, a multi-view cattle identification method integrating positioning information in a grazing scene is characterized in that: LabelMe is used to select cattle, and the label is the cattle. The collected single-target cattle images are classified according to the cattle identification number. The label is the cattle identification number and the shooting angle number, so as to annotate the obtained cattle image data.

5. The multi-view cattle identification method for grazing scenes by integrating positioning information according to claim 1, characterized in that: A cattle image dataset was constructed and divided into training set, validation set, and test set in a ratio of 8:1:1; The cattle image dataset includes cattle detection image dataset and cattle recognition image dataset; The cattle detection image dataset includes multi-target cattle images, location information of each cattle, and cattle labels; The cattle identification image dataset includes images of a cow taken from multiple angles, its identification number, and angle sequence number.

6. The multi-view cattle identification method for grazing scenes by integrating positioning information according to claim 1, characterized in that: The training set is preprocessed using data augmentation methods, and the validation set and test set are standardized and normalized. The data augmentation methods include cropping, flipping, rotation, and image transformation.

7. The multi-view cattle identification method for grazing scenes by integrating positioning information according to claim 1, characterized in that: The constructed cattle identification model has a structure of a deep learning model nested in a model, in which a machine learning model is nested inside the deep learning model. The deep learning model adopts YOLOv8, and the machine learning model uses metric learning and utilizes the triplet loss function to update the model weight parameters.

8. The multi-view cattle identification method for grazing scenes by integrating positioning information according to claim 1, characterized in that: The evaluation indicators of the cattle identification model use the first accuracy and the average precision. The accuracy is calculated by dividing the number of correctly classified samples by the number of all samples. The first accuracy is calculated by the accuracy of the output classification with the highest probability. The formula is as follows: Among them, Accuracy is the first accuracy, mAP is the average precision, P i is the average precision of the ith category, k is the total number of categories, TP is the number of cases where the model correctly identifies positive examples as positive examples, TN is the number of cases where the model correctly identifies negative examples as negative examples, FP is the number of cases where the model incorrectly identifies negative examples as positive examples, and FN is the number of cases where the model incorrectly identifies positive examples as negative examples.

9. The multi-view cattle identification method for grazing scenes by integrating positioning information according to claim 1, characterized in that: The final cattle identification model is deployed to the core controller of the inspection robot. The patrol function of the robot is used to traverse each cattle in the pasture. The specific contents of cattle identification are as follows: Scheduled and fixed-point patrol: The inspection robot patrols the ranch at pre-set time intervals and locations; Collecting image data: When patrolling the pasture, the inspection robot will capture images of cattle and complete the data collection of cattle in the grazing scene; Predict cattle identity: Call the cattle identification model that integrates positioning information deployed on the inspection robot to determine the cattle identification number and perform cattle identification.

Images