METHOD AND DEVICE FOR TRACKING ANIMALS

Description

[0001] The invention relates to a method for tracking animals, in particular cattle, and to a device for carrying out this method.

[0002] In livestock farming, particularly in dairy farming, there is a strong trend, both internationally and nationally, towards the use of technology and electronics for monitoring animal health and managing herds. Conventional herd management systems typically equip animals with an electronically readable identifier, such as an RFID chip, often in the form of an ear tag. Such a system requires suitable readers positioned at specific locations in a barn or pasture, such as at a feeding station, to detect the presence of animals within sufficient proximity. This type of system involves significant hardware requirements and, consequently, high investment costs.Furthermore, such a system does not allow continuous monitoring of the animals, as presence data is only obtained when the animal is in sufficient proximity to the reading device.

[0003] In animal health monitoring systems, cows must be equipped with accelerometers, which are placed, for example, on the ear, neck, or throat. The data collected by the sensors is read via a wireless connection and analyzed by a central computer. Even with this system, reading the sensor data is a complex process. The accelerometers allow the recording of the characteristic head and neck movement patterns of cows during estrus. It is known that cows' activity increases significantly during estrus. Furthermore, reduced rumination time is observed during this period. Early detection of estrus, as well as the identification of physiological and pathological processes, is considered particularly desirable for dairy farms.

[0004] Other sensor systems for detecting estrus may include a rumen sensor that measures the forestomach temperature. The temperature profile is influenced by water intake and allows for the detection of cyclical estrus patterns.

[0005] Various systems for monitoring and tracking animals are disclosed in documents CN 111 294 565 A, WO 2011 / 039112 A2, CN 108 990 831 A, US 2019 / 037800 A1, CN 111 685 060 A and CN 108 717 523 B.

[0006] The present invention aims to provide a method and a device for tracking and monitoring animals, which requires minimal investment and allows for reliable and continuous animal data collection. In particular, it aims to ensure continuous local tracking as well as continuous monitoring of health-related parameters. Data collection should be automated as much as possible and therefore require no human intervention.

[0007] To solve this problem, the invention, according to a first aspect, provides a method for tracking animals, in particular cattle, comprising the following steps: Continuous or intermittent digital image acquisition of multiple animals to obtain digital image data; identification of individual animals using a feature extraction algorithm from the image data or a comparison algorithm to compare the image data with previously acquired image data; tracking the locomotion of the identified animals, preferably using a Kalman filter, to obtain initial movement data; analysis of the initial movement data to create a locomotion profile of each animal; tracking the movement of individual body parts of the animals using body features recognized by a feature extraction algorithm and / or recognizing and tracking the body position of the animals from the image data to obtain secondary movement data; analysis of the secondary movement data to determine the movement sequence and / or activity of the animals in order to create a behavioral profile of each animal.wherein the comparison algorithm is designed to determine a similarity between an identification vector generated from the image data and a previously generated identification vector, wherein the comparison algorithm is specifically designed to determine the cosine similarity, wherein the previously generated identification vector is adapted to the identification vector generated from the image data by means of exponential smoothing, or, if the similarity between the identification vector generated from the image data and the previously generated identification vector is too low, a new identification vector is stored.

[0008] The invention thus eliminates the need for animal-mounted sensor or identification systems and instead relies on electronic image data acquisition, which can be achieved using a camera or similar device. It is therefore sufficient to equip the area where the animals to be tracked are located, such as a barn or an outdoor area, with at least one camera. The acquired image data is used for identifying individual animals, tracking their location, and analyzing their behavior. The image data is acquired continuously or intermittently, i.e., at regular intervals, so that, for example, a video is recorded.

[0009] To identify individual animals, the image data can first be preferably analyzed to recognize image areas in which the individual animals are located. For this purpose, an artificial neural network is used, for example, which generates enclosing rectangles (position left, top, right, and bottom) around each animal. To improve the quality of further processing, this neural network can additionally recognize a rotation angle (position left, top, right, bottom, and rotation of the entire rectangle), as this results in fewer overlaps.

[0010] The identification of individual animals is achieved, for example, using a feature extraction algorithm from the image data or from the image areas where an animal has been detected, as described above. Feature extraction algorithms are well-known in the field of electronic image processing and are based, for example, on segmentation methods. These methods create coherent regions of an image by grouping neighboring pixels according to a specific homogeneity criterion, thus yielding edge profiles and the like. With image data obtained from cattle, feature extraction can be used, for example, to detect characteristics of the animal's individual coat pattern. These characteristics are then compared with reference data previously collected from the animals to determine the presence of a specific animal within the camera's field of view.

[0011] Alternatively, individual animals can be identified using a comparison algorithm to compare the image data with previously acquired image data stored in a database. According to the invention, the comparison algorithm is designed to determine the similarity between an identification vector generated from the image data and a previously generated identification vector, specifically determining the cosine similarity. The comparison algorithm yields a matrix of match points. This is followed by an assignment of the identified animals to previously known animals, for example, using the Hungarian method. This assignment preferably also includes the individual match point value from the matrix generated above. If this value falls below a certain threshold, a temporary animal record can be created.If the assignment corresponds to an animal individual in the database with a sufficiently high probability, all data can be assigned to that individual. If the assignment corresponds to a temporary animal individual with a sufficiently high probability, all data can be assigned to that temporary animal individual. If the assigned temporary animal individual can then also be assigned to an animal individual in the database using the current identification vector, all data from the temporary animal individual can be added to the data stored in the database, and the temporary animal individual can be deleted. If this does not occur over a longer period, the temporary animal individual can be preferentially entered into the database.

[0012] In this process, multiple identification vectors are stored for each individual animal. If the current identification vector differs only marginally from a stored one, the stored identification vector is adjusted using exponential smoothing to account for any changes in the animal (e.g., growth). Otherwise, the current identification vector is stored in addition to the stored one. To prevent the number of stored vectors from becoming too large, the number can be limited. When a limit is exceeded, an older stored vector is deleted. It is possible to record how often each vector has enabled an assignment, and then delete the one with the lowest number of occurrences.

[0013] Once (or before) an animal has been identified in one of the ways described above, its movement relative to the camera can be detected and its movement path determined by comparing sequentially captured image data. Image data from multiple cameras can be used to track an animal as it moves from the field of view of one camera to the field of view of a neighboring camera. If the animals are tracked over a specific period, a movement profile can be obtained for each animal. Analyzing this movement profile allows conclusions to be drawn about the motor activity and health status of each individual animal.

[0014] The processing of digital image data allows not only the tracking of animals but also the analysis of their behavior. For this purpose, the physical characteristics identified through feature extraction are used not only to identify and track the animals' movements but also to recognize individual body parts. Based on the recognition of individual body parts, their movement or changes in their relative arrangement can be analyzed to detect the animals' body position, movement patterns, and / or activities, thereby creating a behavioral profile for each animal.

[0015] Alternatively, the behavioral profile can be generated without feature extraction, namely directly based on the image data, by subjecting the image data to recognition and tracking of the animals' body position, and / or by feeding the image data into an independent artificial neural network.

[0016] A distinction is made here between the movement sequence and the locomotion of an animal. Locomotion refers to a change in the animal's location. Movement sequence refers, for example, to changes in body position over time, such as standing, lying down, mounting another animal, and so on. Additionally, movement sequence detection can also include recognizing whether the animal is currently eating or drinking. This is done, for example, by determining a specific dwell time at the feeding trough or watering basin. This estimation becomes more accurate through the recognition of body position, as the precise position of the skull provides a better indication of whether the animal is feeding or drinking.

[0017] The results obtained by the method according to the invention, namely the movement profile and the behavior profile, can be analyzed in a subsequent step to calculate a characteristic value that is representative of animal welfare. In particular, movement profiles and behavior profiles can be created solely on the basis of the digital image data, which, in combination, allow for a particularly detailed and reliable analysis of animal welfare.

[0018] The locomotion profile and behavioral profile can encompass a wide variety of parameters, such as feeding times, lying times, distance traveled, standing times, locations visited, feed intake, feeding duration, estrus behavior, calving, pathological behavior, and / or the repurchase rate of each animal. Preferably, at least three, and preferably at least four, parameters are recorded, selected from the group of parameters mentioned above.

[0019] For example, the animal's average speed of movement can be recorded over a certain period and stored as a baseline value. This baseline value can then be used to infer increased or decreased activity, depending on whether the animal falls below or exceeds the baseline. Furthermore, a baseline value can be generated that is representative of how often the animal changes position (ratio of lying time to moving time), which can also be included in the activity analysis by determining whether the activity level exceeds or falls below the baseline value.

[0020] The method according to the invention preferably also allows for calving detection based on the reduced activity of the animal during birth. Shortly before calving, changes in activity and feeding behavior may occur. During birth, a cow prefers to lie down, for example, which can be detected by means of pose recognition. Furthermore, an unidentifiable individual cow should be identifiable shortly after calving.

[0021] Additionally, an audio analysis is possible to detect any bellowing of the cow during labor.

[0022] The calculation of a key indicator representative of animal welfare can, for example, include determining the distance traveled per day, the utilization of the available space and, if necessary, the detection of the degree of soiling, whereby the key indicator is mathematically calculated from the aforementioned values.

[0023] According to a preferred embodiment of the invention, the movement and behavioral profiles are recorded continuously or at regular intervals throughout the animals' lives. This provides a comprehensive overview of animal welfare during the animal's lifetime. In the case of animals whose meat is used in the food industry after slaughter, this data can be used, for example, to provide consumers with information about the welfare of the specific animal offered for sale or consumption in retail or restaurants. A high level of animal welfare is present, for example, if the animal was kept in a species-appropriate manner, had sufficient space to roam, and, if applicable, did not exhibit any pathological behavior.

[0024] Within the framework of the inventive method, the first movement data can preferably be analyzed to determine the current location of the animals. Furthermore, the current location of the animals can be compared with a predefined location of a feeding station to record the feeding duration of each animal. The feeding duration allows conclusions to be drawn about the animal's welfare and health status.

[0025] A preferred approach is to analyze the second set of movement data in order to record when the animals are standing and when they are lying down, and to calculate standing and lying times.

[0026] It is also preferable to analyze the second set of movement data to record the rumination rate and / or rumination duration of each animal. The rumination rate and / or rumination duration can be used as indicators of estrus behavior. For the detection and analysis of rumination, the lower jaw and nostrils of a cow can be identified during feature extraction, and the relative movement of these three points to each other can be evaluated.

[0027] The identification of animals based on feature extraction from image data can be optimized by applying artificial intelligence. In this context, a preferred embodiment of the invention provides that the identification of an animal using the feature extraction algorithm comprises two steps: in a first step, an animal-specific feature map is created from the digital image data; and in a second step, the animal-specific feature map is assigned to an animal, the assignment comprising the following steps: Providing a plurality of artificial neural networks, each network being individualized or trained with respect to a particular animal, feeding the feature map to each of the plurality of artificial neural networks to obtain a match score, comparing the match scores obtained by the artificial neural networks, and identifying the animal whose artificial neural network, individualized for that animal, has the highest match score.

[0028] According to a second aspect, the invention provides a device for tracking animals, in particular cattle, comprising at least one digital image acquisition unit and an electronic evaluation unit, to which the digital image data of the at least one digital image acquisition unit are fed, wherein the evaluation unit is set up to carry out the following steps: Identifying individual animals using a feature extraction algorithm from the image data or a comparison algorithm to compare the image data with previously acquired image data; tracking the locomotion of the identified animals to obtain initial movement data; analyzing the initial movement data to create a locomotion profile of each animal; tracking the movement of individual body parts of the animals using body features recognized by a feature extraction algorithm and / or recognizing and tracking the body position of the animals from the image data to obtain second movement data; analyzing the second movement data to determine the movement sequence and / or activity of the animals in order to create a behavioral profile of each animal.

[0029] A preferred design provides that the electronic evaluation unit is set up to analyze the initial movement data and record the current location of the animals.

[0030] Preferably, the electronic evaluation unit is set up to compare the current location of the animals with a predefined location of a feeding station in order to record the feeding duration of each animal.

[0031] The electronic evaluation unit is advantageously set up to analyze the second movement data in order to record the animals' standing and lying positions and to calculate standing and lying times.

[0032] According to a preferred embodiment, the electronic evaluation unit is set up to analyze the second movement data in order to record the rumination rate of each animal.

[0033] In particular, it may be provided that the locomotion profile and the behavior profile include at least three, preferably at least four, parameters selected from the group consisting of feeding times, lying times, locomotion times, locomotion distance, standing times, places of residence, feed removal, feeding duration, estrus behavior, calving, pathological behavior and / or the repurchase rate of each animal.

[0034] Preferably, the electronic evaluation unit is set up to determine the welfare of each animal from its movement profile and behavior profile, preferably to calculate it in the form of a key figure.

[0035] The electronic evaluation unit is particularly preferably configured to identify an animal using the feature extraction algorithm in two steps, wherein in a first step an animal-specific feature map is created from the digital image data and in a second step the animal-specific feature map is assigned to an animal, the assignment comprising the following steps: Providing a plurality of artificial neural networks, each network being individualized with respect to a specific one of the plurality of animals, feeding the feature map to each of the plurality of artificial neural networks to obtain a match score, comparing the match scores obtained by the artificial neural networks, and identifying the animal whose artificial neural network, individualized for that animal, has the highest match score.

[0036] The invention is explained in more detail below using a non-restrictive embodiment.

[0037] The example shown concerns a system and a procedure for the identification and analysis of cattle, which includes the following processing steps: a) Image data import b) Instance segmentation c) Extraction of each individual cow d) Keypoint detection e) Person re-identification f) Coat lightness determination near certain keypoints g) Ear tag detection h) Position detection i) Temporal analysis of position j) Cow identification k) Evaluation l) Saving the acquired data

[0038] These analysis steps are partly dependent on each other, but can also be carried out partly in parallel. a) Reading the image data

[0039] The image data can be read in the form of individual images, recorded videos, or live data from a camera, for example. b) Instance Segmentation

[0040] As part of the image processing, the images are segmented as a preliminary step to feature extraction. The goal is to identify the image areas containing individual cattle within the imported image data. This can be achieved using a deep learning instance segmentation model. A mask RCNN model is a suitable option. This model first identifies interesting regions, known as "regions of interest (ROIs)," and then analyzes these ROIs in more detail. The output is a unique image mask for each identified cattle.

[0041] Existing systems can be trained for the specific task of recognizing cattle. This is done using sample data consisting of input and desired output data. c) Cutting out each individual cow

[0042] Using the image masks obtained through instance segmentation, the individual cattle are then cut out into separate images. Furthermore, any potentially distracting background will be replaced with a color that is highly unlikely to be present in the image data (e.g., fuchsia). d) Keypoint detection

[0043] These individual images of the cattle are then analyzed using a deep learning image classification model. Like an instance segmentation model, an image classification model recognizes different areas of the image. However, instead of creating image masks, it generates bounding boxes, which indicate their contents using a rectangle. This allows for the identification of points of interest on the cattle's body, such as the muzzle, forehead, shoulder, knee, and tail base. The software "YOLOv3" can be used as a model, for example, because it offers very high processing speed with a sufficiently good recognition rate. YOLOv3 achieves this speed by reading the image data only once. Sample data should also be generated for training. e) Person Reidentification

[0044] Re-identification is divided into several steps, as described in Fig. 1 is shown.

[0045] In the first step, features are extracted from the image data. The image of the cow is preprocessed by a neural network trained to recognize features (a "feature network"). The result is a feature map that describes the cow.

[0046] In the second step, identification takes place by further processing the obtained trait map using cow-specific identification networks (Id-Net-Cow 0,1,2...n). These cow-specific identification networks have a probability (p 1,2,..n) that the result is the cow for which the network was trained. During ongoing training, these networks are adapted and optimized. If a new cow is added to the herd, a separate network is created for it.

[0047] The third step involves evaluation: if the identification network has the highest probability of exceeding a certain threshold, then the individual cow can be identified. If the threshold is not exceeded, this means that the individual could not be reliably identified, and identification is attempted using object tracking instead. If this also fails to identify the individual, it can be assumed that a new cow is involved after a certain period of time.

[0048] To continuously train the re-identification process, feature maps can be generated from the isolated images provided by object tracking using the feature network. These feature maps can then be used to train the identification networks. A new identification network is created for each new cow. f) Determining coat brightness near certain key points

[0049] As a further "weak" identification feature, brightness values ​​at certain key points, such as the forehead, knees, and base of the tail, can be measured and abbreviated. The idea behind this is that a rough, but resource-efficient, identification can be performed based on stored brightness values. Therefore, only brightness values, and not color values, should be used so that this approach also works with night vision cameras that only provide grayscale images. g) Ear tag recognition

[0050] If an ear tag is detected during keypoint recognition, standard OCR methods are used to try to read it and use it for cattle identification. h) Position detection

[0051] Using previously measured reference points, several positional data points can be determined. Firstly, a rough center point of the cow in space. Secondly, the positions of key points in space should also be determined using a mathematical model or a deep learning model. This positional data can then be used to create a simplified model of the cow's skeleton and to obtain movement behavior data from it. i) Temporal analysis of the position

[0052] The calculated center point of the cattle will be used for a temporal analysis of their position. This means calculating the possible movement range of each individual animal, enabling tracking even without successful identification. This is important to bridge periods of non-identification and is also needed for recognizing unknown individuals. j) Cattle identification

[0053] The data obtained through person re-identification, coat light measurement, and ear tag recognition are used to identify the individual cow. If this fails, it could be because these steps have failed or because a new cow has joined the herd. In the former case, temporal position analysis, combined with a successful identification before or after the initial identification, is used to assign the data to the cow. However, if this fails over a certain period, it can be assumed that a new cow has arrived, and images of the identified keypoints are collected for the automatic training of a new person re-identification evaluation model for this new individual. Saving the collected data

[0054] All data obtained in the previously explained steps can be stored in a database, regardless of its quality. k) Evaluation

[0055] The stored raw data can now be used for further analysis. This analysis is preferably performed periodically and will also be recorded in the database.

[0056] Possible evaluations could include: Heat detection based on increased activity; heat detection via behavior; lameness detection using the simplified skeleton; behavioral change detection via anomaly detection

[0057] Estrus detection based on increased activity: Female cattle exhibit increased activity during estrus. This can be detected relatively easily by observing their central position in space and its rate of change. Heat detection via behavior:

[0058] In the early stages of estrus, female cattle mount other heifers. This can be identified by their simplified skeletal structure. In the later stages of estrus, female cattle no longer avoid being mounted. This can be identified by the simplified skeletal structures of the two heifers involved. Lameness detection using the simplified skeleton:

[0059] Cattle that are lame typically exhibit an upward-curving back. This can also be identified through a simplified skeletal analysis. Behavioral change through anomaly detection:

[0060] Behavioral modification can be implemented using a continuously and individually trained deep learning model. For this purpose, a model is continuously trained for each cow. This model is intended to predict the cow's potential behavior. A significant deviation from reality could indicate a behavioral change. Display of data

[0061] The following options are available for retrieving the data obtained through analysis, either automatically or by the end user: Web API, web interface, app for mobile devices • Alerting Web API:

[0062] The core of the data retrieval process can be the web API. All relevant data from the database can be accessed via this API. This interface could also be made accessible to external systems to supply existing herd management systems with data. Web interface:

[0063] The web interface can present the data provided by the web API in a usable format for the end user. It can be modeled on the interface of established herd management systems. Implementing a simple herd management system would also be conceivable. App for mobile devices:

[0064] There could also be an app for mobile devices, whose content would mirror the web interface. Implementation as a Progressive Web App would be conceivable. Alarm:

[0065] An alert can be triggered based on certain analysis results, such as the presence of a female in heat. This alert can be delivered via app push notification or SMS notification.

Claims

1. Method for tracking animals, in particular cattle, comprising the following steps: - continuous or intermittent digital image acquisition of a plurality of animals to obtain digital image data, - identifying individual animals using a comparison algorithm to compare the image data with previously obtained image data, - tracking the movement of the identified animals, preferably using a Kalman filter to obtain first movement data, - analysing the first movement data to create a movement profile of each animal, - tracking the movement of individual body parts of the animals using body features detected by one or the feature extraction algorithm and / or detecting and tracking the body position of the animals from the image data, to obtain second movement data, - analysing the second motion data in order to determine the sequence of movements and / or the activity of the animals in order to create a behavioural profile of each animal, characterised in that the comparison algorithm is designed to determine a similarity between an identification vector generated from the image data and a previously generated identification vector, wherein the comparison algorithm is designed in particular to determine the cosine similarity, wherein the previously generated identification vector is adapted to the identification vector generated from the image data by means of exponential smoothing or, if the similarity between the identification vector generated from the image data and the previously generated identification vector is too low, a new identification vector is stored.

2. Method according to claim 1, characterised in that the movement profile and the behaviour profile are recorded continuously or at regular intervals over the lifetime of the animals.

3. Method according to claim 1 or 2, characterised in that the first movement data is analysed to detect the current location of the animals.

4. Method according to claim 1, 2 or 3, characterised in that the current location of the animals is compared with a predefined location of a feeding station in order to record the feeding time of each animal.

5. Method according to any one of claims 1 to 4, characterised in that the second movement data are analysed in order to detect a standing and a lying position of the animals and to calculate standing and lying times.

6. Method according to any one of claims 1 to 5, characterised in that the second movement data is analysed to detect the rate of rumination of each animal.

7. Method according to any one of claims 1 to 6, characterised in that the movement profile and the behaviour profile comprise at least three, preferably at least four characteristic values which are selected from the group consisting of feeding times, lying times, movement times, movement distance, standing times, whereabouts, feed removal, feeding duration, rutting behaviour, calving, pathological behaviour and / or the ruminating rate of each animal.

8. Method according to any one of claims 1 to 7, characterised in that the animal welfare of each animal is determined from the movement profile and the behaviour profile, the animal welfare preferably being calculated in the form of a key figure.

9. Method according to any one of claims 1 to 8, characterised in that the identification of an animal by means of the algorithm for feature extraction comprises two steps, wherein in a first step an animal-individual feature map is created from the digital image data and in a second step the animal-individual feature map is assigned to an animal, wherein the assignment comprises the following steps: - Providing a plurality of artificial neural networks, wherein each network is individualised or trained with respect to a particular one of the plurality of animals, - Feeding the feature map to each of the plurality of artificial neural networks to obtain a match value for each, - Comparing the matching values obtained from the artificial neural networks and identifying the animal having the highest matching value for the artificial neural network that is individualised to it.