An electronic device for controlling identification of an animal in an environment and related methods
Patent Information
- Authority / Receiving Office
- CA · CA
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2025-09-25
AI Technical Summary
Existing camera-based identification solutions are degraded by environmental factors, such as animals bumping into the camera sensor, spiders, spiderwebs occluding the field of view, rain, stains, and dirt, leading to inaccurate animal identification in farming environments.
An electronic device with memory and processor circuitry adjusts sensor data to compensate for environmental factors, generating feature vectors and comparing them with reference vectors to improve identification accuracy, using 3D image data from sensors like Time-of-Flight cameras.
The device provides robust and resilient animal identification, enhancing accuracy and reducing maintenance costs while being compact and portable, suitable for controlling machinery like milking and feeding machines.
Abstract
Description
[0001] AN ELECTRONIC DEVICE FOR CONTROLLING IDENTIFICATION OF AN ANIMAL IN AN
[0002] ENVIRONMENT AND RELATED METHODS
[0003] TECHNICAL FIELD
[0004] The present disclosure pertains to the field of image processing, in particular image processing for animal monitoring. The present disclosure relates to an electronic device for controlling identification of an animal in an environment and related methods.
[0005] BACKGROUND
[0006] Identification of an animal plays an important role in animal breeding and animal production systems, allowing producers to keep records on animal information, such as one or more of: a birth date, production records, health history, parentage data, body condition score and any other suitable management information related to an animal. Identification of an animal may be used to control passage of such animal by automatically opening or closing gates based on an individual credential associated with the animal.
[0007] SUMMARY
[0008] Camera-based identification solutions may provide advantages in terms of portability, cost, and deployment ease. However, camera-based identification may be degraded by environmental factors, such as animals bumping into the camera sensor, spiders, spiderwebs occluding a part of a field of view, rain, stains, dirt, etc.
[0009] Accordingly, there is a need for devices and methods, which may mitigate, alleviate, or address the shortcomings existing and may provide for an identification of an animal which is more robust and resilient to the environmental factors mentioned.
[0010] Disclosed is an electronic device comprising memory circuitry, processor circuitry, and an interface. The electronic device is configured to obtain sensor data from a sensor, wherein the sensor data comprises three-dimensional image data representative of at least a part of a gait cycle of a bovine animal in an environment. The electronic device is configured to adjust the sensor data to compensate for environmental factors. The electronic device is optionally configured to generate, based on the adjusted sensor data, a first feature vector associated with the bovine animal. The electronic device is optionally configured to obtain a plurality of reference feature vectors associated with a plurality of referenced bovine animals. The electronic device is optionally configured to compare the first feature vector with the plurality of reference feature vectors. The electronic device is optionally configured to control an identification of the bovine animal based on the comparison. Disclosed is a method performed by an electronic device, such as for controlling identification of an animal, such as a bovine animal. The method comprises obtaining sensor data from a sensor. The sensor data comprises three-dimensional image data representative of at least a part of a gait cycle of a bovine animal in an environment. The method comprises adjusting the sensor data to compensate for environmental factors. The method comprises generating S106, based on the adjusted sensor data, a first feature vector associated with the bovine animal. The method comprises obtaining a plurality of reference feature vectors associated with a plurality of referenced bovine animals. The method comprises comparing the first feature vector with the plurality of reference feature vectors. The method comprises controlling an identification of the bovine animal based on the comparison.
[0011] It is an advantage of the present disclosure that the disclosed electronic device and the disclosed method may improve accuracy of individual animal identifications (such as, to find, based on the sensor data, a match between a detected animal and a referenced animal, or to categorize the detected animal as a non-referenced animal). The disclosed electronic device and the disclosed method may provide a more robust and resilient identification of an animal, such as a bovine animal in conditions involving various environment factors existing in farming areas (such as animals bumping into the camera sensor, spiders, spiderwebs occluding a part of a field of view, rain, stains, dirt, etc). The disclosed robust animal identification may be particularly advantageous for tracking and monitoring each animal of a livestock. Further, the disclosed animal identification may be used to control one or more machines, such as a milking machine, a feeding machine and / or other types of machines. The disclosed electronic device may benefit from an improved battery usage while being compact and portable in some examples. The disclosed electronic device may lead to more efficient maintenance costs or efforts and possibly to a reduction thereof.
[0012] Disclosed is an electronic device, comprising memory circuitry, processor circuitry, and an interface. The electronic device is configured to obtain first sensor data from a first sensor. The first sensor data comprises three-dimensional image data representative of a bovine animal in an environment. The electronic device is configured to generate, based on the first sensor data, a first feature vector associated with the bovine animal. The electronic device is configured to obtain a plurality of reference feature vectors associated with a plurality of referenced bovine animals. The electronic device is configured to compare the first feature vector with the plurality of reference feature vectors. The electronic device is configured to generate, based on the comparing, a confidence value, wherein the confidence value is indicative of an accuracy of a match between the bovine animal and a referenced bovine animal of the plurality of referenced bovine animals. The electronic device is configured to control, based on the confidence value, an identification of the bovine animal.
[0013] Disclosed is a method, performed by an electronic device, such as for controlling identification of an animal, such as a bovine animal. The method comprises obtaining first sensor data from a first sensor. The first sensor data comprises three-dimensional image data representative of a bovine animal in an environment. The method comprises generating, based on the first sensor data, a first feature vector associated with the bovine animal. The method comprises obtaining a plurality of reference feature vectors associated with a plurality of referenced bovine animals. The method comprises comparing the first feature vector with the plurality of reference feature vectors. The method comprises generating, based on the comparing, a confidence value, wherein the confidence value is indicative of an accuracy of a match between the bovine animal and a referenced bovine animal of the plurality of referenced bovine animals. The method comprises controlling, based on the confidence value, an identification of the bovine animal.
[0014] It is an advantage of the present disclosure that the disclosed electronic device and the disclosed method may improve the accuracy of the control of the identification of animals. For example, the disclosed techniques enables the update of reference data used for identification of animals, such as bovine animals. The disclosed electronic device and the disclosed method allows for non-referenced animals to be identified. The disclosed robust animal identification may be particularly advantageous for tracking and monitoring each animal of a livestock, by consolidation with second sensors that are not necessarily of the same type as the first sensor. Further, the disclosed animal identification may be used to control one or more machines, such as a milking machine, a feeding machine and / or other types of machines. The disclosed electronic device may benefit from an improved battery usage while being compact and portable in some examples. In some examples, the disclosed electronic device allows for increasing accuracy of the identification, inter alia, by for example including other data (such as historical sensor data, second sensor data etc.). In some examples, this may be used to monitor an overall performance of the identification and detect and / or predict degradations.
[0015] The disclosed electronic device and the disclosed method may benefit from being carried out in a compact portable device (such as a camera) in some examples while maintaining an improved battery usage as sensor data obtained from one or more sensors (such as, cameras) may be triggered occasionally and processed locally (such as, in a smart camera and / or a computer located near to the one or more sensors). Further, the disclosed electronic device and method may benefit from a reduced power consumption in the identification process, and a reduced bandwidth usage in the identification process as well as an extended product lifetime. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The above and other features and advantages of the present disclosure will become readily apparent to those skilled in the art by the following detailed description of examples thereof with reference to the attached drawings, in which:
[0017] Fig. 1 is a diagram illustrating an example wireless communication system comprising an example network node and an example wireless device according to this disclosure, Figs. 2A-2B show a flow-chart illustrating an example method, performed by an electronic device, according to this disclosure,
[0018] Fig. 3 is a block diagram illustrating an example electronic device according to this disclosure, Fig. 4A-4B show a flow-chart illustrating an example method, performed by an electronic device, according to this disclosure, and
[0019] Fig. 5 is a block diagram illustrating an example electronic device according to this disclosure.
[0020] DETAILED DESCRIPTION
[0021] Various examples and details are described hereinafter, with reference to the figures when relevant. It should be noted that the figures may or may not be drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures. It should also be noted that the figures are only intended to facilitate the description of the examples. They are not intended as an exhaustive description of the disclosure or as a limitation on the scope of the disclosure. In addition, an illustrated example needs not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular example is not necessarily limited to that example and can be practiced in any other examples even if not so illustrated, or if not so explicitly described.
[0022] The figures are schematic and simplified for clarity, and they merely show details which aid understanding the disclosure, while other details have been left out. Throughout, the same reference numerals are used for identical or corresponding parts.
[0023] In a farming environment (such as, in a dairy farm) where a camera is configured to record an animal, debris (such as, straw and / or saw dust) and animals bumping into the camera may distort the performance results.
[0024] The inventors have found that compensating for such environmental factors improves the robustness and resilience of the identification of animals. For example, when identifying individual animals (such as bovine animals) with a camera image, the disclosed technique allows for other cameras mounted in other positions, like other height, other angle and with other lenses etc, to provide sensor data and to extract the feature vectors (such as features represented by feature vectors) for identification by compensating or normalizing to render the identification independent of the environmental factors. For this can be robust against one camera being trained in one position and then slightly tilted to a new position, for example due to wind or an animal bumping into the camera. For example, change in image quality can also depend on degradation due to dirt on the camera lens increasing over time.
[0025] The present disclosure may provide a more accurate and robust identification of an animal by allowing for appropriate adjustment to be performed on the sensor data obtained.
[0026] For example, identification of an animal can be advantageous for allowing passage of a bovine animal (such as, a cow) to one or more of a milking machine, a feeding area, and a resting area depending on its needs and previous behaviour.
[0027] A feature vector disclosed herein (such as a first feature vector, a second feature vector, and / or a third feature vector) may be seen as a vector comprising numeric and / or symbolic characteristics representative of an object, such as of an animal in the environment. A feature vector may comprise one or more attributes of the object which characterize the object alone or in relation to other objects. In other words, a feature vector may be seen as a vector comprising data indicative of a plurality of features. A feature may be one or more of: a colour component, a length component, a volume component, an area component, a height component, a width component, a depth component, a shape component, a size component, a Gray-level intensity value component, and any other suitable features.
[0028] A feature vector may comprise color components of an object, such as a level intensity value for respective Red Green Blue, RGB, components for describing the object. A feature vector may comprise one or more of a height, a width, an intensity (such as intensity in Infrared spectrum for each point or pixel) and a depth associated with a 3-dimensional, 3D, shape of an object. Features and feature vector can be used interchangeably in this disclosure.
[0029] In the present disclosure, the object is an animal, such as a walking animal, such as an animal for farming, such as one or more of: a bovine animal, an equine animal, and a swine animal.
[0030] An animal may be one or more of: a cow, a calf, a horse, a goat, a pig, and a sheep. In other words, an animal may be a bovine animal (such as, cattle), a porcine animal, an equine animal, and / or any other suitable animal breed. The disclosed techniques may be applicable to a swine animal, an equine animal, and / or any other suitable animal breed. The present disclosure may provide one or more techniques (such as, computer vision techniques) for identifying an animal, inter alia, based on a body part of an animal and respective space and time coordinates in a 3D point coordinate space. In other words, the present disclosure may allow a more accurate identification of topological and / or morphological features of an animal (such as, body shape, contour of an animal).
[0031] A gait pattern disclosed herein may be seen as a pattern characterizing a movement of one or more parts of an animal (such as one or more joints, one or more key points, such as the sacrum of the animal) over time.
[0032] The present disclosure may enable a control of machinery adapted for an individual animal. The machinery can be included in a farm management system.
[0033] Fig. 1 is a diagram illustrating an example system 1 comprising an example remote device 400, and an example device 300, 300A according to this disclosure.
[0034] As discussed in detail herein, the present disclosure relates to a system 1. The system 1 may be one or more of: an animal monitoring system, a farming system, an animal milking system, an abattoir system, a wildlife monitoring system (such as for monitoring bison), and an animal feeding system.
[0035] The system 1 described herein may comprise one or more of: device 300, 300A, and / or one or more remote devices 400. The devices 300, 300A may be configured to communicate with the remote device 400 via a wireless link (or radio access link) 10, 10A and / or a wired link 12, 12A.
[0036] The device 300, 300A may refer to a monitoring electronic device that may be installed in a barn and / or a facility for monitoring activity of an animal.
[0037] The remote device 400 may be installed remotely from a barn and / or a facility for remote monitoring of an activity of an animal. The remote device 400 may be remotely located from, in vicinity of or far away from a barn. The remote device 400 may be one or more of: a back-office device (such as, a computer, a laptop, a PC, a tablet, and / or a mobile phone) and a server device (such as, part of a cloud architecture).
[0038] The devices 300, 300A may be useful for directing a bovine animal to a feeding and / or milking and / or resting area. The devices 300, 300A may be part (such as, a subsystem) of a farm management system. The use of devices 300, 300A may enable personalisation of control machinery included in the farm management system.
[0039] The electronic devices disclosed herein (such as in Figs. 3 and 5) may be implemented in a monitoring electronic device such as device 300, 300A, and / or in a remote device, such as remove device 400.
[0040] Fig. 2A-2B shows a flow diagram of an example method 100, performed by an electronic device according to the disclosure, such as for controlling identification of an animal, such as a bovine animal, such as for compensating for environmental factors affecting identification of an animal, such as a bovine animal. The electronic device is the electronic device disclosed herein, such as electronic device 300, 300A of Fig. 1 and 500 of Fig. 3.
[0041] The electronic device performing method 100 may be seen as an animal monitoring device and / or an animal identifying device and / or a device configured to identify a bovine animal.
[0042] The method 100 may be applied to an animal, such as one or more of: walking mammals, livestock animals, and farming animals. The method 100 may be applied to one or more of: a bovine animal (such as, cattle), a swine animal, an equine animal, and any other suitable animal breed. In other words, the method 100 may be applied to one or more of: a cow, a calf, a horse, a goat, a pig, and a sheep.
[0043] The method 100 comprises obtaining S102 (such as generating, receiving, and / or retrieving) sensor data from a sensor. The sensor data can be obtained from one or more sensors. The sensor can be seen as a camera. In one or more example methods, the sensor is a Time-of- Flight camera.
[0044] It may be envisaged to have sensors of other types. For example, the sensor can be of first type or of a second type. For example, a sensor of a first type, such as a first sensor is a ToF camera. In one or more example methods, the three-dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal. For example, the three-dimensional coordinates can be represented by one or more point clouds.
[0045] For example, a sensor of a second type, such as a second sensor is a two-dimensional camera, having the same field of view as a first sensor. The second sensor of the second type can be a Red Green Blue camera and / or a monochrome camera. The sensor data comprises three-dimensional image data representative of at least a part of a gait cycle of a bovine animal in an environment. A gait cycle may include four parts corresponding to a four legs cycle of motion. For example, a part of the gait cycle can be Y% of the gait cycle where Y is a positive real number. For example, the part of the gait cycle can be at least 25%, at least 50%, or at least 75% of the gait cycle. In one or more examples, a part of the gait cycle can be captured by one or more frames, such as one or more frame images provided in the sensor data. In some examples, 50% of gait cycle may be sufficient for an efficient learning / identifyi ng . In some examples, less than 50% of the gait cycle is sufficient and more passages may be needed for the electronic device to robustly learn and identify the shape.
[0046] In some examples, the sensor data comprises three-dimensional image data representative of at least one gait cycle.
[0047] The electronic device may detect at least a part (such as 50%) of gait cycle for controlling identification of the bovine animal. In other words, the electronic device may detect the bovine animal walking in the field of view of the sensor. The electronic device may store one or more images (samples) of the bovine animal walking in the environment for controlling identification of the bovine animal. The electronic device may store the one or more images associated with 25%, 50%, 75%, or an entire walking passage of the bovine animal. The electronic device may comprise a single sensor to obtain the one or more images of the bovine animal (such as, of one or more passages). The electronic may comprise one or more sensors (such as, multiple camera system) to obtain the one or more images of the bovine animal (such as, of a single passage). In other words, the sensor (and / or one or more sensors) may be placed at end parts of the field of view (such as, where the bovine animal is expected to enter and exit the field of view of the sensor) to detect changes. This may act as a trigger for starting and stopping storage (such as, recording) of sensor data. Targeting a monitoring around when an event is triggered may mitigate noise and / or unexpected events from triggering recording.
[0048] In some examples, sensor data (such as image data) from a full gait cycle can be used for training and / or inference. For example, image data or sensor data of the back of a bovine animal changes depending on what part of the gait cycle is captured. For example, when the shape of the back is going to be compared to the reference sensor data (such as from earlier recordings), the matching is much more effective when the sensor data (such as image data) are in the same stage of the gait cycle. It may be appreciated that many images for training and / or inference may lead to a more robust identification that is not dependent on every image being 100% correct.
[0049] In some examples, the sensor data is representative of a capture of a full gait cycle or 1.5 gait cycle based on a field of view that captures the full gait cycle and many recordings to be able to at least capture a full gait cycle, for example to get the best possible material for controlling identification. In some examples, the sensor data can include a gait cycle part indicator for each frame. For example, for each frame, a feature vector is obtained with a gait cycle indicator, and the best match of previously obtained feature vectors for the corresponding gait cycle part indicator can be used to identify the best matched individual.
[0050] In some examples, the sensor data can include a gait cycle part indicator for each frame. For example, for each frame, a point in multidimensional space is obtained with a gait cycle indicator, and the nearest point for the corresponding gait cycle part indicator can identify a match using a K-nearest neighbours technique.
[0051] In some examples, the gait cycle part indicator can indicate the leg moving (such as right front leg, right back leg, left front leg or left back leg). It may be envisaged to capture several passages of the bovine animal to get many parts (such as all parts) of the gait cycle, and several gait cycles, for example, providing different field of views.
[0052] The method 100 comprises adjusting S104 the sensor data to compensate for environmental factors. Environmental factors can be factors related to the farming environment and / or to the intrinsic imperfection of the sensor. Factors related to the farming environment are for example to titling, mounting angle, cows bumping, dirt, dust, insects, background etc. For example, the sensor can have intrinsic imperfections of optical components of a camera. It may be appreciated that each camera has its own imperfections (such as lens not completely spheric) which can be adjusted for. For example, the disclosure allows for first characterising the camera characteristics (such as spherical characteristics of the lens, lens / optical property) and for calculating the imperfections of the camera so that compensation can be deployed. For example, an image sensor component may have some pixel defects, different resolutions, different pixel formats which can be captured automatically and automatically compensated, thanks to a creation of a 3D map during the characterization.
[0053] The method 100 comprises generating S106, based on the adjusted sensor data, a first feature vector associated with the bovine animal. For example, the first feature vector is indicative of one or more spatial features of a part of the bovine animal, such as of a shape of a part of the bovine animal. The first feature vector can be seen as a 3D feature vector, such as a ToF feature vector. S106 may comprise generating a plurality of feature vectors associated with the bovine animal, including the first feature vector, and a second feature vector, and optionally a third feature vector etc.
[0054] The method 100 comprises obtaining (such as receiving and / or retrieving) S108 a plurality of reference feature vectors associated with a plurality of referenced bovine animals. For example, the plurality of reference feature vectors are obtained from a separate device, such as a database and / or a remote repository. For example, the plurality of reference feature vectors are obtained from a memory circuitry of the electronic device. The reference feature vectors are for example generated based on sensor data previously obtained to determine a “fingerprint” of parts of the bovine animals to be used as identifier and stored in a database, a lookup table, and / or a repository. For example, each reference feature vector is uniquely associated with a bovine animal, such as using a bovine identifier. The bovine identifier can be obtained with the corresponding reference feature vector.
[0055] The method 100 comprises comparing S110 the first feature vector with the plurality of reference feature vectors. Comparing S110 the first feature vector (and optionally the second feature vector, and optionally the third feature vector respectively) with the reference feature vectors obtained comprises determining a Euclidean distance between the first feature vector (and optionally the second feature vector, and optionally the third feature vector respectively) and the reference feature vectors obtained. In one or more example methods, comparing S110 the first feature vector with the plurality of reference feature vectors comprises comparing S110A, for a corresponding part of the gait cycle, the first feature vector with the plurality of reference feature vectors. Other measurement of the difference or distance between feature vectors can be used.
[0056] The method 100 comprises controlling S112 an identification of the bovine animal based on the comparison. For example, controlling S112 the identification of the bovine animal comprises identifying the bovine animal as a referenced bovine animal corresponding to at least one of the referenced feature vectors or as a non-reference bovine animal. For example, in S112, an identifier (such as a unique identifier) may be generated and assigned to the bovine animal based on the comparison.
[0057] In one or more example methods, adjusting S104 the sensor data to compensate for environmental factors comprises transforming S104A the sensor data. The transformation S104A can be seen as a normalization. For example, the transformation S104A can be seen as normalizing the extracted feature vectors (such as image features) for more resilience to disturbances and use the extracted feature vectors in other places and on other sensors, such as cameras. This allows some independence to camera characteristics, such as mounted height, angle, field of view, zoom level etc.
[0058] In one or more example methods, transforming S104A the sensor data comprises performing S104AA a rotation based on a normal vector of a floor plane. In one or more examples, transforming the sensor data comprises determining a normal vector of a floor plane. In one or more examples, determining the normal vector of a floor plane comprises determining one or more parameters representative of position and tilt of the floor (such as in relation to the second sensor (such as, the ToF camera). The present disclosure may allow a normalised view of a bovine animal (and / or of a plurality of bovine animals) in the 3D space (such as, a point cloud space) associated with the sensor. In one or more examples, transforming the sensor data comprises rotating the one or more 3D coordinates (such as, point cloud coordinates) based on the normal vector of the floor plane. Stated differently, transforming the sensor data comprises aligning the normal vector of the floor plane with a z-axis in a 3D space associated the sensor data.
[0059] For example, the rotation S104AA can be seen as calibrating the X / Y-plane towards the floor. For example, to be able to compare shapes, even if the sensor moves and to be able to reuse the feature vectors of the bovine animal between different cameras, the Z-direction is to point exactly upwards. This can be achieved by for example calibrating by rotation for each image or as often as needed calibrating by rotation to a floor that is horizontal. The rotation S104AA can be performed periodically, such as according to a time period that may be based on the sampling rate of the sensor, and / or with a minimum periodicity, such as hourly, and / or triggered by detection of a degradation in the identification, and / or by the input from another sensor, such as an accelerometer detecting motion or change in orientation.
[0060] It may be appreciated that when the sensor is moved (due to bovine animals bumping into the sensor), another field of view is obtained and the rotation of S104AA allows to level against the floor, to find the X / Y-plane. It may be appreciated that without the adjustment S104 the sensor data provides incorrect measurements that may lead to incorrect identification.
[0061] The adjustment by rotation of S104AA can be represented by a compensation matrix that may be updated periodically, such as at every frame, such as between every passage.
[0062] In one or more example methods, the rotation S104AA based on the normal vector of the floor plane comprises a rotation of one or more points of a spine area of the bovine animal. For examples, the rotation of the one or more points of the spine area can be seen as spine normalization that straighten the spine in the X / Y-plane before extracting features or feature vectors to reduce differences due to a bent spine.
[0063] In one or more example methods, transforming S104A the sensor data comprises removing S104AB, from the sensor data, background data associated with a background in the environment, and body part data associated with parts of the body of the bovine animal. In one or more examples, removing the background data associated with a background in the environment comprises performing background subtraction on the sensor data for provision of the sensor data (such as, an image) solely comprising the bovine animal. The background in the environment may comprise the surroundings of the bovine animal, such as one or more of: fences, dirt, milk machinery, feeders, and any other suitable element of the environment.
[0064] The electronic device may extract the bovine animal from the sensor data based on one or more of: a primary sensor data and a secondary sensor data. The primary sensor data may be seen as a background reference 3D image. The background reference 3D image is an image of the environment without the bovine animal present. In other words, the background reference 3D image may solely include the background environment. The primary sensor data may be seen as a complete 3D image (such as, the image 20 of Fig. 2). The complete 3D image may comprise the bovine animal and the background environment. In one or more examples, removing the background data associated with a background in the environment comprises comparing the primary sensor data and the secondary sensor data for provision of the sensor data (such as, an image) solely comprising the bovine animal. The background reference image may be updated regularly as the sensor may move slightly over time and / or due to changes occurring in the environment where the bovine animal is in. Put differently, the method may comprise storing the background reference image whenever there is a change in the environment surrounding the bovine animal and / or in a field of view (such as, range) of the sensor.
[0065] Any remaining parts not removed in the background subtraction may be of a small size. The method may comprise removing such remaining elements (such as, element which may not belong the bovine animal) by applying thresholding techniques to the sensor data after a first pre-processing stage (such as, the background subtraction).
[0066] In one or more examples, the body part data associated with parts of the body of the bovine animal is removed upon acquiring a plurality of frames (such as, a plurality of images) of the bovine animal in 3D. In one or more example methods, the parts of the body to be removed include a head, a neck area, a shoulder area, a part of a back area, and / or a tail of the bovine animal. In one or more examples, removing the body part data associated with parts of the body of the bovine animal may comprise aligning a spine of the bovine animal to an x-axis for provision of a normal vector of a floor plane. In one or more examples, the parts of the body are removed after aligning the spine of the bovine animal to an x-axis. For example, the parts of the body may be cropped off by removing one or more data columns associated with the second sensor data whose sum is below a first threshold.
[0067] In one or more examples, by removing body parts, the adjustment allows to use for example the back of the bovine animal, such as the rigid part of the back. For example, even when the full gait cycle is used, some parts are independent of the gait cycle, like the neck, which could be kept high, low, left right etc. during the full gait cycle. It may be seen as beneficial to select and work with parts that are more rigid and stable like the rear part of the back of the bovine animal.
[0068] In one or more example methods, the method 100 comprises updating S103 background data associated with a background in the environment. For example, the background data can be updated in S103, such as periodically, such as for each capture, such as for each frames, optionally continuously. The update of the background data enables correctly discriminating the bovine animal from the background. This may help handling changes in background as well as changes in camera position / orientation.
[0069] In one or more example methods, adjusting S104 the sensor data to compensate for environmental factors comprises determining S104B whether a quality parameter of the sensor data meets a criterion. In other words, the quality parameter may be seen as a parameter indicative of a quality of the sensor data, such as a noise ratio, missing pixel ratio, etc. In one or more example methods, the quality parameter comprises a variance of and / or a standard deviation associated with elements of the sensor data over time. For example, the quality parameter can measure the distance to the floor, and the distance variance to floor, which when it increases shows that the quality drops. For example, the quality parameter can measure the back features (height contour), and the variance in the back features, when it increases shows that the back features are not crisp. For example, the quality parameter can help detect and manage sensor data (such as image) degradation. For example, the quality parameter can be related to measurement of dirt / dust-level, presence of insect-stains, and / or presence of spiderweb(s). For example, when an image is partially occluded (such as stationary or moving spiderweb), the adjustment of the sensor data provides only the frames that contains the region of interest in the non-occluded area. An occlusion is fixed and determined when the sensor data provides the same pixels being discarded. In one or more example methods, adjusting S104 the sensor data to compensate for environmental factors comprises, upon determining that the quality parameter of the sensor data does not meet the criterion, triggering S104C a cleaning and / or maintenance operation of a part of the sensor. For example, the quality parameter dropping below a threshold triggers automatic cleaning of the part of the sensor, such as the lens of a camera. In one or more example methods, triggering S104C the cleaning and / or maintenance operation of a part of the sensor comprises transmitting S104CA a message to an operator. For example, the quality parameter dropping below a threshold (such as non-occluded area is below a threshold) triggers a notification or alert to the personnel of the farming facility. For example, the quality parameter dropping below a threshold triggers automatic increase of the light (such as flashlight) of the sensor.
[0070] In one or more example methods, adjusting S104 the sensor data to compensate for environmental factors comprises upon determining that the quality parameter of the sensor data meets the criterion, forgoing S105 the triggering.
[0071] In one or more example methods, the method 100 comprises obtaining S102 the sensor data comprises controlling S102A an operation parameter of the sensor. For example, the controlling S102A of the operation of the sensor takes place upon determining that the quality parameter does not meet the criterion (such as when no feature vector can be extracted). For example, when handling black bovine animals, the sensor obtains less reflection, which deteriorates the quality of the sensor data, such as images. For example, in S102A, the operation controlled can be to control (such as change) exposure time and / or aperture to make the sensor more sensitive to light. For example, the control of the operation can be performed on cow per cow basis, and after obtaining a 1st frame for which the quality parameter does not meet the criterion, or after each frame.
[0072] In one or more example methods, controlling S112, based on the comparison, the identification of the bovine animal comprises identifying S112A the bovine animal as a referenced bovine animal of the plurality of referenced bovine animals. For example, the comparison shows a Euclidean distance being the closest to a referenced feature vector associated with a reference bovine animal, which leads to the bovine animal detected to be the referenced bovine animal.
[0073] In one or more example methods, controlling S112, based on the comparison, the identification of the bovine animal comprises identifying S112B the bovine animal as a non-referenced bovine animal. For example, the comparison shows a Euclidean distance being too far (based on a threshold) from any of referenced feature vectors, which leads to the bovine animal detected to be a non-referenced bovine animal (such as a new bovine animal, which has never been detected before). The sensor data obtained for the non-referenced bovine animal can be used to provide reference feature vector for the non-referenced bovine animal so that the nonreferenced bovine animal can be referenced. For example, the reference feature vector is stored in the database, lookup table, and / or repository comprising the plurality of referenced feature vectors.
[0074] Fig. 3 shows a block diagram of an example electronic device 500 according to the disclosure. The electronic device 500 comprises memory circuitry 501, processor circuitry 502, and a wireless interface 503. The electronic device 500 may be configured to perform any of the methods disclosed in Fig. 2A-B. In other words, the electronic device 500 may be configured for controlling identification of an animal, such as a bovine animal, in an environment. The electronic device 500 may be seen as an animal monitoring device and / or an animal control device and / or a sensor configured to identify an animal, such as one or more of: walking mammals, livestock animals, and farming animals. An animal may be one or more of: a cow, a calf, a horse, a goat, a pig, and a sheep. In other words, an animal may be a bovine animal (such as, cattle), a swine animal, an equine animal, and any other suitable animal breed.
[0075] The electronic device 500 may be part of a system, such as an animal monitoring system, a farming system, a milking and / or feeding system. In some examples, the electronic device 500 may be implemented as a device 300, 300A of Fig. 1 and / or as a remote device 400 of Fig. 1.
[0076] The interface 503 may be configured for wired and / or wireless communications.
[0077] The electronic device 500 is configured to obtain (such as, via the interface 503 and / or using the memory circuitry 501) sensor data from a sensor. The sensor data comprises three- dimensional image data representative of at least a part of a gait cycle of a bovine animal in an environment. In one or more example electronic devices, the sensor is a Time-of-Flight camera. In one or more example electronic devices, the three-dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal. The sensor may be internal to the electronic device 500 or external to the electronic device 500.
[0078] The electronic device 500 is configured to adjust (such as, via the processor circuitry 502) the sensor data to compensate for environmental factors.
[0079] The electronic device 500 is configured to generate (such as, via the processor circuitry 502), based on the adjusted sensor data, a first feature vector associated with the bovine animal. The electronic device 500 is configured to obtain (such as, via the interface 503 and / or the memory 501) a plurality of reference feature vectors associated with a plurality of referenced bovine animals.
[0080] The electronic device 500 is configured to compare (such as, via the processor circuitry 502) the first feature vector with the plurality of reference feature vectors.
[0081] The electronic device 500 is configured to control (such as, via the processor circuitry 502) an identification of the bovine animal based on the comparison.
[0082] In one or more example electronic devices, the electronic device 500 is configured to adjust (such as, via the processor circuitry 502) the sensor data to compensate for environmental factors by transforming the sensor data. In one or more example electronic devices, transforming the sensor data comprises performing a rotation based on a normal vector of a floor plane. In one or more example electronic devices, the rotation based on the normal vector of the floor plane comprises a rotation of one or more points of a spine area of the bovine animal.
[0083] In one or more example electronic devices, transforming the sensor data comprises removing, from the sensor data, background data associated with a background in the environment, and body part data associated with parts of the body of the bovine animal.
[0084] In one or more example electronic devices, the electronic device 500 is configured to update (such as, via the processor circuitry 502) background data associated with a background in the environment.
[0085] In one or more example electronic devices, the electronic device 500 is configured to adjust (such as, via the processor circuitry 502) the sensor data to compensate for environmental factors by determining whether a quality parameter of the sensor data meets a criterion. In one or more example electronic devices, the quality parameter comprises a variance of and / or a standard deviation associated with elements of the sensor data over time.
[0086] In one or more example electronic devices, the electronic device 500 is configured to adjust the sensor data to compensate for environmental factors by, upon determining that the quality parameter of the sensor data does not meet the criterion, triggering (such as, via the processor circuitry 502) a cleaning and / or maintenance operation of a part of the sensor.
[0087] In one or more example electronic devices, triggering the cleaning and / or maintenance operation of a part of the sensor comprises transmitting (such as, via the interface 503) a message to an operator. In one or more example electronic devices, the electronic device 500 is configured to obtain the sensor data by controlling (such as via the processor circuitry 502) an operation parameter of the sensor.
[0088] In one or more example electronic devices, the electronic device 500 is configured to compare the first feature vector with the plurality of reference feature vectors by comparing (such as via the processor circuitry 502), for a corresponding part of the gait cycle, the first feature vector with the plurality of reference feature vectors.
[0089] In one or more example electronic devices, the electronic device 500 is configured to control, based on the comparison, the identification of the bovine animal by identifying (such as via the processor circuitry 502) the bovine animal as a referenced bovine animal of the plurality of referenced bovine animals.
[0090] In one or more example electronic devices, the electronic device 500 is configured to control, based on the comparison, the identification of the bovine animal by identifying (such as via the processor circuitry 502) the bovine animal as a non-referenced bovine animal.
[0091] Processor circuitry 502 is optionally configured to perform any of the operations disclosed in Figs. 2A-2B (such as any one or more of S102, S102A, S104, S106, S104A, S104AA, S104AB, S104B, S104C, S104CA, S105, S108, S110, S110A, S112, S112A, S112B). The operations of the network node 500 may be embodied in the form of executable logic routines (for example, lines of code, software programs, etc.) that are stored on a non-transitory computer readable medium (for example, memory circuitry 501) and are executed by processor circuitry 502).
[0092] Furthermore, the operations of the network node 500 may be considered a method that the network node 500 is configured to carry out. Also, while the described functions and operations may be implemented in software, such functionality may also be carried out via dedicated hardware or firmware, or some combination of hardware, firmware and / or software.
[0093] Memory circuitry 501 may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random-access memory (RAM), or other suitable device. In a typical arrangement, memory circuitry 501 may include a nonvolatile memory for long term data storage and a volatile memory that functions as system memory for processor circuitry 502. Memory circuitry 501 may exchange data with processor circuitry 502 over a data bus. Control lines and an address bus between memory circuitry 501 and processor circuitry 502 also may be present (not shown in Fig. 3). Memory circuitry 501 is considered a non-transitory computer readable medium. Memory circuitry 501 may be configured to store reference feature vectors, first vector, sensor data, identifiers in a part of the memory.
[0094] Fig. 4A-4B shows a flow diagram of an example method 200, performed by an electronic device according to the disclosure, such as for controlling identification of an animal, such as a bovine animal.
[0095] The electronic device is the electronic device disclosed herein, such as electronic device 300, 300A of Fig. 1 and 600 of Fig. 5.
[0096] The electronic device performing method 200 may be seen as an animal monitoring device and / or an animal identifying device and / or a device configured to control identification of a bovine animal.
[0097] The method 200 may be applied to an animal, such as one or more of: walking mammals, livestock animals, and farming animals. The method 200 may be applied to one or more of: a bovine animal (such as, cattle), a swine animal, an equine animal, and any other suitable animal breed. In other words, the method 200 may be applied to one or more of: a cow, a calf, a horse, a goat, a pig, and a sheep.
[0098] The method 200 comprises obtaining S202 first sensor data from a first sensor. The first sensor data comprises three-dimensional image data representative of a bovine animal in an environment. Sensor data can be obtained from one or more sensors including a first sensor. The first sensor can be seen as a camera. In one or more example methods, the first sensor is a Time-of-Flight camera.
[0099] It may be envisaged to have sensors of other types. For example, the sensor can be of first type or of a second type. For example, a sensor of a first type, such as a first sensor is a ToF camera.
[0100] In one or more example methods, the three-dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal. For example, the three-dimensional coordinates can be represented by one or more point clouds. For example, a sensor of a second type, such as a second sensor is a two-dimensional camera, having the same field of view as a first sensor. The second sensor of the second type can be a Red Green Blue camera and / or a monochrome camera.
[0101] The method 200 comprises generating S204, based on the first sensor data, a first feature vector associated with the bovine animal. For example, the first feature vector is indicative of one or more spatial features of a part of the bovine animal, such as of a shape of a part of the bovine animal. The first feature vector can be seen as a 3D feature vector, such as a ToF feature vector. S204 may comprise generating a plurality of feature vectors associated with the bovine animal, including the first feature vector, and a second feature vector, and optionally a third feature vector etc. associated with the bovine animal.
[0102] The method 200 comprises obtaining (such as receiving and / or retrieving) S206 a plurality of reference feature vectors associated with a plurality of referenced bovine animals. For example, the plurality of reference feature vectors are obtained from a separate device, such as a database and / or a remote repository. For example, the plurality of reference feature vectors are obtained from a memory circuitry of the electronic device. The reference feature vectors are for example generated based on sensor data previously obtained to determine a “fingerprint” of parts of the bovine animals to be used as identifier and stored in a database, a lookup table, and / or a repository. In other words, the reference feature vectors are feature vectors of an already or earlier identified bovine animal for which the feature vector extracted is referenced in the database, the lookup table, and / or the remote repository. For example, each reference feature vector is uniquely associated with a bovine animal, such as using a bovine identifier. The bovine identifier can be obtained with the corresponding reference feature vector.
[0103] The method 200 comprises comparing S208 the first feature vector with the plurality of reference feature vectors. Comparing S208 the first feature vector (and optionally the second feature vector, and optionally the third feature vector respectively) with the reference feature vectors obtained comprises determining a Euclidean distance between the first feature vector (and optionally the second feature vector, and optionally the third feature vector respectively) and the reference feature vectors obtained. In one or more example methods, comparing S208 the first feature vector with the plurality of reference feature vectors comprises comparing, for a corresponding part of the gait cycle, the first feature vector with the plurality of reference feature vectors. Other measurements of the difference between feature vectors can be used, such as cosine distance. The method 200 comprises generating S210, based on the comparing, a confidence value. The confidence value is for example indicative of an accuracy of a match between the bovine animal and a referenced bovine animal of the plurality of referenced bovine animals. The confidence value can be seen as parameter indicative of how confident the assessment is in the match (such as identity match) provided. The confidence value (such as, confidence level, accuracy parameter) may be seen as a value quantifying a match between a detected bovine animal with one or more previously detected bovine animals referenced by their respective reference feature vectors. In other words, the confidence value may indicate the confidence of a detected bovine animal matching with one or more previously detected bovine animals (such as how close the first feature vector of detected bovine animal is to the reference feature vector of a previously detected bovine animal). The confidence value may be percentage (such as, a confidence score of 0-100%).
[0104] In one or more example methods, generating S210 the confidence value based on the comparing comprises determining S210A a distance parameter between the first feature vector and any of the reference feature vectors. The electronic device can calculate and use the confidence value of the identity match in absolute and / or relative terms. An absolute confidence value may provide the distance (such as the absolute distance) between the first feature vector and any (such as all) of the reference feature vectors. A relative confidence value may provide the relative distance between the first feature vector and one or more given reference feature vectors, such as K-nearest neighbors.
[0105] In one or more examples, generating S210 the confidence value based on the comparing comprises generating the confidence value based on the comparing and based on filtering out the bovine animals that have just / immediately previously been detected.
[0106] The electronic device can generate the confidence value of the identity match in one or more dimensions, such as highest points of the back of the bovine animal, such as a distance between hips, and / or local curvatures etc. The electronic device may reduce dimensions because some features are redundant and do not contribute sufficiently to distinguishing between the bovine animals The confidence evaluation can also be impacted by sensor data quality and / or age (such as image quality and / or age of image).
[0107] The confidence value can be used to detect new non-referenced bovine animals based on the feature vectors of the detected bovine animal having a distance to closest neighbor higher than a threshold. The method 200 comprises controlling S212, based on the confidence value, an identification of the bovine animal. In one or more example methods, controlling S212, based on the confidence value, the identification of the bovine animal comprises identifying S212A, based on the confidence value, the bovine animal as the referenced bovine animal. For example, in S212A, an identifier (such as a unique identifier) may be generated and assigned to the bovine animal.
[0108] In one or more example methods, identifying S212A, based on the confidence value, the bovine animal as the referenced bovine animal comprises determining S212AA whether the confidence value meets a criterion (such a reference criterion).
[0109] In one or more example methods, identifying S212A, based on the confidence value, the bovine animal as the referenced bovine animal comprises: upon determining that the confidence value meets the criterion, identifying S212AB the bovine animal as the referenced bovine animal.
[0110] For example, the confidence value meets the criterion when the confidence value being a distance is lower than a threshold. In some examples, the threshold is pre-determined and / or can be configured by an operator.
[0111] In one or more example methods, the method 200 comprises S214 one or more reference feature vectors of the referenced bovine animal using the first feature vector upon determining that the confidence value meets the criterion. In other words, when the confidence value is satisfactory, the first feature vector can form part of the reference feature vectors for an identified bovine animal.
[0112] In one or more example methods, controlling S212, based on the confidence value, the identification of the bovine animal comprises identifying S212B the bovine animal as a nonreferenced bovine animal upon determining that the confidence value does not meet the criterion. For example, a new (not yet referenced) bovine animals can be detected this way, and the electronic device can maintain a temporary identifier (such as an internal number) until sufficient sensor data has been collected to accurately identify the bovine animal. For example, when the feature vectors have been generated for the newly identified bovine animal, the bovine animal can be referenced with corresponding feature vectors as reference feature vectors for that bovine animal, and stored in a database, lookup table, and / or remote repository.
[0113] In one or more example methods, controlling S212, based on the confidence value, the identification of the bovine animal comprises identifying S212B more than one bovine animal as referenced bovine animals. The controlling S212 may further include evaluations using second sensors, which can further enhance the identification of two or more bovine animals. In one or more examples, the electronic device takes reference sensor data (such as reference images) periodically and / or at a specific time (such as event driven). For example, when inference is performed (for example using linear discrimination (with dimensionality reduction) followed by K-nearest neighbor classification) and the result has high confidence, the electronic device uses the extract feature vectors as reference feature vectors for the bovine animal detected. For example, such reference sensor data can be used for training the identification model. For example, older training data can be pushed out automatically as new training data is captured. In one or more examples, the electronic device makes use of the last registered reference images or reference feature vectors as well as stores the reference feature vectors of increased quality. Older reference images or older reference feature vectors may be deleted to provide space for storage and computation, this deletion may be automated based on, for example, remaining storage, file size, or the number of reference images or reference feature vectors stored for an identified animal.
[0114] In one or more example, the reference feature vectors obtained include metadata such as confidence values (that for example can be per feature, can be per image), and / or quality value of the obtained reference feature vectors.
[0115] In one or more example, in the control of the identification S212, a higher weight can be given to the reference feature vectors recently provided.
[0116] In one or more example methods, the method 200 comprises obtaining S216 second sensor data from a second sensor. In one or more examples, the second sensor is of a different type than the first sensor. For example, the second sensor can be as a sensor not correlated with the first sensor. The second sensor can be an ID reader and / or an RGB camera.
[0117] In one or more example methods, controlling S212, based on the confidence value, the identification of the bovine animal comprises controlling S212C the identification of the bovine animal based on the confidence value and the second sensor data.
[0118] In one or more example methods, information from other systems and / or sensors can be used to consolidate or control the identification of the bovine animal based on the confidence level of information. The more uncorrelated the systems are, the more accuracy is gained in the identification when combining the information (such as sensor data) from the other systems with the result of identification based on the first sensor data. Examples of systems and / or additional sensors to use as second source of information and / or second sensor include one or more of: RGB cameras, infra-red, IR camera (such as providing grey scale image produced as a side result to the ToF image), Electronic Identification, EID, system, and Radio Frequency Identification, RFID (that is attached to the cows’ ears or as necklace).
[0119] Combining data from uncorrelated systems (namely, first sensor data and second sensor data) may increase the accuracy of the overall identification of the bovine animal. For example, when EID is acceptable, but biometrics captured by the first feature vector from the first sensor data poorly match reference feature vectors, then the identification is to upgrade to biometrics database, namely the reference feature vectors. For example, when new cows are identified according to the EID system, then the identification is to upgrade to the biometrics database. For example, when the first sensor (such as ToF system) detects a cow, but the EID does not identify a cow, then the electronic device can warn the personnel that EID might be broken, completely lost, or not initialized at all.
[0120] Fig. 5 shows a block diagram of an example electronic device 600 according to the disclosure. The electronic device 600 comprises memory circuitry 601, processor circuitry 602, and an interface 603. The electronic device 600 may be configured to perform any of the methods disclosed in Figs. 4A-B. In other words, the electronic device 600 may be configured for controlling identification of an animal, such as a bovine animal, in an environment. The electronic device 600 may be seen as an animal monitoring device and / or an animal control device and / or a sensor configured to identify an animal, such as one or more of: walking mammals, livestock animals, and farming animals. An animal may be one or more of: a cow, a calf, a horse, a goat, a pig, and a sheep. In other words, an animal may be a bovine animal (such as, cattle), a swine animal, an equine animal, and any other suitable animal breed.
[0121] The electronic device 600 may be part of a system, such as an animal monitoring system, a farming system, a milking and / or feeding system. In some examples, the electronic device 600 may be implemented as a device 300, 300A of Fig. 1 and / or as a remote device 400 of Fig. 1.
[0122] The interface 603 may be configured for wired and / or wireless communications.
[0123] The electronic device 600 is configured to obtain (such as via the interface 603 and / or the memory circuitry 601) first sensor data from a first sensor. For example, the first sensor data comprises three-dimensional image data representative of a bovine animal in an environment. In one or more example electronic devices, the first sensor is a Time-of-Flight camera. In one or more example electronic devices, the three-dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal.
[0124] The electronic device 600 is configured to generate (such as via the processor circuitry 602), based on the first sensor data, a first feature vector associated with the bovine animal.
[0125] The electronic device 600 is configured to obtain (such as via the interface 603 and / or the memory circuitry 601) a plurality of reference feature vectors associated with a plurality of referenced bovine animals.
[0126] The electronic device 600 is configured to compare (such as via the processor circuitry 602) the first feature vector with the plurality of reference feature vectors.
[0127] The electronic device 600 is configured to generate (such as via the processor circuitry 602), based on the comparing, a confidence value, wherein the confidence value is indicative of an accuracy of a match between the bovine animal and a referenced bovine animal of the plurality of referenced bovine animals.
[0128] The electronic device 600 is configured to control (such as via the processor circuitry 602), based on the confidence value, an identification of the bovine animal.
[0129] In one or more example electronic devices, the electronic device 600 is configured to control (such as via the processor circuitry 602), based on the confidence value, the identification of the bovine animal by identifying, based on the confidence value, the bovine animal as the referenced bovine animal.
[0130] In one or more example electronic devices, identifying, based on the confidence value, the bovine animal as the referenced bovine animal comprises determining (such as via the processor circuitry 602) whether the confidence value meet a criterion .
[0131] In one or more example electronic devices, identifying, based on the confidence value, the bovine animal as the referenced bovine animal comprises, upon determining that the confidence value meets the criterion, identifying (such as via the processor circuitry 602)the bovine animal as the referenced bovine animal.
[0132] In one or more example electronic devices, the electronic device 600 is configured to update (such as via the processor circuitry 602) one or more reference feature vectors of the referenced bovine animal using the first feature vector upon that the confidence value meets the criterion.
[0133] In one or more example electronic devices, the electronic device 600 is configured to control (such as via the processor circuitry 602), based on the confidence value, the identification of the bovine animal by identifying the bovine animal as a non-referenced bovine animal upon determining that the confidence value does not meet the criterion.
[0134] In one or more example electronic devices, the generation of the confidence value based on the comparing comprises determining (such as via the processor circuitry 602) a distance parameter between the first feature vector and any of the reference feature vectors.
[0135] In one or more example electronic devices, the electronic device 600 is configured to obtain (such as via the interface 603, and / or memory circuitry 601), second sensor data from a second sensor , wherein the second sensor is of a different type than the first sensor.
[0136] In one or more example electronic devices, the control, the identification of the bovine animal based on the confidence value comprises controlling (such as via the processor circuitry 602) the identification of the bovine animal based on the confidence value and the second sensor data.
[0137] The electronic device 600 is optionally configured to perform any of the operations disclosed in Figs. 4A-B (such as any one or more of S202, S204, S206, S208, S210, S210A, S212, S212A, S212AA, S212AB, S212B, S212C, S214, S216). The operations of the electronic device 600 may be embodied in the form of executable logic routines (for example, lines of code, software programs, etc.) that are stored on a non-transitory computer readable medium (for example, memory circuitry 601) and are executed by processor circuitry 602).
[0138] Furthermore, the operations of the electronic device 600 may be considered a method that the electronic device 600 is configured to carry out. Also, while the described functions and operations may be implemented in software, such functionality may also be carried out via dedicated hardware or firmware, or some combination of hardware, firmware and / or software.
[0139] Memory circuitry 601 may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random-access memory (RAM), or other suitable device. In a typical arrangement, memory circuitry 601 may include a nonvolatile memory for long term data storage and a volatile memory that functions as system memory for processor circuitry 602. Memory circuitry 601 may exchange data with processor circuitry 602 over a data bus. Control lines and an address bus between memory circuitry 601 and processor circuitry 602 also may be present (not shown in Fig. 5). Memory circuitry 601 is considered a non-transitory computer readable medium.
[0140] Memory circuitry 601 may be configured to store feature vectors, reference feature vectors, and / or identifiers in a part of the memory.
[0141] Examples of methods and products (electronic devices) according to the disclosure are set out in the following items:
[0142] Item 1. An electronic device comprising memory circuitry, processor circuitry, and an interface, wherein the electronic device is configured to: obtain sensor data from a sensor, wherein the sensor data comprises three- dimensional image data representative of at least a part of a gait cycle of a bovine animal in an environment; adjust the sensor data to compensate for environmental factors; generate, based on the adjusted sensor data, a first feature vector associated with the bovine animal; obtain a plurality of reference feature vectors associated with a plurality of referenced bovine animals; compare the first feature vector with the plurality of reference feature vectors; and control an identification of the bovine animal based on the comparison.
[0143] Item 2. The electronic device according to item 1 , where the sensor is a Time-of-Flight camera.
[0144] Item 3. The electronic device according to any of the previous items, wherein the three- dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal.
[0145] Item 4. The electronic device according to any of the previous items, wherein the electronic device configured to adjust the first vector to compensate for environmental factors by transforming the sensor data.
[0146] Item 5. The electronic device according to item 4, wherein transforming the sensor data comprises performing a rotation based on a normal vector of a floor plane. Item 6. The electronic device according to item 5, wherein the rotation based on the normal vector of the floor plane comprises a rotation of one or more points of a spine area of the bovine animal.
[0147] Item 7. The electronic device according to any of items 4-6, wherein transforming the sensor data comprises removing, from the sensor data, background data associated with a background in the environment, and body part data associated with parts of the body of the bovine animal.
[0148] Item 8. The electronic device according to any of the previous items, wherein the electronic device is configured to update background data associated with a background in the environment.
[0149] Item 9. The electronic device according to any of the previous items, wherein the electronic device configured to adjust the first vector to compensate for environmental factors by: determining whether a quality parameter of the sensor data meets a criterion, and upon determining that the quality parameter of the sensor data does not meet the criterion, triggering a cleaning and / or maintenance operation of a part of the sensor.
[0150] Item 10. The electronic device according to item 8, wherein triggering the cleaning and / or maintenance operation of a part of the sensor comprises transmitting a message to an operator.
[0151] Item 11. The electronic device according to any of items 8-9, wherein the quality parameter comprises a variance of and / or a standard deviation associated with elements of the sensor data over time.
[0152] Item 12. The electronic device according to any of the previous items, wherein the electronic device is configured to obtain the sensor data by controlling an operation parameter of the sensor.
[0153] Item 13. The electronic device according to any of the previous items, wherein the electronic device is configured to compare the first feature vector with the plurality of reference feature vectors by comparing, for a corresponding part of the gait cycle, the first feature vector with the plurality of reference feature vectors.
[0154] Item 14. The electronic device according to any of the previous items, wherein the electronic device is configured to control, based on the comparison, the identification of the bovine animal by identifying the bovine animal as a referenced bovine animal of the plurality of referenced bovine animals.
[0155] Item 15. The electronic device according to any of the previous items, wherein the electronic device is configured to control, based on the comparison, the identification of the bovine animal by identifying the bovine animal as a non-referenced bovine animal.
[0156] Item 16. A method, performed by an electronic device, the method comprising: obtaining (S102) sensor data from a sensor, wherein the sensor data comprises three-dimensional image data representative of at least a part of a gait cycle of a bovine animal in an environment; adjusting (S104) the sensor data to compensate for environmental factors; generating (S106), based on the adjusted sensor data, a first feature vector associated with the bovine animal; obtaining (S108) a plurality of reference feature vectors associated with a plurality of referenced bovine animals; comparing (S110) the first feature vector with the plurality of reference feature vectors; and controlling (S112) an identification of the bovine animal based on the comparison.
[0157] Item 17. The method according to item 16, where the sensor is a Time-of-Flight camera.
[0158] Item 18. The method according to any of items 16-17, wherein the three-dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal.
[0159] Item 19. The method according to any of items 16-18, wherein adjusting (S104) the sensor data to compensate for environmental factors comprises transforming (S104A) the sensor data. Item 20. The method according to item 19, wherein transforming (S104A) the sensor data comprises performing (S104AA) a rotation based on a normal vector of a floor plane.
[0160] Item 21. The method according to item 20, wherein the rotation based on the normal vector of the floor plane comprises a rotation of one or more points of a spine area of the bovine animal.
[0161] Item 22. The method according to any of items 19-21, wherein transforming (S104A) the sensor data comprises removing (S104AB), from the sensor data, background data associated with a background in the environment, and body part data associated with parts of the body of the bovine animal.
[0162] Item 23. The method according to any of items 16-22, wherein the method comprises updating (S103) background data associated with a background in the environment.
[0163] Item 24. The method according to any of items 16-23, wherein adjusting (S104) the sensor data to compensate for environmental factors comprises: determining (S104B) whether a quality parameter of the sensor data meets a criterion, and upon determining that the quality parameter of the sensor data does not meet the criterion, triggering (S104C) a cleaning and / or maintenance operation of a part of the sensor.
[0164] Item 25. The method according to item 24, wherein triggering (S104C) the cleaning and / or maintenance operation of a part of the sensor comprises transmitting (S104CA) a message to an operator.
[0165] Item 26. The method according to any of items 24-25, wherein the quality parameter comprises a variance of and / or a standard deviation associated with elements of the sensor data over time. Item 27. The method according to any of items 16-26, wherein obtaining (S102) the sensor data comprises controlling (S102A) an operation parameter of the sensor.
[0166] Item 28. The method according to any of items 16-27, wherein comparing (S110) the first feature vector with the plurality of reference feature vectors comprises comparing (S110A), for a corresponding part of the gait cycle, the first feature vector with the plurality of reference feature vectors.
[0167] Item 29. The method according to any of items 16-28, wherein controlling (S112), based on the comparison, the identification of the bovine animal comprises identifying (S112A) the bovine animal as a referenced bovine animal of the plurality of referenced bovine animals.
[0168] Item 30. The method according to any of items 16-29, wherein controlling (S112), based on the comparison, the identification of the bovine animal comprises identifying (S112B) the bovine animal as a non-referenced bovine animal.
[0169] Examples of methods and products (electronic devices) according to the disclosure are set out in the following clauses:
[0170] Clause 1. An electronic device comprising memory circuitry, processor circuitry, and an interface, wherein the electronic device is configured to: obtain first sensor data from a first sensor, wherein the first sensor data comprises three-dimensional image data representative of a bovine animal in an environment; generate, based on the first sensor data, a first feature vector associated with the bovine animal; obtain a plurality of reference feature vectors associated with a plurality of referenced bovine animals; compare the first feature vector with the plurality of reference feature vectors; generate, based on the comparing, a confidence value, wherein the confidence value is indicative of an accuracy of a match between the bovine animal and a referenced bovine animal of the plurality of referenced bovine animals; and control, based on the confidence value, an identification of the bovine animal.
[0171] Clause 2. The electronic device according to clause 1 , where the first sensor is a Time-of- Flight camera. Clause 3. The electronic device according to any of the previous clauses, wherein the three- dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal.
[0172] Clause 4. The electronic device according to any of the previous clauses, wherein the electronic device is configured to control, based on the confidence value, the identification of the bovine animal by identifying, based on the confidence value, the bovine animal as the referenced bovine animal.
[0173] Clause 5. The electronic device according to clause 4, wherein identifying, based on the confidence value, the bovine animal as the referenced bovine animal comprises: determining whether the confidence value meet a criterion; and upon determining that the confidence value meets the criterion, identifying the bovine animal as the referenced bovine animal.
[0174] Clause 6. The electronic device according to clause 5, wherein the electronic device is configured to update one or more reference feature vectors of the referenced bovine animal using the first feature vector upon that the confidence value meets the criterion.
[0175] Clause 7. The electronic device according to any of clauses 5-6, wherein the electronic device is configured to control, based on the confidence value, the identification of the bovine animal by identifying the bovine animal as a non-referenced bovine animal upon determining that the confidence value does not meet the criterion.
[0176] Clause 8. The electronic device according to any of the previous clauses, wherein the generation of the confidence value based on the comparing comprises: determining a distance parameter between the first feature vector and any of the reference feature vectors.
[0177] Clause 9. The electronic device according to any of the previous clauses, wherein the electronic device is configured to obtain, second sensor data from a second sensor, wherein the second sensor is of a different type than the first sensor. Clause 10. The electronic device according to clause 9, wherein the control, the identification of the bovine animal based on the confidence value comprises controlling the identification of the bovine animal based on the confidence value and the second sensor data.
[0178] Clause 11. A method, performed by an electronic device, the method comprising: obtaining (S202) first sensor data from a first sensor, wherein the first sensor data comprises three-dimensional image data representative of a bovine animal in an environment; generating (S204), based on the first sensor data, a first feature vector associated with the bovine animal; obtaining (S206) a plurality of reference feature vectors associated with a plurality of referenced bovine animals; comparing (S208) the first feature vector with the plurality of reference feature vectors; generating (S210), based on the comparing, a confidence value, wherein the confidence value is indicative of an accuracy of a match between the bovine animal and a referenced bovine animal of the plurality of referenced bovine animals; and controlling (S212), based on the confidence value, an identification of the bovine animal.
[0179] Clause 12. The method according to clause 11 , where the first sensor is a Time-of-Flight camera.
[0180] Clause 13. The method according to any of clauses 11-12, wherein the three-dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal.
[0181] Clause 14. The method according to any of clauses 11-13, wherein controlling (S212), based on the confidence value, the identification of the bovine animal comprises identifying (S212A), based on the confidence value, the bovine animal as the referenced bovine animal.
[0182] Clause 15. The method according to clause 14, wherein identifying (S212A), based on the confidence value, the bovine animal as the referenced bovine animal comprises:
[0183] - determining (S212AA) whether the confidence value meet a criterion; and - upon determining that the confidence value meets the criterion, identifying (S212AB) the bovine animal as the referenced bovine animal.
[0184] Clause 16. The method according to clause 15, wherein the method comprises updating (S214) one or more reference feature vectors of the referenced bovine animal using the first feature vector upon determining that the confidence value meets the criterion.
[0185] Clause 17. The method according to any of clauses 15-16, wherein controlling (S212), based on the confidence value, the identification of the bovine animal comprises identifying (S212B) the bovine animal as a non-referenced bovine animal upon determining that the confidence value does not meet the criterion.
[0186] Clause 18. The method according to any of clauses 11-17, wherein generating (S210) the confidence value based on the comparing comprises: determining (S210A) a distance parameter between the first feature vector and any of the reference feature vectors.
[0187] Clause 19. The method according to any of clauses 11-18, wherein the method comprises obtaining (S216) second sensor data from a second sensor, wherein the second sensor is of a different type than the first sensor.
[0188] Clause 20. The method according to clause 19, wherein controlling (S212), based on the confidence value, the identification of the bovine animal comprises controlling (S212C) the identification of the bovine animal based on the confidence value and the second sensor data.
[0189] The use of the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. does not imply any particular order, but are included to identify individual elements. Moreover, the use of the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. does not denote any order or importance, but rather the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. are used to distinguish one element from another. Note that the words “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. are used here and elsewhere for labelling purposes only and are not intended to denote any specific spatial or temporal ordering. Furthermore, the labelling of a first element does not imply the presence of a second element and vice versa. It may be appreciated that Figures comprise some circuitries or operations which are illustrated with a solid line and some circuitries, components, features, or operations which are illustrated with a dashed line. Circuitries or operations which are comprised in a solid line are circuitries, components, features, or operations which are comprised in the broadest example. Circuitries, components, features, or operations which are comprised in a dashed line are examples which may be comprised in, or a part of, or are further circuitries, components, features, or operations which may be taken in addition to circuitries, components, features, or operations of the solid line examples. It should be appreciated that these operations need not be performed in order presented. Furthermore, it should be appreciated that not all of the operations need to be performed. The example operations may be performed in any order and in any combination. It should be appreciated that these operations need not be performed in order presented. Circuitries, components, features, or operations which are comprised in a dashed line may be considered optional.
[0190] Other operations that are not described herein can be incorporated in the example operations. For example, one or more additional operations can be performed before, after, simultaneously, or between any of the described operations.
[0191] Certain features discussed above as separate implementations can also be implemented in combination as a single implementation. Conversely, features described as a single implementation can also be implemented in multiple implementations separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations, one or more features from a claimed combination can, in some cases, be excised from the combination, and the combination may be claimed as any sub-combination or variation of any sub-combination.
[0192] It is to be noted that the word "comprising" does not necessarily exclude the presence of other elements or steps than those listed.
[0193] It is to be noted that the words "a" or "an" preceding an element do not exclude the presence of a plurality of such elements.
[0194] It is to be noted that the term "indicative of" may be seen as “associated with”, “related to”, “descriptive of’, “characterizing”, and / or “defining”. The terms “indicative of”, “associated with”, “related to”, “descriptive of”, “characterizing”, and “defining” can be used interchangeably. The term “indicative of” can be seen as indicating a relation. For example, weight data indicative of weight may comprise one or more weight parameters.
[0195] It is to be noted that the word "based on" may be seen as “as a function of” and / or “derived from”. The terms “based on” and “as a function of” can be used interchangeably. For example, a parameter determined “based on” a data set can be seen as a parameter determined “as a function of’ the data set. In other words, the parameter may be an output of one or more functions with the data set as an input.
[0196] A function may be characterizing a relation between an input and an output, such as mathematical relation, a database relation, a hardware relation, logical relation, and / or other suitable relations.
[0197] It should further be noted that any reference signs do not limit the scope of the claims, that the examples may be implemented at least in part by means of both hardware and software, and that several "means", "units" or "devices" may be represented by the same item of hardware.
[0198] Language of degree used herein, such as the terms “approximately,” “about,” “generally,” and “substantially” as used herein represent a value, amount, or characteristic close to the stated value, amount, or characteristic that still performs a desired function or achieves a desired result. For example, the terms “approximately”, “about”, “generally,” and “substantially” may refer to an amount that is within less than or equal to 10% of, within less than or equal to 5% of, within less than or equal to 1% of, within less than or equal to 0.1% of, and within less than or equal to 0.01% of the stated amount.
[0199] The various example methods, devices, nodes, and systems described herein are described in the general context of method steps or processes, which may be implemented in one aspect by a computer program product, embodied in a computer-readable medium, including computerexecutable instructions, such as program code, executed by computers in networked environments. A computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Generally, program circuitries may include routines, programs, objects, components, data structures, etc. that perform specified tasks or implement specific abstract data types. Computer-executable instructions, associated data structures, and program circuitries represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes.
[0200] Although features have been shown and described, it will be understood that they are not intended to limit the claimed disclosure, and it will be made obvious to those skilled in the art that various changes and modifications may be made without departing from the scope of the claimed disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense. The claimed disclosure is intended to cover all alternatives, modifications, and equivalents.
Claims
CLAIMS1. An electronic device comprising memory circuitry, processor circuitry, and an interface, wherein the electronic device is configured to: obtain first sensor data from a first sensor, wherein the first sensor data comprises three-dimensional image data representative of a bovine animal in an environment; generate, based on the first sensor data, a first feature vector associated with the bovine animal; obtain a plurality of reference feature vectors associated with a plurality of referenced bovine animals; compare the first feature vector with the plurality of reference feature vectors; generate, based on the comparing, a confidence value, wherein the confidence value is indicative of an accuracy of a match between the bovine animal and a referenced bovine animal of the plurality of referenced bovine animals; and control, based on the confidence value, an identification of the bovine animal.
2. The electronic device according to claim 1 , where the first sensor is a Time-of-Flight camera.
3. The electronic device according to any of the previous claims, wherein the three- dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal.
4. The electronic device according to any of the previous claims, wherein the electronic device is configured to control, based on the confidence value, the identification of the bovine animal by identifying, based on the confidence value, the bovine animal as the referenced bovine animal.
5. The electronic device according to claim 4, wherein identifying, based on the confidence value, the bovine animal as the referenced bovine animal comprises: determining whether the confidence value meet a criterion; and upon determining that the confidence value meets the criterion, identifying the bovine animal as the referenced bovine animal.
6. The electronic device according to claim 5, wherein the electronic device is configured to update one or more reference feature vectors of the referenced bovine animal using the first feature vector upon that the confidence value meets the criterion.
7. The electronic device according to any of claims 5-6, wherein the electronic device is configured to control, based on the confidence value, the identification of the bovine animal by identifying the bovine animal as a non-referenced bovine animal upon determining that the confidence value does not meet the criterion.
8. The electronic device according to any of the previous claims, wherein the generation of the confidence value based on the comparing comprises: determining a distance parameter between the first feature vector and any of the reference feature vectors.
9. The electronic device according to any of the previous claims, wherein the electronic device is configured to obtain, second sensor data from a second sensor, wherein the second sensor is of a different type than the first sensor.
10. The electronic device according to claim 9, wherein the control, the identification of the bovine animal based on the confidence value comprises controlling the identification of the bovine animal based on the confidence value and the second sensor data.
11. A method, performed by an electronic device, the method comprising: obtaining (S202) first sensor data from a first sensor, wherein the first sensor data comprises three-dimensional image data representative of a bovine animal in an environment; generating (S204), based on the first sensor data, a first feature vector associated with the bovine animal; obtaining (S206) a plurality of reference feature vectors associated with a plurality of referenced bovine animals; comparing (S208) the first feature vector with the plurality of reference feature vectors; generating (S210), based on the comparing, a confidence value, wherein the confidence value is indicative of an accuracy of a match between the bovineanimal and a referenced bovine animal of the plurality of referenced bovine animals; and controlling (S212), based on the confidence value, an identification of the bovine animal.
12. The method according to claim 11 , where the first sensor is a Time-of-Flight camera.
13. The method according to any of claims 11-12, wherein the three-dimensional image data comprises one or more three-dimensional coordinates associated with each element of the environment and of the bovine animal.
14. The method according to any of claims 11-13, wherein controlling (S212), based on the confidence value, the identification of the bovine animal comprises identifying (S212A), based on the confidence value, the bovine animal as the referenced bovine animal.
15. The method according to claim 14, wherein identifying (S212A), based on the confidence value, the bovine animal as the referenced bovine animal comprises: determining (S212AA) whether the confidence value meet a criterion; and upon determining that the confidence value meets the criterion, identifying (S212AB) the bovine animal as the referenced bovine animal.
16. The method according to claim 15, wherein the method comprises updating (S214) one or more reference feature vectors of the referenced bovine animal using the first feature vector upon determining that the confidence value meets the criterion.
17. The method according to any of claims 15-16, wherein controlling (S212), based on the confidence value, the identification of the bovine animal comprises identifying (S212B) the bovine animal as a non-referenced bovine animal upon determining that the confidence value does not meet the criterion.
18. The method according to any of claims 11-17, wherein generating (S210) the confidence value based on the comparing comprises:determining (S210A) a distance parameter between the first feature vector and any of the reference feature vectors.
19. The method according to any of claims 11-18, wherein the method comprises obtaining (S216) second sensor data from a second sensor, wherein the second sensor is of a different type than the first sensor.
20. The method according to claim 19, wherein controlling (S212), based on the confidence value, the identification of the bovine animal comprises controlling (S212C) the identification of the bovine animal based on the confidence value and the second sensor data.