Monitoring system for individual growth monitoring of livestock animals

Abstract

A system and method for monitoring the growth of livestock includes a shed environment with bedding on the floor, a 3D camera having a downward field of view in which animals may be present during visits so as to repeatedly acquire 3D images of the field of view, and an image processor for extracting a 3D point cloud representation. The image processor detects the presence of an animal in the point cloud representation, and if present, selects a portion of the animal's point cloud in the representation; determines position-related first height information, i.e., the height of the animal's portion of the point cloud above the floor; and determines the position of the animal's forelegs and / or hind legs, and if not present, determines position-related second height information, i.e., the height of the bedding above the floor. The image processor determines the animal's height during a visit based on the first height information, the second height information, and the position of the forelegs and / or hind legs. This provides reliable height measurements in the animal's natural environment without requiring physical labor.

Description

[0001] The present invention relates in a first aspect to a growth monitoring system for monitoring the growth of an individual animal, and in a second aspect to a method for monitoring the growth of an individual animal.

[0002] Monitoring growth helps assess an animal's health, which in turn aids in animal management. For example, determining when a certain action should be taken may depend on the animal's size. For calves, examples are weaning and first insemination, while for meat animals, slaughter time typically depends on weight or size.

[0003] Monitoring growth requires the ability to consistently determine animal size. For most livestock, and certainly young animals, growth almost always occurs in three spatial dimensions (plus body weight). Therefore, this invention focuses on using height as the chosen dimension for monitoring growth.

[0004] While weighing systems are commonly used to monitor growth, the accuracy and usefulness of weight measurements are not ideal. For example, the weight of the contents of the gastrointestinal tract and bladder can vary significantly. Methods for measuring the size of livestock are less common. This almost always requires physical labor to provide useful measurements.

[0005] Document NL2011952A1 discloses a 3D camera system for monitoring animal growth, which is arranged to determine multiple volume-related parameters, such as various height parameters, and preferably also to determine weight. The system is used in locations with fencing to restrict animal movement and is arranged to determine the animal's height relative to the floor or another horizontal surface, and can be used, for example, daily or weekly.

[0006] In practice, these measurements have been found to be generally unreliable, and measuring animals in confined environments can lead to agitation. Furthermore, agitation should be avoided as much as possible, especially for young animals; therefore, frequently transferring young animals to confined environments is not a good practice.

[0007] The purpose of this invention is to overcome the above-mentioned shortcomings.

[0008] The specific objective of this invention is to provide a system for monitoring the growth of livestock that provides more accurate measurements, causes less stress to the animals, and does not require a large amount of physical labor.

[0009] This invention achieves these objectives at least in part through a growth monitoring system, particularly a growth monitoring system for monitoring the growth of individual livestock animals (such as cows) with forelegs and hind legs, the growth monitoring system comprising: a pen environment in which one or more such livestock animals can move and the floor of the pen environment is covered with bedding; a 3D camera having a field of view including portions of the pen environment in which the individual livestock animal may be present during a visit, and the 3D camera being arranged to repeatedly acquire 3D images of the field of view; and an image processor arranged to process the acquired images to extract a 3D point cloud representation of the field of view from the processed acquired images, wherein... The image processor is configured to detect the presence of one of the livestock animals in the 3D point cloud representation, and if so, select the animal portion of the point cloud representation and determine position-related first height information, i.e., the height of the animal portion of the point cloud representation above the floor, and determine the position of the forelegs and / or hind legs of the individual livestock animal based on the animal portion of the point cloud; if not, determine position-related second height information, i.e., the height of the bedding above the floor, wherein the image processor is configured to determine the height of the individual livestock animal at the time of the visit based on the first height information, the second height information, and the position of the forelegs and / or the hind legs. Here, "position-related height information" indicates that the measured height (or distance, a coarse, unprocessed measurement) varies with position. Furthermore, this position can be considered as a position within the image itself, or preferably a position in the real world, especially a position on the animal. In each case, this is clear in the context.

[0010] The inventors have recognized that a system capable of more frequent measurements in a more natural environment for the animal can be better, less agitated, and more accurate. The current system almost consistently uses bedding on the floor. This bedding is typically quite soft and can be compacted when an animal stands on it. This causes the height and other properties of the bedding to change over time, leading to inaccurate measurements of animal height. To address this problem, the system according to the invention corrects for this by taking an image of an empty field of view (i.e., without an animal), measuring the height of the bedding, and then considering the bedding height measured before and / or after the animal's visit to determine the animal's height. Therefore, even if the height of the bedding changes over time due to the addition of new material, the removal of old material, or simply because the bedding is gradually compacted by the animal's weight, this is always corrected. Thus, measurements in a more natural environment reduce animal agitation. Furthermore, the ability to measure generally reduces the likelihood of mismeasurements, or at least reduces the influence of outliers. And in each case, a one-time fixed calibration measurement is not required, as the image processor can always use the image of the empty field of view to find the correction value.

[0011] In this invention, the downward field of view is not only the spatial angle of the 3D camera, but also the spatial portion of the camera's image. Furthermore, "downward" is intended to include a field of view that widens in the downward direction. The center line of the field of view typically defines this direction and is therefore also in the downward direction, and is preferably vertical or at least within 20° of the vertical direction. Additionally, "visiting" is intended to refer to a situation where livestock are within the 3D camera's field of view, allowing the livestock to be detected by the image processor.

[0012] A 3D camera can be any type of camera capable of providing 3D information, such as a stereoscopic camera, or preferably a time-of-flight camera. Stereoscopic cameras can use ambient light reflected from objects in their field of view, or their own emitted radiation, to create patterns of artificial textures for stereo matching. Time-of-flight cameras also emit radiation (primarily infrared or visible radiation) and use direct measurements of round-trip time or, most commonly, phase-modulated radiation and phase-shift measurements, to determine the distance to the camera. However, other 3D technologies are not excluded.

[0013] Furthermore, a 3D point cloud representation is a collection of depth information per pixel. Additionally, the 3D camera measures the distance to the nearest reflecting object in the direction corresponding to that pixel for each pixel. Based on all these pixel-distance pairs, the image processor can generate a surface consisting of one or more objects reflecting radiation toward the 3D camera. However, it should be noted that the image processor does not necessarily generate such a surface, as calculations can be performed using pixel-distance pairs. In cases where distance cannot be determined for a pixel, such as due to the absence of an object in that direction, or the distance being too far or the reflection too little, the image processor can use an "invalid" value or a default value to indicate that the pixel "no object found." In practice, objects will always exist, either as padding or as a floor surface after all padding has been removed from a particular location, but they may still be undeterminable or unmeasurable due to distance or insufficient reflection. Furthermore, although not strictly necessary, the image processor can present the collected information to the user, for example, as a representation of one or more point clouds or surfaces found by the camera on a screen or display. These one or more surfaces together (including the default value for pixels where distance information is unavailable) are referred to as a (3D) surface representation.

[0014] Therefore, the point cloud representation provides distance information for each pixel (i.e., for each direction relative to the center of the 3D camera's field of view). This represents the field of view in more or less spherical coordinates, which the image processor can easily convert to Cartesian coordinates. Furthermore, this provides height information for the point cloud representation, and thus also positional information about the height. If the floor level has already been determined using the 3D camera itself or any other measurement method in a previous reference measurement, this height represents the height above the floor.

[0015] The image processor is also configured to detect the presence of one of the livestock animals in the point cloud representation, i.e., in the field of view or in a portion of the enclosure environment as seen by a 3D camera. This can be achieved using any known technique. A simple example is examining dynamic differences between consecutive images; if a sufficiently large and preferably consecutive portion of the image changes, or if the measured distance or height changes above a predefined threshold, the image processor can infer the presence of an animal. Of course, other techniques, such as more sophisticated image recognition, are also possible.

[0016] Furthermore, the image processor is configured to determine the position of the animal's forelegs and / or hind legs. For example, this very generally involves determining the front of the animal. Almost all livestock have an elongated body shape when viewed from above, with the forelegs on one side, clearly the front, and the hind legs on the opposite side. Moreover, it is recommended to consistently measure the animal's height at approximately the same position. Therefore, the image processor determines the position of the forelegs and / or hind legs. This can be done based on the characteristics of the animal's body shape. Similarly, in most cases, the forelegs will be located on the side where the head is situated, and the head can be directly identified, for example, by its inherent shape, or by whether it is higher or lower than the rest of the rear. However, it should be noted that the position of the forelegs and / or hind legs can be determined in different ways and with greater precision. This will be further explained below.

[0017] After the image processor determines the position of the forelegs and / or hind legs, it determines the height of the animal based on any known metrics. Such metrics may be, for example, the highest point on the animal's point cloud representation, or the highest point on the animal's spine (e.g., determined along the centerline of the animal's point cloud representation and excluding the head / neck), or the height of feature points (e.g., specific bones such as the hip bone, or the tail root, etc.).

[0018] The image processor then corrects the height of corresponding points in the point cloud representation by subtracting the height of the bedding point cloud representation to obtain the true height of the livestock. The image processor can obtain the bedding point cloud representation by viewing a 3D image in which livestock are not, and in particular, not detected, or even partially detected. This can be obtained, for example, by viewing an image in which virtually all (relevant) pixels are at a height below a predetermined threshold, i.e., farther than a predetermined distance threshold. Here, the position-related height information of the bedding can be obtained as simply as subtracting the average from the position-related height information of the livestock point cloud representation. In other words, the measured bedding height can be averaged across the field of view to obtain a value. This value eliminates the inherent coarseness of the bedding (such as straw protrusion) and has already provided a height-corrected estimate (albeit still quite coarse) to obtain the correct, true height of the animal. A more accurate method will be further explained below.

[0019] Therefore, it is clear that the image processor processes at least two 3D images to obtain the animal's true height; that is, at least one image has a point cloud representation of the animal, and at least one image has a point cloud representation of the bedding at the same location. In both cases, but especially in the case of the bedding point cloud representation, having more than one image allows for a better representation because not all pixels allow for the determination of correct distance information. It should be noted that the bedding is typically relatively far from the 3D camera, thus often providing only a blurry image, and is generally relatively difficult to image and match correctly to determine the distance of each pixel. Therefore, the resulting single image typically contains a relatively large number of pixels with invalid values. When multiple images are combined, the information can be combined, for example, by averaging, resulting in fewer ignored pixels and thus a more complete and accurate point cloud representation.

[0020] Preferably, the system is arranged such that the time to acquire subsequent images(s) is not much later or earlier than the time to acquire the previous image(s). This can be achieved, for example, by having the system acquire a 3D image of the field of view without the animal when the image processor detects that the detected animal is no longer present in the 3D image. Furthermore, the system is arranged to repeatedly acquire and process the 3D camera until at least one 3D image without the detected livestock is acquired. Similarly, the system can be made to repeatedly (e.g., continuously) acquire and process 3D images, and use the last acquired 3D image without the detected livestock as the image for determining bedding height information. This will be further explained below.

[0021] Specific embodiments are described in the dependent claims and in the following portions of the specification.

[0022] In the most general embodiment of the invention described above, there is no specific limitation on the location for determining the height of the livestock. This depends on the selected height metric, such as hip height. To support more specific height determination, in embodiments, the image processor is arranged to determine the location-related second height information by subdividing the field of view into a predetermined number of subfields, determining the height information of each subfield, and determining the height of the livestock based on the second height information at the positions of the forelegs and / or hind legs respectively in that subfield. By dividing the 3D image of the field of view in this way, more accurate corrections can be applied to determine the height of the livestock. For example, the image processor subdivides the field of view into multiple segments in a row, with the number ranging from 2 to at most a number of pixels. In practice, a number between 5 and 50 (e.g., approximately 10) is sufficient in many cases.

[0023] The image processor then determines the bedding height values ​​for these segments, or alternatively, at least the bedding height values ​​for one or more segments where forelegs and / or hind legs are located. Depending on how the image processor is positioned to determine the true height of the livestock, one or more height values ​​may be used to refine the determined initial height information in order to arrive at the true height of the livestock.

[0024] To determine the position of the forelegs and / or hind legs, the image processor can use any known technique, such as object recognition of limbs, or animal-related information using these positions relative to the front and / or rear of the animal. For example, in a 3D image where livestock occupies 10 segments, the hind legs are likely located in the segment at the rear of the animal, while the forelegs are usually located around the third segment from the front of the animal. The image processor can also use other techniques. For example, the image processor can be configured to determine position based on prominent bones (e.g., based on the hip or shoulder). Furthermore, if the number of segments varies, the position of the legs will change accordingly.

[0025] By limiting the portion of the image containing bedding height information in this way, height correction can be made more accurate. It should be noted that the image processor can also be arranged to subdivide the field of view into multiple subfields along its length, such as between two and ten. After all, the bedding on the left side of livestock does not need to be as high as on the right. This subdivision allows for the accounting for this difference.

[0026] If the image processor is configured to subdivide the field of view into a subfield grid or matrix, forming a first-row × second-row rectangular subfield, it provides even more accurate corrections. For example, this allows determining the height of the padding for each of an animal's four legs.

[0027] In a particular embodiment, the image processor is arranged to subtract the second height information at the hind leg and / or foreleg positions from the first height information at the hind leg and / or foreleg positions, respectively, to obtain the corrected hind leg height and / or foreleg height. It should be noted that "foreleg height" and "hind leg height" refer to "animal height at the forelegs" and "animal height at the hind legs," respectively. Thus, the animal height can be determined, for example, as the height at the hind legs, or the height at the forelegs, or the average of these two height values. This is an example of how to define the useful height of an animal so that it can be reliably determined and easily monitored.

[0028] In an embodiment, the image processor is arranged to determine an interpolated height, preferably a linear interpolated height, of second information between the positions of the forelegs and hind legs; and to subtract this interpolated height from the animal's partial point cloud to form corrected animal height information; and to determine the height of the livestock based on this corrected animal height information. Here, the understanding is utilized that in the case of grid segments or matrix segments, different bedding heights may result in different correction values ​​for the forelegs and hind legs, or even all four legs. In such cases, in principle, all point cloud representations of the animal would require different corrections because the animal may be tilted relative to the horizontal plane. In a first approximation, this correction is a linear interpolation between the height corrections determined for the forelegs and / or for the hind legs. For example, if the left foreleg is on 12 cm of bedding, the right foreleg on 14 cm of bedding, the left hind leg on 10 cm of bedding, and the right hind leg on 12 cm of bedding, then the geometric center of the animal's legs is effectively on (12 + 10 + 14 + 12) / 4 = 12 cm of bedding. For each part of the animal's point cloud representation, a corresponding correction for the determined height can be calculated. It is important to note that this interpolation is based on bedding measurements at the leg location, not at any other location. For example, in this embodiment, the animal's point cloud representation should not be corrected by subtracting the bedding height at the corresponding location or by using some average bedding height. After all, the bedding height between the legs is irrelevant, as it does not affect the animal's height or posture.

[0029] In an embodiment, a 3D camera is arranged to preferably repeatedly acquire 3D images each time the livestock visits, and the image processor is arranged to determine the second height information based on a 3D image acquired immediately after the visit and / or based on a 3D image acquired between the visit and the livestock, preferably any of the livestock's most recent visits. Repeatedly capturing 3D images allows the image processor to determine the first and second height information, and thus also the true height of the livestock. Furthermore, it is preferable that the image processor and 3D camera are arranged accordingly. This allows for smooth measurement and discards outliers, such as those due to measurement errors or other reasons. In some cases, to obtain the first height information, it is sufficient to arrange the image processor to capture at least one 3D image of the livestock at each visit and at least one 3D image of the empty field of view (preferably immediately after the visit, i.e., without any intermediate visits by another animal). This ensures that the bedding remains as close as possible to the view of the animal in the 3D image when it leaves the field of view. Alternatively or additionally, the system is arranged to repeatedly acquire images of the bedding, and to select images immediately before the visit, since there are no intermediate visits by another animal. Similarly, this ensures that the bedding remains as close as possible to the position it was when the animal entered the field of vision. Even more advantageously, the system is arranged so that the second height information is based on the average of a 3D image taken immediately before the visit and a 3D image taken immediately after the visit. This will better account for any effects that the livestock themselves, especially due to their own weight, might have on the bedding.

[0030] The manner in which the image processor determines the position of an animal's forelegs and / or hind legs, or at least the front or rear, is not particularly limited. For example, the image processor may be arranged to determine the position of a specific body part, such as the hip bone protrusion, to determine the position of the hind legs. In an embodiment, the image processor is arranged to determine the body contour of the livestock based on a point cloud of the animal parts, and to determine the position of the hind legs and / or forelegs based on the body contour. The image processor may define the body contour, for example, as a "curve" or a narrow portion of the point cloud (where height values ​​in the first height information decrease with at least a predetermined steepness), and may also define the body contour as a point cloud or surface defined by the curve, i.e., as a region. The image processor is then arranged to determine the positions(s) using object recognition and other imaging techniques known per se, such as determining where the narrowest part (neck) is, or where there is a specific shape, such as the rear end of the animal. Additional information stored in the image processor may be useful, such as historical or documentary data. A significant advantage of using the body contour is its ease of determination and the fact that it can be evaluated in 2D, requiring less computational power. Furthermore, the body contour is a useful attribute for further evaluating the measurements. Therefore, if the image processor is configured to determine the body outline, it can be used for further purposes, but it can certainly also be configured to use other information from point cloud representations, especially point clouds of animal parts.

[0031] For example, in one embodiment, the image processor is configured to evaluate the animal posture of the livestock based on the partial point cloud and discard the partial point cloud if it does not meet a posture usefulness metric. It has been found that livestock do not always adopt postures that allow for reliable determination of their true height using the partial point cloud or initial height information. For example, to obtain reliable results, it is desirable to always use similar postures, such as, in particular, the drinking posture. In this position, the animal's head is lower than the rest of its body. It should be noted that if the animal is standing upright (i.e., not drinking), the head height can vary significantly, but this is highly relevant if the animal's height is defined as the height of its highest point. After all, the head is the highest point, and its variability is high. Such a definition would lead to unreliable height measurements. Therefore, it is not only desirable to have a useful definition of the livestock's height, but also to be useful if it is possible to determine whether the livestock's posture conforms to that definition and to discard the partial point cloud if it does not (i.e., the measurement result).

[0032] Therefore, in an exemplary embodiment, the posture usefulness metric includes the fact that the highest portion of the animal partial point cloud is located in the posterior part of the animal, such as the hindquarters. It has been found that if the animal is drinking, for example, at a drinking station—a crucial feature in raising calves or other young livestock—the highest portion of the animal is most commonly located in the hindquarters, particularly the hindquarters. In any case, if the highest point, or preferably the highest region, is located in the hindquarters, the true height can generally be determined more reliably by the image processor. The height at, for example, the location of the hip bone protrusion can then be obtained, or it can be used as the 95th percentile or some other predetermined percentile value of the height in the pixel direction of the corrected height information (i.e., the second height information corrected by the first height information). Similarly, the “location of the highest point” can be considered, for example, as the location of the smallest segment or even pixel with the highest absolute height (which is, of course, the corrected value), or a slightly average location can be determined, such as the centroid of a predetermined percentage of all segments / pixels constituting the animal partial point cloud with the highest absolute height, i.e., the geometric mean location. In each case, this "highest point" should be located in the hindquarters of the animal so that the livestock is in that particular desired posture. However, it should be noted that different postures are still possible, and therefore the criteria for the usefulness of a posture will also differ.

[0033] Specifically, the posture usefulness metric may include: the symmetry of the animal partial point cloud about its long axis is at least as high as a symmetry threshold. The symmetry of the animal partial point cloud of livestock indicates how upright the animal is standing. It is conceivable that if the animal leans to one side, or bends to the left or right, the symmetry of the animal partial point cloud will decrease. Symmetry can be defined according to any desired metric, as long as a higher value is obtained for an animal partial point cloud that is more symmetrical about its long axis (= longitudinal axis, or the spine of a healthy animal). Alternatively, asymmetry (DoA) can be used, which can be similarly defined, or simply defined as DoA = 1 - DoS. For example, if a line is drawn between the beginning and end of the spine, such as from the tail to the neck, and twice the absolute average (the vertical width on the left minus the vertical width on the right) is divided by the total vertical width, the final number will be between zero (for a perfectly upright and symmetrical animal) and one (for a perfectly symmetrical animal). It has been found that perfectly symmetrical animals, or more precisely, those with a higher degree of symmetry, yield more reliable height measurements.

[0034] To date, multiple animals have not been identified, and the embodiments are primarily applicable to individually kept animals, in which case the system does not need to identify livestock. However, livestock are typically kept in groups, and some form of ID is used for useful identification and monitoring in such cases. To allow the system to identify individual animals, in the embodiments, the system further includes an animal identification device arranged to determine the identity of an individual livestock animal upon arrival. Here, the animal identification device may include a common tag reader that reads the animal's ID tag (such as an ID tag worn around the neck). The animal identification device may also be a software module that identifies animals optically, such as based on patterns on the skin, as long as the ID can be uniquely determined.

[0035] In another aspect of the invention, the present invention relates to a method for monitoring the growth of an individual livestock animal, such as a cow, with forelegs and hind legs in a barn environment, wherein one or more such livestock animals can move around in the barn environment and the floor of the barn environment is covered with bedding, the method comprising: repeatedly acquiring 3D images of the field of view using a 3D camera having a field of view, the field of view including portions of the barn environment in which the individual livestock animal may be present during a visit; processing the acquired images using an image processor, wherein the processing includes extracting a 3D point cloud representation of the field of view; and detecting the growth of the individual livestock animal in the barn environment. If one of the livestock animals is present in the point cloud representation, then the animal portion of the point cloud representation is selected, and the first position-related height information, i.e., the height of the animal portion of the point cloud above the floor, is determined. The position of the forelegs and / or hind legs of the individual livestock animal is determined based on the animal portion of the point cloud. If not, the second position-related height information, i.e., the height of the bedding material above the floor, is determined. The height of the individual livestock animal at the time of the visit is determined based on the first height information, the second height information, and the position of the forelegs and / or the hind legs.

[0036] This relates more or less to the method counterpart of the system claims of this invention. Therefore, the advantages and functions largely depend on the advantages and functions of the system claims. The specific features and advantages of the system claims also apply to the method claims, and will not be repeated here for the sake of brevity.

[0037] Therefore, the present invention provides a system and method for repeatedly determining the height information of animals, such as calves. Furthermore, the method and system are simple, require no physical labor, and allow determination in the most natural possible environment for livestock. Specifically, the method and system are carried out in a littered barn, rather than in a dedicated, flat but barren and hard-walled enclosure. Moreover, repeatedly determining height allows for closer monitoring of animal development, health, etc., compared to occasionally (e.g., every two weeks) manually measuring animal height. Therefore, the "monitoring" of the present invention is the repeated measurement and determination of height information. What is done with the obtained information is not actually part of the present invention. Here, it is sufficient that the present invention provides more possibilities for useful monitoring of animal growth.

[0038] The invention will now be further explained with reference to several non-limiting embodiments and accompanying drawings, in which:

[0039] - Figure 1 A schematic top view of an embodiment of the growth monitoring system 1 according to the present invention is shown;

[0040] - Figure 2 It shows Figure 1 A schematic side view of calf drinking station 5;

[0041] - Figure 3A A schematic 3D image of compartment 6 is shown;

[0042] - Figure 3B It shows Figure 3A The average height of each segment;

[0043] - Figure 4A A schematic 3D image of calf 3 in compartment 6 is shown;

[0044] - Figure 4B It shows Figure 4A A graph of the (first) height information h1 of the 3D calf image;

[0045] - Figure 5 The diagram schematically shows the point cloud of the animal part of a calf; and

[0046] - Figure 6 It shows Figure 5 The graph showing the symmetry of the calves.

[0047] Figure 1 A schematic top view of an embodiment of the growth monitoring system 1 according to the present invention is shown. System 1 includes a calf shed environment 2a for accommodating calves 3 and a cow shed environment 2b for accommodating cows 4.

[0048] In the calf barn environment 2a, a calf drinking station 5 is provided, which has a compartment 6, drinking utensils 7, a 3D camera 8, and an animal ID tag reader 9 for reading ID tags 10 on calves or cows. In the cow barn environment 2b, a concentrate feeding station 20, a drinking trough 30, and a door device 40 are provided, and in each case, at least one 3D camera 8 and at least one animal ID tag reader 9 are also provided. The feeding station 20 further includes a feeding trough 21, and the door device 40 includes an openable door 41.

[0049] In the calf pen environment 2a, multiple calves 3 are separated from their mothers to provide them with optimized feeding and care. Additionally, calf drinking stations 5 are provided, where each calf 3 can receive an amount of milk adjusted according to its developmental stage via drinking utensils 7 (such as artificial nipples or troughs). Furthermore, other feeding stations may be provided, such as stations where calves can receive a slowly increasing amount of solid feed / roughage during their development. This is not further shown in the figure.

[0050] When calf 3 drinks in calf drinking station 5, she enters compartment 6 and is identified by her ID tag 10, which is read by animal ID tag reader 9. The control unit (not shown separately here) determines the animal ID based on the read ID tag information and determines how much milk to supply to the calf in drinking vessel 7.

[0051] When calf 3 is located at drinking vessel 7 and identified, 3D camera 8 acquires one or more 3D images of the compartment 6 containing calf 3. 3D camera 8 also acquires one or more 3D images of an empty compartment 6, for example, after calf 3 has left, or the last image of an empty compartment before calf 3 entered. The control unit can determine, for example, whether a sufficient number of pixels in the field of view of 3D camera 8 are above a height threshold to determine when a calf is not detected in the acquired 3D images. 3D camera 8 can be, for example, a time-of-flight camera, a stereoscopic camera, or any other type of camera capable of acquiring 3D images of its environment. Further details regarding the processing of the 3D images and the setup of the system according to the invention will be discussed later. Figures 2 to 6 Further discussion.

[0052] Figure 1The other stations shown operate in largely the same manner. In the cow shed environment 2b, multiple cows 4 can move around and obtain concentrate, for example, at the feeding trough 21 in the concentrate feeding station 20. When a cow 4 enters station 20, she is identified by animal ID tag reader 8 via her ID tag 10. One or more 3D images of the empty feeding station 20 during and after feeding are obtained by 3D camera 9. Similarly, due to the larger size of the drinking trough 30, multiple 3D images of the drinking trough 30, where multiple 3D cameras 9 and multiple animal ID tag readers 8 are present, can be obtained. Likewise, a cow 4 can appear at a door device 40 with an openable door 41. Again, she will be identified by tag reader 8 via her tag 10. If permitted, she will be able to enter, for example, a paddock or pasture when the control unit opens the door 41. While the cow waits at door device 40, 3D camera 8 will obtain one or more 3D images. When cow 4 passes through door device 40, 3D camera 8 will capture one or more additional 3D images.

[0053] It should be noted that it is not necessary to have both a calf pen environment 2a and a cow pen environment 2b simultaneously, as the invention is also applicable to one or more such environments. Furthermore, the relevant animals are not necessarily calves and cows, but can also be calves only, cows only, heifers only, any combination thereof, any other livestock animal of any corresponding age group (such as sheep or horses), and / or any combination thereof.

[0054] Furthermore, it should be noted that multiple stations equipped with 3D cameras 8 and animal ID tag readers 9 are not required. According to the invention, a single station where the animal, or each animal appears, is sufficient for monitoring. Additionally, it should be noted that one or more milking stations can be set up in the cow shed environment 2b, where cows can be milked. However, lactating cows will be at least 2 years old, and growth monitoring is less relevant, but it is still possible according to the invention.

[0055] Now will be used Figures 2 to 6 This will explain how the invention works. Figure 2 It shows Figure 1 A schematic side view of a calf drinking station 5. In all figures, similar parts are identified by the same reference numerals, with one or more apostrophes (') if necessary. Station 5 has a compartment 6 with a first upright 11a and a second upright 11b, and a horizontal beam 11c supporting an animal ID tag reader 9. Upright 12 supports a 3D camera 8 (with a field of view 13 limited by dashed lines) and has a built-in control unit 17 with image processing capabilities. Drinking utensils 7 can be filled by a filling system 14. A bedding layer 15 is provided on the floor 16.

[0056] When calf 3 enters compartment 6 of drinking station 5, she is identified by animal ID tag reader 9, as described above. Additionally, 3D camera 8 acquires one or more images of calf 3 within compartment 6, which is within the 3D camera's field of view 13. The calf drinks from her milk ration supplied by filling system 14 at drinking vessel 7, based on her identity and the amount she has previously consumed. After calf 3 leaves compartment 6, the 3D camera again acquires one or more images of compartment 6.

[0057] The 3D camera can also acquire one or more images of compartment 6 before calf 3 enters it. This allows for obtaining an average image of empty compartment 3. For this purpose, the 3D camera 8 can repeatedly acquire images, using only one or more images taken immediately before calf 3 enters compartment 6. To prevent excessive memory usage, the oldest image can be overwritten if no calf enters compartment 6 within a predetermined time. Furthermore, whether a calf has entered compartment 6 can be determined by the control unit 17 from the images using standard image processing techniques such as differential image processing and object recognition.

[0058] Control unit 17 is now able to determine the height of calf 3 based on the acquired image. This will refer to... Figure 3A , Figure 3B and Figure 4A , Figure 4B To clarify.

[0059] Figure 3A A schematic 3D image of compartment 6 is shown. This image is a 2D collection of pixels, where each pixel has, for example, distance information. This depth information can be indicated in the image through false color, intensity, etc. This is in Figure 3A The distance between the 3D camera and the bedding material in compartment 6, and the depth information derived from that distance, indicate the thickness of the bedding layer 15 (e.g., straw), and thus the basic height of the calf when standing. This basic height could be the height measured after the arrival of calf 3, the height measured before the arrival of calf 3, or the average of the two measurements.

[0060] The image is subdivided into multiple segments, here 10 segments S1, ... S2 10 However, multiple other segments are also possible, where a larger number of segments provides the opportunity to provide more finely divided height information, even down to the pixel level of the camera used. Interpolation based on a finite number of segments can also be used. It should be noted that obtaining relevant (secondary) height information from the padding material 15 (e.g., straw) is not always straightforward. After all, individual straws may protrude upwards, and so on. Therefore, according to an embodiment, for each segment, the control unit 17 obtains the height above the floor 16 (e.g., average or median height), or some other statistically meaningful height information. Figure 3BThe figure depicts height information, for example, determined by averaging the heights of pixels within a segment. Figure 3A A graph showing the average value of the second height information h2 for each determined segment. The average height of the segments determined in this way is regarded as the basic height, i.e., the height of the bedding, and is used to determine the actual height of the calf 3.

[0061] Figure 4A A schematic 3D image of a calf 3 in compartment 6 is shown. The calf 3 has a head 22, forelegs 23a, hind legs 23b, and hip bone 24. The estimated location of the spine is indicated by a "+" sign. The relative positions of the calf and its surroundings are also indicated. Figure 3A The same 10 sections.

[0062] In the 3D image (represented as a 3D point cloud), the control unit 17 can identify the calf 3 using object recognition technology, for example, after human training and AI or deep learning techniques. First, the difference between the image of the compartment with the calf and a reference 3D image of the empty compartment can be simply examined. If a sufficiently continuous area has sufficient height above the floor 16, or more precisely above the bedding 15, and has a shape resembling a calf (e.g., a discernible head portion and an aspect ratio within a certain range), the control unit can determine that the calf 3 is present in the 3D image. The calf is then represented by the calf portion of the point cloud as pixels identified as belonging to the calf. Other pixels in the field of view image can be discarded, and preferably have already been discarded.

[0063] For a 3D calf image, determine the (first) height information h1 and place it into... Figure 4B In the curve diagram. This can be (semi-)continuous height, or the average height of each segment. The height of a calf at a certain position can be determined, for example, as follows, but other methods are not excluded. First, based on considerations of symmetry, the position of the spine is determined as the center line of the outer circumference of the calf's body 3, that is, the center line of the total circumference excluding the head portion 22. This is achieved through... Figure 4A The "+" sign in the text is arbitrary. Then, vertical slices are made across the surface perpendicular to the spine, and the highest point in that slice is determined to determine the height of the segment (still above the floor, not the inherent height). Alternatively, shape approximations, such as local parabolas, can be used, and the highest point can then be mathematically determined. This is done over the entire spine, or at least over all segments where the calf image exists. It should be noted that, for example, in... Figure 4A In the example shown, the calf is not present in the last three segments, so no relevant height information for these segments can be given.

[0064] Then the determined height information is compared with the following: Figure 3A , Figure 3BThe basic height determined by the aforementioned compartment 6 is compared. Then, in the basic embodiment, it can be determined by... Figure 4A , Figure 4B The calf's height is determined by subtracting the basic height information from the established height information for the calf and determining the maximum value. In this way, the constantly changing height of the bedding 15 no longer affects the accurate height measurement of the calf. It should be noted that any height of the head portion 22 is not included, as the head can be raised or lowered, which does not indicate or affect the calf's true height.

[0065] In more complex systems, the control unit determines the position of the hind leg 23b in a single step. This can be achieved in several ways. For example, the position of the calf's rear end can be determined as a location where the height value drops sharply or falls below a threshold. The control unit then determines the position of the hind leg at a certain distance in the spinal direction, anterior to the rear end position. Alternatively, the control unit determines the position of the hind leg based on the rear end position and a calf-related first displacement value. This displacement value is stored in the control unit and looked up according to the established calf identity. Yet another alternative, the control unit determines the position of the hip bone 24 as the two highest points near the calf's rear end, and then determines the position of the hind leg as said position or the position corrected by a predetermined value or a calf-related second displacement value, which is also stored in the control unit and looked up according to the established calf identity.

[0066] After determining the position of hind leg 23b, the control unit identifies the segment in which the hind leg is located and then determines the basic height of that segment. The control unit then determines the calf's height as the actual height of the segment containing hind leg 23b, which is the determined height at the hind leg position minus the basic height of the same segment. It should be noted that this dimension is not necessarily the actual height of the calf's highest point, as the height at the shoulder (foreleg) or other parts of the spine may be higher. However, it is a highly repeatable height and is very useful for monitoring calf development.

[0067] Clearly, the calf's height can also be determined as the height at the shoulder, i.e., the height at the position of the forelegs 23a. This height is determined in a similar manner to the height at the hind legs 23b. First, for example, the position of those forelegs 23a is determined based on the position of the calf's neck. The neck position is determined as the narrowest point in the calf's top view. The rear end position, or the position of any other fixed part of the calf (such as the hip bone 24 mentioned earlier), can also be used. In any case, the control unit applies a predetermined value or a calf-related third displacement value to the determined position (neck, rear end, hip bone, etc.) to determine the position of the forelegs. The foreleg position is located in segments S1, ... S 10In one step, the basic height of the section is determined based on the height measured in the compartment. Then, the actual height of the calf is determined as the height of the calf in the compartment at its position in that section, minus the basic height of the bedding material in that section.

[0068] In even more complex embodiments, the height of both the hind legs and the forelegs is determined. Here, the calf's height can be determined as the average of the two height values. Alternatively, the calf's true height can be determined as the height at either the hind legs or the forelegs, but each should be corrected for the height at the other legs (i.e., the corresponding forelegs or hind legs). This can correct for the tilt posture of the spine (which may affect the measurement of the true height).

[0069] Alternatively or alternatively, this height may be determined as the highest point on the entire calf. Nevertheless, one or more basic heights used for correction are one or more basic heights of the foreleg 23a section and / or the hind leg 23b section, since the bedding does not affect the height of the calf portion between the legs.

[0070] As an alternative to the above, the image, such as the image of a compartment, can be segmented in the vertical direction, particularly in the length direction. However, the height measured in this way may have slightly greater inaccuracies because the height of the bedding (which is used to correct the measured calf height to obtain the calf's true height) may vary more in the length direction than in the vertical lateral direction.

[0071] In a further development, the image of the compartment is segmented in two dimensions. This allows for even more precise height measurements. The segmentation can be relatively coarse, such as dividing into two sections, or finer, such as more sections (e.g., between 3 and 10), or even the finest, i.e., down to the pixel level of the 3D camera used. In all cases, it is equally preferred to measure the basic height information of the bedding after the calf's visit and / or in the absence of a calf, or more generally, to measure the height of the compartment or space in which the calf will stand during the measurement. Furthermore, it is preferable to determine the position of each leg of the calf. The basic height information of the section in which the corresponding leg is located is then determined. The control unit then corrects the measured calf height by subtracting the corresponding basic height information of the corresponding leg and inserting height corrections for relevant parts of the calf (e.g., the portion between the two legs). The resulting calf height information is used to determine the true height. As before, this may involve determining the height of the hind legs, shoulders, the highest part of the calf, etc., as needed.

[0072] Figure 5The point cloud of the animal part of a calf, having a head portion 22' and a body portion 25, is schematically shown. The spine is indicated by a plus sign, and a straight line 26 is drawn from the starting point (indicated as "0") at the posterior end of the spine to the ending point (indicated as "L") near the head portion 22'. The width of the animal part point cloud relative to the left side of the drawn line 2 is indicated by "a", while the width on the right side is indicated by "b". Figure 6 schematically shown Figure 5 The asymmetry of the animal, DoA, varies with its position on the length of line 26 between “0” and “L”.

[0073] Figure 5 This illustrates one aspect of the invention, in which an image processed by an image processor into a point cloud of the animal portion is not always used. For example, if the head portion 22' is higher than the body 25 or its highest point, it is assumed that the animal is not drinking or not keeping its head down for any other reason. In cases where the pose usefulness metric includes that the head should be kept down, the image will be discarded. Different metrics can certainly be used, such as making the head portion 22' at least 10% higher than the highest point of the body 25. Alternatively or additionally, the pose usefulness metric may include asymmetry (DoA). As explained in the introduction, DoA can be defined in many different ways, but a useful definition is found to be the difference in width between the left and right sides relative to a straight line along the body 25. To express this numerically, the formula DoA = 2 * |a - b| / (a + b) is used herein. Figure 6 As shown in the curve graph. Therefore, for a perfectly symmetrical body 25, the straight line 26 coincides with the spine (the "+" sign in the figure), and at all points along this line, the width "a" on the left is equal to the width "b" on the right. Thus, the DoA is zero at every point, and therefore the average DoA is also zero, which is the perfect score. The more asymmetrical the animal's body pose (animal partial point cloud), the higher the score. Thus, the pose usefulness index can include, for example, that the average DoA should be below a predefined threshold, such as 0.05 or 0.1. If the DoA is high, the image is discarded. It should be noted that extreme cases of asymmetrical animal partial point clouds may occur, such as if the animal is licking its back.

[0074] Using a pose usefulness index allows the image processor to select only images that are reasonably similar to each other. Therefore, the true height of the animal calculated using such selected images becomes more reliable. Furthermore, because the system of this invention can repeatedly acquire 3D images of the animal and select them as needed, complex methods for correcting the animal's pose on any image can be eliminated from the need to calculate the true height.

[0075] Generally, the system according to the invention can reliably determine the height of an animal in its normal environment (i.e., a cattle pen with bedding). This operation can be repeated without further human intervention. Further actions and general management can be taken using the repeatedly determined height information. However, such further management actions are not, in principle, the subject of this invention. It should be noted, however, that the ability to monitor the animal's height, etc., almost continuously makes it possible to manage the animal in almost real-time, thereby preventing any adverse situations as much as possible.

Claims

1. A growth monitoring system for monitoring the growth of an individual livestock animal with forelegs and hind legs, comprising: - A pen environment in which one or more livestock animals are able to move around and the floor of the pen environment is covered with bedding; - A 3D camera with a downward field of view that includes the portion of the shed environment in which an individual livestock animal may appear during a visit, and the 3D camera is arranged to repeatedly acquire 3D images of the field of view; - An image processor configured to process the acquired image to extract a 3D point cloud representation of the field of view from the processed acquired image. - Wherein, the image processor is configured to detect the presence of one of the livestock animals in the 3D point cloud representation, and - If so, select the animal portion of the point cloud represented by the point cloud, and determine the first height information of positional correlation, i.e., the height of the animal portion of the point cloud represented by the point cloud above the floor, and determine the position of the forelegs and / or hind legs of the individual livestock based on the animal portion of the point cloud. - If not, determine the second height information related to the location, i.e., the height of the padding material above the floor. The image processor is configured to determine the height of the individual livestock during the visit based on the first height information, the second height information, and the position of the forelegs and / or the hind legs.

2. The system as claimed in claim 1, wherein, The livestock includes cows.

3. The system as described in claim 1, wherein, The image processor is configured to determine the position-related second height information by subdividing the field of view into a predetermined number of subfields, determining the height information of each subfield, and determining the height of the livestock based on the second height information at the positions of the forelegs and / or the hind legs respectively in the subfields.

4. The system as claimed in claim 1, wherein, The image processor is configured to subtract the second height information at the positions of the hind legs and / or the forelegs from the first height information at the positions of the hind legs and / or the forelegs, respectively, to obtain the corrected hind leg height and / or foreleg height.

5. The system according to any one of claims 1 to 4, wherein, The image processor is configured to determine the interpolated height between the positions of the forelegs and the hind legs of the second height information, subtract the interpolated height from the animal partial point cloud to form corrected animal height information, and determine the height of the livestock based on the corrected animal height information.

6. The system of claim 5, wherein, The interpolation height includes the linear interpolation height.

7. The system according to any one of claims 1 to 4, wherein, The 3D camera is arranged to repeatedly acquire 3D images, and the image processor is arranged to determine the second height information based on the 3D image acquired immediately after the visit and / or based on the 3D image acquired between the visit and the livestock's most recent visit.

8. The system of claim 7, wherein, The 3D camera is configured to repeatedly acquire 3D images each time the livestock visit.

9. The system of claim 7, wherein the image processor is configured to determine the second height information based on a 3D image obtained immediately after the visit and / or based on a 3D image obtained between the visit and the most recent visit of any of the livestock.

10. The system according to any one of claims 1 to 4, wherein, The image processor is configured to determine the body outline of the livestock based on the partial point cloud of the animal, and to determine the position of the hind legs and / or the forelegs based on the body outline.

11. The system according to any one of claims 1 to 4, wherein, The image processor is configured to evaluate the animal pose of the livestock based on the animal partial point cloud, and discard the animal partial point cloud if it does not meet the pose usefulness index.

12. The system of claim 11, wherein, The posture usefulness index includes: the highest part of the animal partial point cloud is located in the rear part of the livestock.

13. The system of claim 12, wherein, The rear portion includes the hindquarters of the livestock.

14. The system of claim 11, wherein, The posture usefulness index includes: the symmetry of the animal partial point cloud about its long axis is at least as high as the symmetry threshold.

15. The system of any one of claims 1 to 4, further comprising an animal identification device arranged to identify the individual livestock animal upon the visit.

16. A method for monitoring the growth of individual livestock animals with forelegs and hind legs in a pen environment, wherein one or more such livestock animals are able to walk in the pen environment and the floor of the pen environment is covered with bedding, the method comprising: - 3D images of the field of view are repeatedly obtained by a 3D camera with a field of view, which includes the portion of the shed environment in which individual livestock may be present during a visit; - The acquired image is processed by an image processor, wherein the processing includes - Extract the 3D point cloud representation of this field of view. - Detect whether one of the livestock animals exists in the point cloud representation. • If so, then - Select the animal portion of the point cloud represented by the point cloud; - Determine the first height information related to the location, that is, the height of the animal part of the point cloud above the floor; as well as - Determine the position of the forelegs and / or hind legs of the individual livestock animal based on the partial point cloud of the animal. • If not, then - Determine the location-related second height information, i.e., the height of the padding material above the floor; - The height of the individual livestock during the visit is determined based on the first height information, the second height information, and the position of the forelegs and / or the hind legs.

17. The method of claim 16, wherein the livestock animal comprises a cow.

Images