Animal weight determination based on 3D imaging

Abstract

The present disclosure relates to determining the weight of animals, particularly cattle, in a shed environment based on vision techniques, particularly 3D imaging. A first embodiment relates to a method for determining the weight of an animal with a known breed, comprising the steps of obtaining at least one 3D image of the animal's back, extracting data from the at least one 3D image relating to the topology of the animal's back, and calculating the weight of the animal by correlating the extracted data with a reference model comprising information of the back topology versus weight for the breed of the animal.

Description

Technical Field

[0001] The present disclosure relates to vision technology, and more particularly, to the determination of the weight of animals, especially cattle, in a barn environment based on 3D imaging.

Background Art

[0002] When operating large-scale agriculture with a large number of individual animals, it is a challenge to always be aware of the healthy state of every animal. The healthy state of animals is very important not only from the humanitarian perspective of the farmer who is responsible for the healthy state of these animals, but also because healthy and well-nourished animals ensure further productivity and are more valuable to the farmer. The healthy state of animals is highly correlated with the weight of the animals, especially the weight stability of grown animals and the weight gain of still-growing animals. Therefore, frequent monitoring of the weight of animals is of great concern.

[0003] The process of weighing livestock is typically provided by physically placing the animal on a weighing scale and recording the weight of that particular animal either manually or digitally. This is often a laborious and long process because modern farms typically have a large number of animals and each animal would need to be individually placed on the weighing scale one by one in order to log the individual weight of each animal. Furthermore, the animal would need to be identified in the process in order to attribute the correct weight to the correct animal. This identification can occur, for example, by the use of identification tags on the animal and can further extend the process. Additionally, when animals live in barns or pastures where they move around independently and freely, it is even more difficult to track the weighed and unweighed animals. Due to the complexity of the process of weighing animals, weighing is not widely used and the amount of data available when evaluating the weight development of each animal is limited or even non-existent.

[0004] International Publication No. WO2014 / 026765 (Patent Document 1) discloses a mobile 3D camera-based approach for determining parameters for livestock such as body weight, partial body weight, or lean meat percentage. In the method, a 3D image of the livestock is recorded to generate a 3D model, which is analyzed based on predetermined reference values to output and / or store information obtained from the analyzed livestock.

[0005] International Publication No. WO2010 / 127023 (Patent Document 2) describes a non-contact system and method for estimating the volume, mass, or body weight of an animal. Preferably, the animal is imaged using a stereo camera, and a 3D representation of the target animal is derived from the stereo images. A software module is provided to reshape a virtual model using a set of independently configurable shape variables so as to substantially conform to the spatial representation of an individual animal. The mass or body weight of the animal is estimated as a function of the shape variables that characterize the reshaped virtual model.

[0006] International Publication No. WO2015 / 156833 (Patent Document 3) describes a method for estimating body weight from an image of an animal. The system uses markers to characterize the body weight, health, and other parameters of the animal. The system is configured to log these parameters in a time database.

[0007] Nir et al. (Biosystem engineering, 173, p. 4 - 10 (2018)) describe a method for determining the body weight of a cow from image data, estimating the shape of the animal by an ellipse, and calculating the approximate body weight of the animal.

[0008] The problem in using 3D images for weight determination is often that each animal needs to be selected and immobilized in 3D imaging equipment. Often, this is done by placing the animal in a small compartment where side walls or bars basically restrain the animal so that it cannot move. In this case, since the animal needs to be individually confined in a small enclosed space, the use of 3D imaging does not eliminate the need to handle each individual animal, and thus the process is long-term, labor-intensive, and somewhat manual. Furthermore, the setup is often complex and expensive, making weight determination an expensive operation.

[0009] Furthermore, the weight of an animal, especially its weight development, is an important measure for the general physical health of the animal. Thus, it is also an important parameter when evaluating the animal's body condition score. The health of the animal is even more important for having productive animals in the sense that well-nourished animals can conceive from a young age and produce more milk and meat.

Prior Art Documents

Patent Documents

[0010]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0011] The object of the present disclosure is to provide a system and method for automatically or semi-automatically determining the weight of an animal based on imaging of the animal, preferably, exclusively based on imaging of the animal, preferably, 3D imaging from above the animal.

Means for Solving the Problems

[0012] One aspect of the present disclosure relates to a method for determining the weight of an animal, preferably an animal accompanied by a known breed, the method comprising the step of obtaining at least one image, preferably a 3D image, of the back of the animal, preferably the animal. It has previously been shown that the back of an animal conveys a lot of information about a particular animal. The method may further comprise the step of extracting data, preferably data regarding the topology and / or topography of the back of the animal, from the at least one image. Advantageously, the weight of the animal can be calculated by relating the data extracted from each other to a reference model having information on the topology of the back compared to the weight with respect to the breed of the animal.

[0013] The inventors recognize that the weight of an animal is exclusively related to each other in a 3D shape, i.e., the topology and / or topography of the back of the animal. That is, parameters such as the age, lactation, and biological structure of the animal can deviate from the estimate. This makes it possible to estimate the weight of an animal based only on a 3D image obtained from above so that the topology of the back of the animal can be extracted from the 3D image. This also makes it possible to install the system disclosed herein in a barn, cowshed, pigsty, etc., and obtain an image of the animal from above, for example, while the animal is eating, or, for example, while passing through a lock on the way back from milking. In the approach disclosed herein, it has been shown that it requires a reference model that is a reference model for the specific breed of the animal and relates the topology of the back of the animal to the weight of the animal. It has also been shown that only a very limited number of topology predictors extracted from the acquired 3D image are sufficient to accurately determine the weight of the animal. This estimates the need to manually place it on a weighing scale or place it in a closed compartment to select each animal and obtain information about its weight. This further enables continuous monitoring of the weight of animals, even daily and multiple times a day.

[0014] In particular, the inventors recognize that the contour points extracted from the 3D image data of the animal's back are very well related to each other and to the weight of the animal, particularly a cow. Thus, the data can advantageously be extracted in the form of a discrete number of contour points from the at least one 3D image. The contour points can be extracted, for example, typically along a line of maxima along the back of the animal defined as the spine of the animal that defines the vertical direction within the 3D image of the animal. Experiments have typically shown that, similarly assuming a reference model based on contour points, fewer than 20 contour points are sufficient to accurately predict the weight of the animal. Thus, only the contour points extracted from the 3D imaging of the animal's back are directly related to each other and to the weight of the animal. That is, there is no need to include complex 3D point cloud models or animal-specific parameters.

[0015] The present disclosure further relates to a system for determining the weight of an animal of a known breed, comprising an imaging system configured to obtain at least one 3D image of the animal's back and a processing unit configured to execute the disclosed method.

[0016] The disclosed systems and methods for weight determination can eliminate the need to handle each animal individually in the process of determining their weight, which allows for the possibility of frequent monitoring of each individual animal in a large herd. For example, the animal can walk through a lock / narrow passage connecting the rest area of the shed to the feeding and / or milking area. The passage can be very narrow, allowing only one animal to pass through at a time. One or more images of the animal's back can then be obtained by one or more cameras installed above the passage. The camera can be any type of camera that provides 3D information, such as, but not limited to, a time-of-flight (ToF) camera, a stereo camera, a structured light camera, a bright-field camera, or a combination thereof. The imaging system can comprise a 2D camera and a depth sensor. The imaging system can be configured to obtain a topographic image.

[0017] The possibility that the animal can walk while the image is being obtained allows for frequent monitoring of the animal's weight as this can occur without interfering with the animal's daily life. This frequent monitoring of the animal's weight further enables the farmer to collect data for statistics on the animal's weight. This data can be used in the mathematical modeling of the animal's weight to distinguish weight contributions into short-term variations and long-term changes. Short-term variations can be due to the animal's swelling and / or feed content and / or the animal's edema. However, long-term changes reflect the actual weight changes in the animal's body such as: - changes in muscle and fat mass, and / or - distribution, and / or - growth of the skeleton in the case of animals that are not fully grown, and / or - growth of the fetus in the case of pregnant animals, and / or - the event of giving birth in the case of pregnant animals

[0018] The short-term weight development of the animal, such as changes within a day, is related to the duration since the last feeding and / or milking. Therefore, in a preferred embodiment, the times of these events are logged. In this way, it will be possible to create a model of the time-dependent short-term variations in the animal's weight. In this regard, it should be noted that the daily variations in the animal's weight can be an indicator of the animal's health. For example, large daily variations can be an indicator of health problems. The present invention provides, for example, the following. (Item 1) A method for determining the weight of an animal with a known breed, the method comprising: obtaining at least one 3D image of the back of the animal; extracting data in the form of a discrete number of contour points from the at least one 3D image regarding the topology of the back of the animal, the contour points being extracted with respect to the line of maxima along the back of the animal; calculating the weight of the animal by relating the contour points to each other with respect to a reference model; and the reference model has information on the topology of the back versus the weight regarding the breed of the animal. (Item 2) The method according to item 1, wherein the extracted data consists of 5 to 30 contour points. (Item 3) The method according to any one of items 1-2, wherein the extracted data comprises less than 20 contour points. (Item 4) The method according to any one of items 1-3, wherein the extracted data consists of 10 to 20 contour points. (Item 5) The data is extracted from the 3D image by contour plotting the back of the animal to generate a contour line for the spine defined as a line of passage of the maximum value, and the contour points on the contour line are based on a relative decrease in height with respect to the height of the spine at a given position along the spine. The method according to any one of items 1-4. (Item 6) The method according to any one of items 1-5, wherein the extracted data comprises less than 5 contour points selected from the spine and less than 5 contour points selected from each of less than 5 contour lines for the spine. (Item 7) The method according to any one of items 1-6, wherein the extracted data comprises 1 to 10 contour points selected from the line of maxima along the back of the animal, such as 4 contour points selected from the line of maxima along the back of the animal, and 1 to 10 contour points, such as 4 contour points selected from each of 1 to 10 contour lines for the line of maxima along the back of the animal. (Item 8) A single contour point of the contour line is defined as a predetermined decrease in the height of the back of the animal with respect to the height of the spine, and the decrease in height with respect to a point on the spine of the animal is found along a line perpendicular to the spine. The method according to any one of items 1-7. (Item 9) The contour line is generated at a discrete interval at a height of 15 cm or less, preferably 10 cm or less, with respect to the height of the spine, according to the method according to any one of items 5-8 above. (Item 10) The method according to any one of items 1-9, further comprising the step of identifying the animal based on the at least one 3D image. (Item 11) The method according to any one of items 1-10, wherein the animal is moving during acquisition of the at least one 3D image. (Item 12) The method according to any one of items 1-11, wherein the at least one 3D image is based on a plurality of 3D images acquired while the animal is moving. (Item 13) The method according to any one of items 1-12, wherein the animal is preferably a bovine subject including both female and male cattle, regardless of whether it is an adult animal or a neonatal animal. (Item 14) The breed of the animal is selected from the group consisting of Jersey breed, Friesian cattle, Holstein Swartbont cattle, German Holstein Schwarzbunt cattle, American Holstein cattle, Red and White Holstein breed, German Holstein Schwarzbunt cattle, Danish Red group, Finnish Ayrshire group, Swedish Red and White group, Danish Holstein group, Swedish Red and White group, and Nordic Red group, according to the method according to any one of items 1-13 above. (Item 15) A system for determining the weight of an animal with a known breed, the system comprising: An imaging system configured to obtain at least one 3D image of the back of the animal; A processing unit And The processing unit is To extract data in the form of a discrete number of contour points from the at least one 3D image regarding the topology of the back of the animal, the contour points being extracted with respect to the line of maximum values along the back of the animal; Calculating the weight of the animal by relating the contour points to each other with respect to a reference model And is configured to perform The reference model is a system having information on the topology of the back versus the weight with respect to the breed of the animal. (Item 16) The system according to item 15, configured to obtain the at least one 3D image while the animal is standing in the stall and / or while walking through the stall. (Item 17) The system according to any one of items 15-16, configured to obtain the at least one 3D image when triggered by the animal approaching and / or entering the lock. (Item 18) The system according to any one of items 15-17, wherein the processing unit is configured to execute the method according to items 1-14.

Brief Description of the Drawings

[0019] The present invention will be described in more detail below with reference to the drawings. The drawings are illustrative and are intended to illustrate some of the features of the method and system and are not to be construed as limiting the disclosed invention.

[0020]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

DETAILED DESCRIPTION OF THE INVENTION

[0021] The ordinary meaning of the term "topography" is that it is the study and description of the physical characteristics or shape of an area (e.g., its hills, valleys, or rivers, or the representation of these features on a map), that is, usually used in conjunction with geography. In this case, the topography of the animal's back is thus understood as the shape of the animal's back, that is, a 3D shape with any "hills" and "valleys" that appear on the animal's back.

[0022] The term "topology" as used herein refers to a particular body area, structure, or part, in this case, typically, the biological structure of the animal's back. The biological structure of an animal is the structure of its body, e.g., the outer shape of the body.

[0023] In a preferred embodiment, the extracted data used for correlation to the reference model has a discrete number of contour points to simplify the calculations. Preferably, fewer than 50 contour points, more preferably fewer than 25 contour points, even more preferably fewer than 20 contour points, and most preferably fewer than 15 contour points such as 12 contour points. It has been shown that only 12 appropriately selected contour points extracted from a 3D image of the back of an animal may be sufficient to determine the weight of the animal. That is, 10 to 50 contour points are sufficient to determine the weight of the animal.

[0024] The spine of the animal is preferably located within at least one 3D image of the back of the animal. The spine of the animal can be defined as a line passing through the maximum of the height as illustrated in FIG. 1. As a result, the spine of the animal can be used to define the vertical direction within the 3D image of the animal.

[0025] In a preferred embodiment, the data is extracted from a 3D image by contour plotting the back of the animal and generating a contour line with respect to the spine. The contour line can be based on the relative decrease in height with respect to the height of the spine at a given position along the spine, i.e., the contour line connects contour points of equal altitude. Thus, a single contour point of the contour line can be defined as a predetermined decrease in the height of the back of the animal with respect to the height of the spine, and the decrease in height with respect to a point on the spine of the animal is found along a line perpendicular to the spine as illustratively shown in FIG. 2.

[0026] As described above, the inventors recognize that an animal's weight can be predicted using only a very limited number of data points extracted from a 3D image. In one embodiment, the extracted data includes fewer than 20 contour points selected from the spine (fewer than 10 contour points, such as 1 to 10 contour points, fewer than 7, 6, or 5 contour points, 4 contour points, etc.), and / or fewer than 20 contour points (fewer than 10 contour points, such as 7, 6, or 5 contour points for the spine, 1 to 10 contour points, 4 contour points, etc.) selected from each of fewer than 10 contour lines (fewer than 7, 6, or 5 contour lines for the spine, 1 to 10 contour lines, 3 contour lines, etc.). The contour lines are preferably generated at discrete intervals at a height of 15 cm or less, preferably 10 cm or less, relative to the height of the spine. For example, contour lines at 2.5 cm, 5 cm, and 10 cm relative to the spine.

[0027] Animal identification is not necessarily required to determine weight, and typically only the animal's breed needs to be ascertained. However, ascertaining the identity of the specific imaged animal is an advantage for further data analysis and monitoring of individual animals. Identification can be manual, for example, by reading the identification number on an animal's ear tag, or by electronic identification means (such as radio frequency ID (RFID), by animal pattern recognition, by a known sequential sequence of the animal, or by other means of identification). The animal can be identified prior to or after image acquisition.

[0028] As shown in International Publication No. WO2017 / 001538, it is possible to (uniquely) identify an animal within a known population of animals based on an image of the animal's back. Thus, the disclosed approach can further include the step of identifying the animal based on the at least one 3D image.

[0029] The recognition that the determination of an animal's weight is possible based on 3D imaging of the animal's back enables the acquisition of one or more images while the animal is moving. Thus, in one embodiment of the present disclosure, the animal is moving during the acquisition of the at least one 3D image. Two or more 3D images can be obtained from the animal. Thus, the at least one 3D image is preferably based on a plurality of 3D images, which can be obtained while the animal is moving. As a result, the data analysis can be based on the median image of two or more images.

[0030] An example of a contour plot is illustrated in FIG. 1, where each line shows the contour plot of the back of a cow. The vertical midline is formed by the maximum value of the height in the image and defines the spine of the animal. In one embodiment, the contour plot is a line and / or points along the back of the animal, all of which correspond to a specific amount of height reduction relative to the spine within that particular area. Preferably, the outer contour of the animal reflecting the periphery of the animal (meaning the physical extent of the animal such as the width of the animal's back) is not included in the weight calculation. Thus, in a preferred embodiment, the weight is based exclusively on the topology and / or topography of the back rather than the periphery and / or width of the animal. The weight is thus based entirely on the topography and / or topology of the back.

[0031] A possible strategy for creating such a contour plot is illustrated in FIG. 2, where the animal's body is represented by the ellipse 20. The spine is found along the back of the animal as the highest point in the central area of the back, i.e., the maximum of the height, when proceeding along the center line connecting the neck to the tail. The position of the spine, i.e., the spinal axis, is represented by the dashed line 21 in FIG. 2. Along the spinal axis 21, a given number of points of interest are selected. In FIG. 2, four points are selected and represented by 22a-d, each marked along the spine with their respective crosses. From a selected point on the spine, e.g., 22a, a line is drawn perpendicular to the axis of the spine, which is represented by 23a in FIG. 2. From the point of interest 22a on the spine, the vertical line 23a is traced in one direction, e.g., towards the right side of the animal, towards the edge of the animal. When a decrease in height of X cm relative to the height of the spine at 22a is reached, this point is recorded in the database. X refers to a real number. Next, this process is then repeated as it proceeds along 23a towards the opposite side of the animal, which could be the left side of the animal, and the same level of decrease in height of X cm is recorded on this opposite side of the spine. This process is repeated for all points of interest 22b-d along the spine 21 and along their respective vertical lines 23b-d. When the positions of all the points representing a given decrease in height of X cm relative to the points of interest 22a-d along the spinal axis 21 are identified, a line is fitted to best explain the position of the points. This fitted line here represents the contour line of the relative decrease in height of X cm for the individual points along the spine. If further contour lines for different values of the decrease in height are desired, this process can be repeated for other values of the relative decrease in height with respect to the height of each point along the spine. Alternatively, all relevant contour points along a given vertical line can be found before moving them onto a line perpendicular to the next point of interest along the spine. These relevant contour plots can be, for example, X cm, Y cm, Z cm, and T cm (X, Y, Z, and T refer to real numbers). In FIG. 1, for example, the values of X, Y, Z, and T are 3, 5, 10, and 15 cm respectively, each resulting in their respective contour lines.

[0032] The explanatory diagram of FIG. 2 is a simplified explanatory diagram for the purpose of illustrating the strategy for creating a contour plot in the simplest possible way. Therefore, the animal is illustrated as an ellipse only for the sake of simplicity of illustration. In other words, the body shape of the animal is not regarded as an ellipse by the strategy for creating the contour plot. In a true data handling process such as that of FIG. 1, since the animal has an irregular shape and is not an ellipse, the spine is not necessarily a perfect straight line. Therefore, the perpendicular line at a given point along the spine can be estimated based on the number of adjacent points on the spine relative to the point of interest. These points can be fitted to a straight line, for example, and the perpendicular line is determined based on this fitted line. Therefore, the best estimate of the perpendicular line for a given point on an irregular line is made when dealing with true data.

[0033] Therefore, the contour plot can be based on a discrete number of points and the best fit to these points. Thus, the contour plot does not involve a fixed same reference for all points of the contour plot, but rather reflects the decrease in height relative to the spine at all positions along the back of the animal. Therefore, in this embodiment, a single contour point of the contour line reflects a specific decrease in height on the back relative to the height of the spine along the line perpendicular to the spine of the animal passing through the contour point, whereby the basis of the contour plot is not the final height relative to a given fixed point, but rather all contour points are calculated relative to their respective distinct reference points along the spine of the animal. This further means that the contour plot of the back of the animal can occur along the length of the back of the animal, in other words, along the spine of the animal. In a preferred embodiment, the line connecting the individual points of the topography of the back is created as the best fit to the points representing a given decrease in height, and thus the fitted line reflects the best fit for the contour line.

[0034] In one embodiment, the animal is walking during image acquisition. The 3D image can thus be acquired while the animal is moving, for example while walking in a straight line. The ability to acquire weight information of a moving animal enables the determination of the weights of a number of animals lined up in a row, and thus this is a great advantage as it eliminates the need to handle individual animals individually. When the animal is walking from the rest area of the shed to the feeding and / or milking area or in the opposite direction, the animal is passing through a narrow passage, which may only allow one animal to pass through at a time. By installing an imaging system above this narrow passage, the animal passes through the area below the camera, and the camera can then capture an image of each individual animal as it passes through the passage. A great advantage of this is that the animals will not overlap in their views from the 3D camera from above. Thus, the animals pass through the narrow passage and through the frame area of the 3D camera when they are directed or guided, for example, from the rest area to the feeding and / or milking area or in the opposite direction. Another advantage of the narrow passage is that it is possible to obtain good control of the animal flow.

[0035] An example of this process of guiding animals through a narrow passage is shown in FIG. 3. Here, the thick line 1 illustrates the boundary of the area where the animals can move. The boundary can be a fence, or a wall, or a similar enclosure mechanism. Each animal is illustrated by an ellipse in FIG. 3 for simplicity. The arrow connected to each animal (ellipse in the figure) illustrates the direction in which the animal is moving. The animal 2 on the left has already passed through the narrow passage, and the animal 3 standing directly below the 3D camera 4 is located within the narrow passage and is currently being imaged by the 3D camera 4. The animal 5 behind has just entered the narrow passage, and thus will be imaged as soon as it passes through the area directly below the 3D camera 4 when the animal 5 reaches the current position of the animal 3. The animals 6 on the right all represent animals that have entered the narrow passage and are still waiting to finally enter the area on the left side of the passage.

[0036] In certain embodiments of the present disclosure, two or more images of the back of an animal are obtained. This can be a plurality of images of the animal while it is walking under the camera or a plurality of images of the stationary animal. If the animal is moving, the image can be requested as long as the animal has any part of its body inside the imaging frame of the camera. Alternatively, the image can be obtained only during the period when the animal has its entire body inside the imaging frame of the camera. Yet another alternative is that the camera can obtain images only for a given preset period or the camera can request a preset number of images.

[0037] In a preferred embodiment, the data analysis is based on the median image of two or more images. Thus, all or some of the obtained images should be combined to generate an average of the shape of the back of the animal. The term "median image" refers to an average image generated as an average of the topography and / or biostructure information of the back of the animal within all the collected images. One advantage of using such a median image is that the movement of the back of the walking animal during image acquisition can be smoothed in the median or average image, thereby eliminating walking-induced variations in topography.

[0038] One significant advantage of using imaging instead of having the animal walk on a scale is that when an animal walks on a scale, it may not have all of its feet on the scale when the weight is recorded. In addition, in such a setting, the animals typically walk very close together, and it is possible for two or more animals to stand partially or fully on the scale simultaneously, which can cause an error in the scale reading when trying to establish the weight of an individual animal.

[0039] In one embodiment, the animal is stationary during data acquisition. Preferably, the method will be compatible with both moving and stationary animals. Preferably, the imaging system is configured to obtain images while the animal is walking. In one embodiment, the imaging system is configured to obtain images while the animal is stationary. Most preferably, the imaging system is capable of acquiring data for accurate weight calculation regardless of whether the animal is moving or not.

[0040] Surprisingly, it has been found that the topography and / or topology of the animal's back is very significantly related to the animal's weight, so other clearly essential features such as the height of the animal's stomach above the floor or the width of the animal's waist and / or shoulders are unnecessary parameters for the purpose of estimating the animal's weight using the disclosed system and / or method. Thus, the inventors have unexpectedly recognized that the shape of the animal's back is sufficient for determining the animal's weight with high precision.

[0041] In a preferred embodiment, the acquisition of at least one image of the animal's back is performed at least once a day, preferably multiple times, to capture time- and condition-specific variations in weight. Condition-specific variations can refer to weight development, which affects the animal's weight and can vary over a given time series, long or short, along with other parameters that the animal depends on, for example, whether the animal is: - Just fed - Just milked - Recently gave birth - Pregnant - Not fully grown Time-specific variations can be, for example, morning weight versus evening weight. The variations occur because in the morning, the animal has been without food for a long time, while in the evening, the animal has accumulated the contents of its digestive system over the whole day or at least part of the day.

[0042] Preferably, all the acquired data is stored in a database. The data can then be accessed at a later time, which can further be used to plot a time-dependent plot of weight development that can help in the process of finding unhealthy animals. The method can, therefore, be combined with methods for pattern recognition and / or machine learning in order to perform an initial diagnosis of animals showing weight development that should be monitored. In certain embodiments, the process of monitoring and modeling the weight change of an animal is based on a large number of measurements obtained over a long period of time, such as multiple days, preferably multiple weeks, more preferably multiple months, etc. Preferably, these data also include multiple data points per day for each day over the acquisition time, and in a preferred embodiment, this data is supported by the feeding and / or milking event schedule of the animal. Since the processes of digestion and feed intake are highly dependent on the time since the last feeding, this data strongly supports the modeling of short-term variations when the weight data is accompanied by data on the amount of time since the last feeding and / or milking.

[0043] In a preferred embodiment of the present disclosure, the time of the latest feeding / milking event is stored together with the acquired data regarding the weight of the animal. This information can be included in the weight calculation algorithm.

[0044] When evaluating weight development, it can be noted that short-term and long-term variations are evaluated separately. Preferably, the data history is used to generate a model for separating the calculated weight measurements into contributions of at least long-term and short-term variations. More preferably, the calculated weight measurements are used to mathematically model the short-term and long-term variations in weight determination. Thus, the model can subdivide the variations in weight measurements into short-term and long-term changes. In certain embodiments of the present disclosure, the short-term variation in weight measurements is the change over a day or daily. These short-term variations in weight determination can be attributed to the animal's distension and / or the feed content and / or edema of the intestinal system. The long-term changes in weight determination can be regarded as gradual changes over a period of at least several days, preferably over several weeks, more preferably over several months. While these long-term changes in the animal's weight can be attributed to the growth / regression of the body of a non-pregnant animal, the long-term changes can be attributed to the growth / regression of the body, muscle, fat, and / or skeleton.

[0045] Generally, the approach disclosed herein can determine the weight of an animal independently of the animal's age, number of lactations, and whether the animal is pregnant or ill. For example, from the perspective of pregnancy, there will be a significant weight gain over a period of time, but since the weight gain due to pregnancy will result in an increase in the topography / topology of the back, the weight gain will be detected by the approach disclosed herein. Thus, the approach disclosed herein can be used to detect pregnancy and / or to monitor pregnancy during the gestation period.

[0046] In a further embodiment, pregnancy is included in the weight determination model. That is, when it is ascertained that a particular animal is pregnant, the model may consider improving the accuracy of weight determination in order to more closely monitor the pregnant animal. Thus, pregnancy can be compensated for, and / or incorporated, and / or otherwise included in the weight modeling such that the process of growing the fetus is not confused with the animal's own weight gain and such that the effect of pregnancy on the animal's dorsal profile can be appropriately included in the model. In a further embodiment, long-term changes in a pregnant animal are attributed to body growth / regression and the growth of the fetus of the pregnant animal. Preferably, pregnancy is included in the algorithm for determining the animal's weight based on data obtained from an optical sensor. The possible pregnancy of an animal can be manually registered in the system for that particular animal. Over the course of the pregnancy, the weight development of the animal can be logged on the condition that the animal is growing a fetus. Pregnancy can change the parameters of the algorithm for calculating the weight, whereby the calculated weight better fits the pregnant animal.

[0047] The event of giving birth is registered in the data and can be compensated for by a mathematical model or otherwise recorded in a database so as not to confuse this event with a sudden and dramatic physical deterioration of the livestock. Further, the weight loss attributed to the event of giving birth is, in certain embodiments of the present disclosure, used to estimate the rate of weight gain during pregnancy that can be attributed to the growth of the fetus. Thus, this contribution to weight gain can be retroactively introduced into the mathematical model. Thus, in certain embodiments of the present disclosure, the event of giving birth is registered in the data and included in the mathematical model so as to ensure an appropriate calculation of the current weight of the animal and possibly also for other purposes. One approach for registering the event of giving birth is that when an animal gives birth, the event of giving birth is logged by a processing unit. The logging can occur manually or automatically. The weight loss following the event of giving birth can further be used to calculate the amount of weight gain of the animal due to weight gain associated with pregnancy over the duration of the pregnancy. Weight gain associated with pregnancy can be due to factors such as the growth of one or more fetuses and one or more placentas, increased blood volume, and additional fat storage, among several parameters. The approach disclosed herein can then compare the weight before and during pregnancy to the weight after giving birth.

[0048] The imaging system typically includes at least an optical device, e.g., a camera, and in order to keep at least one optical device clean, the device may ultimately need to be cleaned and / or possibly protected from dirt in a cabin environment. One possible solution is to use a protective cover that keeps dirt away from the actual optical device. In a preferred embodiment, at least one optical device is protected by a protective cover so as to prevent dirt from directly covering the functional part of the optical device. In a further embodiment, the cover of at least one optical detector is cleaned using an automated cleaning system such as an automatic front windshield wiper. Thus, if the optical device cover gets overly dirty and cannot capture the desired data, the cover can be automatically cleaned by activating a cleaning system such as an automatic front windshield wiper.

[0049] Another approach for keeping at least one optical detector clean is, for example, by protecting the at least one optical detector with a gate or shutter that opens only for a short period when obtaining data. Thus, the detector will be exposed only for a very short amount of time when collecting data by the at least one optical detector. Accordingly, in one embodiment, the at least one optical detector is protected by a gate or shutter that opens only for a short period when obtaining data. Thus, the gate or shutter will open briefly for image acquisition and then close again, avoiding the optical device from getting dirty. In the latter case, the optical device includes both a protective cover and a gate / shutter, and the gate / shutter will thus prevent the cover of the optical device from getting dirty too quickly. To optimize the time that the optical detector is exposed to collect data and thus to optimize the time for the shutter to open, the shutter or gate of the system may include another feedback system for determining when the animal is standing in the appropriate position for data acquisition. This feedback system can be an independent system based on sensors installed next to the 3D camera. The position of the animal can be determined from another detector and / or sensor that is not covered by the shutter or gate, which determines when the animal is in the optimal position for imaging. When this occurs, the shutter opens, a range image is taken, and then the shutter closes immediately. It may also be possible to obtain a series of images before the shutter closes.

[0050] In a further embodiment, at least one optical detector includes a cleaning alert system that senses when the sensor needs to be cleaned and notifies the user through the alert system such as light turning on, noise being reproduced, or a wireless signal being transmitted to a computer. This is particularly useful when the device does not have an automated cleaning system or is insufficient to completely clean the device. In this case, the alert system will make the user aware that further cleaning of the device or the device's cover is required. The user can then manually clean the device or activate an automated cleaning system. The cleaning process can then be automatic or manual. If the automatic cleaning process is sufficient, it may not be necessary to notify the user, and this step can be omitted as long as the system can efficiently self-clean.

[0051] As used herein, an animal can be a bovine subject including both cows and bulls, whether preferably an adult animal or a newborn animal. As a result, the breed of the animal can be selected from the groups of Jersey breed, Holstein breed, Holstein Friesian cattle herd, Holstein Swartbonte cattle herd, German Holstein Schwarzbuentner cattle herd, US Holstein cattle herd, Red and White Holstein breed, German Holstein Schwarzbuentner cattle herd, Danish Red herd, Finnish Ayrshire herd, Swedish Red and White herd, Danish Holstein herd, Swedish Red and White herd, and Nordic Red herd. (Example) (Example 1)

[0052] To generate a reference for Jersey breed cows, the backs of individual animals in a herd of 102 Jersey cows were imaged over a four-week period using the systems and methods of the present disclosure. The animals in the herd had an average weight of 460 kg and a weight range of 350 - 650 kg. During the four-week period, 1,329 measurements of the backs of the animals in the herd were taken, and the number of measurements per animal varied between 3 and 15 measurements over the four-week period. Of the 1,329 measurements, 1,149 of them were taken in the morning on any given day during the four-week period, and 180 were taken in the afternoon. Images were obtained simultaneously (within the same minute) when the measured value of the animal's weight was obtained using a weighing scale. After the generation of the reference model, it was thus possible to test and cross-validate the approach disclosed herein.

[0053] Three contour lines identified by the contour plotting method disclosed herein were generated on each side of the spine. Using the approach of each point representing the same relative decrease in height with respect to the spine, the three contour lines associated with decreases of 2.5 cm, 5 cm, and 10 cm with respect to the spine each had their reference points along the spine as described above. In this example, 100 points along the spine were detected, and the 100 points were registered for each of the height decreases found with respect to the spine. Thus, 300 points on each side of the spine were used to estimate the contour plot used to generate the reference model.

[0054] The model used in this example is a partial least squares (PLS) model. Four hundred profile variables from the back of the cows are analyzed in the partial least squares model. This is due to the autocorrelation between the variables. Using all 400 variables in the model results in an overfitted model, and most of the variables will not contribute to the model using redundant information. A predictive model can be generated using a number of predictors equal to explaining over 98% of the variation in body weight. The prediction of body weight is estimated using the number of variables suggested by the model. The residuals are defined as the difference between the predicted and observed variables for each individual body weight phenotype. For all phenotypes, both the registration of body weight and the profile description of the cow's back are available.

[0055] In Examples 1 and 2 described herein, it was found that only 12 profile points selected from the spine and 3 profile lines are sufficient to predict body weight with adequate accuracy. In a particular case, 4 points selected along the spine, 3 profile points from the profile line at 2.5 cm, 1 profile point from the profile line at 5 cm, and 4 points from the profile line at 10 cm. The spine was generated using a total of 100 profile points, and each of the 3 profile lines was generated using 100 profile points. The predictor points were as follows: C0_5, C0_32, C0_64, and C0_90 from the spine, C1_24, C1_54, and C1_85 from the profile line at 2.5 cm, C2_48 from the profile line at 5 cm, and C3_11, C3_40, C3_72, and C3_98 from the profile line at 10 cm. As can be seen from the predictor points, they are distributed along the profile lines to cover additional areas of the animal's back.

[0056] Similar results can be obtained using other profile points, but the principle is that only a limited number of profile points selected from a limited number of profile lines (including the spine) are sufficient to adequately describe the 3D shape of the animal to determine body weight.

[0057] Figure 4 shows all 1,329 acquired data points for this example. Thus, each data point reflects, based on the approach disclosed herein, the measured weight on a scale of an animal in one instance and the corresponding calculated weight of the same animal. The data is plotted as the calculated weight as a function of the weight measured on the scale. Generally, the data demonstrates the reliability of the approach disclosed herein, even when other relevant parameters such as age, duration since parturition, or the number of offspring the animal had are not included in the determination of weight, by showing a perfect match between the calculated weight and the measured weight of the animal. Thus, in this example, the calculation is purely based on 12 contour point predictors (and nothing else) extracted from an image of the animal's back.

[0058] Considering the deviation of the calculated weight from the measured weight, the still relatively sparse data set reflects a relatively symmetric normal distribution deviation between the calculated data set and the measured data set. This is illustrated in Figure 5. The almost perfectly symmetric distribution of the deviations around 0 demonstrates that there are no systematic errors related to the data analysis. The same information can be drawn from Figure 6 where the residuals are plotted as a function of the measured weight. Generally, the distribution is relatively symmetric around a residual of 0, indicating that there is no strong tendency for the residuals to be systematically wrong with respect to a given weight sub-group of the animals.

[0059] By obtaining the average calculated body weight and the average measured body weight of each animal among all the measurements performed on each animal during a 4-week period (each animal has 3 - 15 pairs of measurements and calculations), the data presented in Figure 7 is reached. As can be seen, the agreement between the measured body weight and the calculated body weight of each animal improves, and is also expected, assuming that each point here reflects a larger amount of data. Each point in Figure 7 (102 points) thus reflects the average body weight (calculated and measured) of an individual animal within the group. Figure 8 reflects the difference between the average measured body weight and the average calculated body weight as a function of the average measured body weight. This figure does not show a weight-dependent bias towards the deviation between the measured body weight and the calculated body weight.

[0060] It is known that animals will lose weight in the days after giving birth because they require all of those resources contained in the extra fat stores that increased during pregnancy to produce milk. It will, therefore, slowly break down the stores in its body. After about 50 days, as milk production decreases, the animal will begin to gain weight, and the animal can then eat sufficiently to actually rebuild its fat stores. The animal will, therefore, gain weight again. After a sufficiently long period, the animal will become pregnant again and begin to gain weight. Therefore, as a sanity check of the current model of this example, 1,329 measurements and calculations of weight have been plotted as a function of the number of days since a particular animal last gave birth. The data is illustrated in FIG. 9 and shows a slow increase in the weight of the animal as a function of the number of days since parturition. The data in FIG. 9 reflects the weight of the animal obtained over four weeks, and it should be noted, therefore, that none of the animals have been tracked over a sufficiently long period to map the significant weight development of their individual weight development as a function of the number of days since parturition. The residuals of the weight calculations and measurements as a function of the number of days since parturition are illustrated in FIG. 10, and again, no clear systematic errors appear either from the weight data or as a function of the number of days since parturition. The model is, therefore, considered to be stable against bias as a function of weight and the number of days since parturition based on the data obtained for this example. (Example 2)

[0061] To test whether the data is applicable across different groups, the same approach as described above in Example 1 was repeated for different groups of Jersey cows on a different farm. The second group included 162 Jersey breed cows with an average weight of 519 kg, i.e., on average, the cows in Group 2 were about 60 kg heavier.

[0062] Figure 11 shows data for group 2 and corresponds to Figure 4 which shows the same type of data for group 1. Figure 11 shows all of the obtained data points for group 2. Thus, each data point reflects the measured weight on a scale of an animal in one instance and the corresponding calculated weight of the same animal based on the approach disclosed herein. The data is plotted as the calculated weight as a function of the weight measured on the scale. Generally, the data reflects a perfect match between the calculated weight and the measured weight of the animals demonstrating the reliability of the approach disclosed herein. Also in this example, the calculation is purely based on 12 contour point predictors (nothing else) extracted from an image of the back of the animals within group 2.

[0063] Figure 12 shows data for group 2 and corresponds to Figure 7 which shows the same type of data for group 1. As can be seen in Figure 12, the match between the measured weight and the calculated weight of each animal is better and is also expected given that each point here reflects a larger amount of data.

[0064] Table 1 below summarizes the data for group 1 and group 2. [Table 1]

[0065] As can be seen from the table above, the STD among the cows was larger than in group 1 where the cows were on average the smallest.

[0066] A PLS model was used to predict the weight based on 400 contours on the back of the cows from both groups. The r between the predicted weight and the observed weight 2 was 0.90 and a cross-validation study where the cows were randomly assigned to 5 different groups over lactation and across groups showed a model reliability of 0.82 and an RMSE of 0.54. The results are independent of the number of days milk is produced and the number of lactations and are based only on the information available in the images. (Example 3)

[0067] To test whether the same approach described above is applicable across different bovine breeds with different data, the same approach as described above in Examples 1 and 2 was repeated for different groups of Jersey, Holstein, and Red Dairy cattle, respectively. Body weight data were recorded for four groups (two groups of Jersey cattle, one group of Red Dairy cattle, and one group of Holstein cattle) over a six-week period. The group sizes were not equal, resulting in an unequal distribution of data among the groups, but this did not affect the results. Using fewer than five contour points selected from the spine and fewer than five contour points selected from each of fewer than five contour lines for the spine, body weight was predicted using an approach as disclosed herein, and the data show that on average there is little difference between the observed body weight and the predicted body weight. In 10-fold cross-validation, the root mean square error (RSME) was estimated, and the data are presented in Table 2 and Figures 13 - 15 below, showing the average observed body weight plotted against the predicted body weight for Red Dairy cattle (Figure 13), Jersey cattle (Figure 14), and Holstein cattle (Figure 15).

Table 2

[0068] Random animal dispersion, total dispersion, and reproducibility of the acquired data are shown in Table 3 below. As can be seen from Table 3, the reproducibility of the measurements is greater than 0.9, i.e., very high.

Table 3

[0069] To analyze whether the prediction can be further improved by including data on the time of year and the number of lactations and weeks of lactation of individual animals, the body weight data were analyzed using the following model. Body weight = mean + group + week of year + number of lactations + weeks of lactation + animal + residual

[0070] Random animal dispersion, total dispersion, and reproducibility are recalculated, and the results are presented below in Table 4.

Table 4

[0071] As can be seen from Table 4, the model is improved only slightly by including additional animal-specific parameters, the dispersion is reduced, and the reproducibility increases slightly further. The approach disclosed herein, based only on the contour points of the discrete numbers extracted from the 3D imaging of the animal's back, is sufficient to accurately predict the animal's weight when a specific reference model, particularly a breed-specific reference model, is available, i.e., it is concluded that animal-specific parameters can be excluded from the weight prediction and a practical implementation can be made feasible in a real-life environment. (Further details)

[0072] 1. A method for determining the weight of an animal with a known breed, the method comprising: - obtaining at least one 3D image of the animal's back; - extracting data from the at least one 3D image regarding the topology of the animal's back; - calculating the weight of the animal by correlating the extracted data with a reference model having information on the topology of the back compared to the weight for the breed of the animal. The method as described in item 1, including.

[0073] 2. The method according to item 1, wherein the extracted data comprises contour points of a discrete number, preferably less than 50 contour points, more preferably less than 25 contour points, even more preferably less than 20 contour points, and most preferably less than 15 contour points such as 12 contour points.

[0074] 3. The method according to any of the preceding items, wherein the animal's spine is located within at least one 3D image of the animal's back.

[0075] 4. The spine of an animal is the method described in any of the preceding items, defined as a line of maximum values.

[0076] 5. The spine of an animal is the method described in any of the preceding items, defining the vertical direction of a 3D image.

[0077] 6. The data is extracted from a 3D image by contour plotting the back of an animal to generate a contour line for the spine, the method described in any of the preceding items.

[0078] 7. The contour points on the contour line are based on the relative decrease in height with respect to the height of the spine at a given position along the spine, the method described in any of the preceding item 0.

[0079] 8. The extracted data comprises 1 to 10 contour points, such as 4 contour points selected from the spine, and 1 to 10 contour points, such as 4 contour points selected from each of 1 to 10 contour lines, such as 3 contour lines for the spine, the method described in any of the preceding items.

[0080] 9. The contour line is generated at discrete intervals at a height less than or equal to 15 cm, preferably less than or equal to 10 cm, with respect to the height of the spine, the method described in any of items 0 - 0 above.

[0081] 10. A single contour point of the contour line is defined as a predetermined decrease in the height of the back of the animal with respect to the height of the spine, and the decrease in height with respect to a point on the spine of the animal is found along a line perpendicular to the spine, the method described in any of the preceding items.

[0082] 11. The method described in any of the preceding items, including the step of identifying an animal based on the at least one 3D image.

[0083] 12. The animal is moving during the acquisition of the at least one 3D image, the method described in any of the preceding items.

[0084] 13. The method according to any of the preceding items, wherein two or more 3D images of an animal are obtained.

[0085] 14. The method according to any of the preceding items, wherein the at least one 3D image is based on a plurality of 3D images obtained while the animal is moving.

[0086] 15. The method according to any of the preceding items, wherein the data analysis is based on a median image of two or more images.

[0087] 16. The method according to any of the preceding items, wherein the animal is a bovine subject, including both female and male cattle, regardless of whether it is an adult animal or a newborn animal.

[0088] 17. The method according to any of the preceding items, wherein the breed of the animal is selected from the group consisting of Jersey breed, Holstein breed, Holstein Friesian herd, Holstein Swartbonte herd, German Holstein Schwarzbunt herd, American Holstein herd, Red and White Holstein breed, German Holstein Schwarzbunt herd, Danish Red herd, Finnish Ayrshire herd, Swedish Red and White herd, Danish Holstein herd, Swedish Red and White herd, and Nordic Red herd.

[0089] 18. A system for determining the weight of an animal with a known breed, the system comprising: - an imaging system configured to obtain at least one 3D image of the back of the animal; - a processing unit, the processing unit - extracting data from the at least one 3D image regarding the topology of the back of the animal; - calculating the weight of the animal by relating the data extracted with respect to a reference model, which has information on the topology of the back in comparison with the weight for the breed of the animal, to each other; a processing unit configured to perform the above; and a system comprising the above.

[0090] 19. The system according to item 18, wherein the at least one 3D image is obtained from above the animal, thereby configured to image the animal in a top view.

[0091] 20. The system according to any one of items 18-0, configured to obtain the at least one 3D image while the animal is standing inside the pen and / or walking through the pen.

[0092] 21. The system according to any one of items 18-0, configured to obtain the at least one 3D image when triggered by the animal approaching and / or entering the pen.

[0093] 22. The system according to any one of items 18-0, wherein the processing unit is configured to execute the method according to any one of items 1-0.

Claims

1. 1. A computer-implemented method for determining a body weight of an animal with a known breed, the computer-implemented method comprising: obtaining at least one 3D image of the animal's back; extracting data in the form of a discrete number of contour points from said at least one 3D image related to a topology of said back of said animal, said contour points being extracted relative to a line of local maxima along said back of said animal; calculating the weight of the animal by directly relating the contour points to a reference model; Including, the reference model comprises information of the back topology versus the weight for the breed of the animal; A computer-implemented method in which the data is extracted from the 3D image by contour plotting the back of the animal to generate a contour line for the spine defined as a passing line of local maxima, contour points on the contour line are based on the relative decrease in height relative to the height of the spine at given locations along the spine, the contour line being generated at discrete intervals at heights of 15 cm or less, for example 10 cm or less, relative to the height of the spine, and the reference model includes non-volumetric topological features specific to the breed of the animal, the non-volumetric topological features being associated with breed-specific weight patterns to take into account differences in body structure.

2. The computer-implemented method of claim 1 , wherein the extracted data consists of between 5 and 30 contour points.

3. The computer-implemented method of any one of claims 1 to 2, wherein the extracted data comprises less than 20 contour points.

4. The computer-implemented method of any one of claims 1 to 3, wherein the extracted data consists of between 10 and 20 contour points.

5. 5. The computer-implemented method of claim 1, wherein the extracted data comprises less than five contour points selected from the vertebrae and less than five contour points selected from each of less than five contour lines to the vertebrae.

6. 6. The computer-implemented method of claim 1, wherein the extracted data comprises 1 to 10 contour points selected from the line of local maxima along the back of the animal, such as 4 contour points selected from the line of local maxima along the back of the animal, and 1 to 10 contour points, such as 4 contour points selected from each of 1 to 10 contour lines, such as 3 contour lines, to the line of local maxima along the back of the animal.

7. 7. A computer-implemented method according to any one of claims 1 to 6, wherein a single contour point of a contour line is defined as a predetermined diminution in height of the back of the animal relative to the height of the spine, and the diminution in height to a point on the spine of the animal is found along a line perpendicular to the spine.

8. The computer-implemented method of any one of claims 1 to 7, comprising identifying the animal based on the at least one 3D image.

9. The computer-implemented method of any one of claims 1 to 8, wherein the animal is moving during the obtaining of the at least one 3D image.

10. The computer-implemented method of any one of claims 1 to 9, wherein the at least one 3D image is based on a plurality of 3D images acquired while the animal is moving.

11. 11. The computer-implemented method of any one of claims 1 to 10, wherein the animals are bovine subjects, including both cows and bulls, whether adult or newborn animals.

12. 12. The computer-implemented method of any one of claims 1 to 11, wherein the breed of the animal is selected from the group of: Jersey breed, Friesian herd, Holstein Swartbond herd, German Holstein Schwarzbund herd, American Holstein herd, Red and White Holstein breed, German Holstein Schwarzbund herd, Danish Red herd, Finnish Ayrshire herd, Swedish Red and White herd, Danish Holstein herd, Swedish Red and White herd, Nordic Red herd.

13. 10. A system for determining the weight of an animal with a known breed using the computer-implemented method of claim 1, the system comprising: an imaging system configured to obtain at least one 3D image of the back of the animal; Processing unit and Equipped with The processing unit includes: - extracting data in the form of a discrete number of contour points from said at least one 3D image relating to a topology of said back of said animal, said contour points being extracted relative to a line of local maxima along said back of said animal; calculating the weight of the animal by directly relating the contour points to the reference model; The device is configured to: The reference model comprises information about the back topology versus the weight for the breed of the animal.

14. 14. The system of claim 13, wherein the system is configured to obtain the at least one 3D image while the animal is standing in the lock and / or walking through the lock.

15. The system according to any one of claims 13 to 14, wherein the system is configured to acquire the at least one 3D image upon triggering by the approach and / or entry of the animal into a lock.

16. The system according to any one of claims 13 to 15, wherein the processing unit is configured to execute the computer-implemented method according to claims 1 to 12.

Images