Systems and methods for detecting lameness in cattle - Patents.com

Abstract

The present disclosure relates to detecting lameness in bovine subjects, in particular dairy cows, in a shed environment based on vision techniques, in particular 3D imaging. A first embodiment relates to a method for detecting lameness in a bovine subject, comprising the steps of acquiring at least one 3D image of the back of the bovine subject, extracting data representative of the spine of the bovine subject, determining at least one curvature of at least one preselected portion of the spine, and detecting lameness and / or determining the degree of lameness of the bovine subject by correlating said curvature with at least one reference value, such as at least one predefined threshold value.

Description

[Technical field]

[0001] The present disclosure relates to detecting lameness in bovine subjects, particularly dairy cows, in a barn environment based on vision techniques, and in particular 3D imaging. [Background technology]

[0002] When operating a large-scale farm with many individual animals, it is a challenge to be constantly aware of the health of every animal. The health of bovine subjects is very important, both from a humane point of view of the farmer who is responsible for the health of these animals, and because healthy and well-nourished animals ensure greater productivity and are of greater value to the farmer. Therefore, it is important to frequently monitor the health of bovine subjects.

[0003] Locomotor diseases causing lameness are considered one of the most serious welfare problems for dairy cows and cause increased costs for dairy farmers. Lameness is not a disease in itself, but it is a symptom of pain or discomfort for the cow when moving. It is characterized by a change in gait or an irregular walking pattern. It is typically caused by nail and / or leg disorders that indicate the cow is trying to reduce the amount of weight that it bears on the affected limb. It is therefore considered to be an indicator of an underlying problem that may cause pain to the cow. It is associated with lower feed intake, reduced milk production and reduced reproduction, and may lead to early culling. Reduced mobility of the cow therefore affects its overall health and welfare and should therefore be avoided. The majority of lameness cases in dairy cows are related to pain-inducing, infectious or non-infectious nail lesions.

[0004] Subjective methods are currently used to assess lameness in cattle on farms, with results typically given in the form of a numerical rating score. Such subjective assessments typically rate individual cattle for the presence or absence of certain behaviors and postures related to gait. These scoring systems focus primarily on the movement or gait associated with the degree of resistance of the cattle to putting weight on the affected limb. The number of scoring categories typically ranges between 2 and 5.

[0005] An example of a widely used subjective scoring system for assessing lameness is the Sprecher system, which is a 5-point rating system ranging from 1 ("normal") to 5 ("severe lameness"). It is widely used due to its clarity and observation of behavior, e.g., the presence of hunched back when standing and walking. The advantage of the Sprecher system is that it is non-extensive and can be easily applied in a farm context. The Sprecher system allows individuals such as dairy farmers and their employees, veterinarians, hoof trimmers and instructors to subjectively perform lameness assessments for use in herd management. However, subjective scoring systems such as the Sprecher system suffer from a lack of precision and are typically only reliable for cows that are either moderately lame, lame or severely lame. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2017 / 001538 [Non-patent literature]

[0007] [Non-Patent Document 1] Christen A.-M., C. Egger-Danner, N. Capion, N. Charfeddine, J. Cole, G. Cramer, A. Fiedler, T. Fjeldaas, N. Gengler, M. Haskell, B. Heringstad, M. Holzhauer, G. de Jong, A. Koeck, J. Kofler, K. Mueller, J. Pryce, Å. M. Sogstad, KF Stock, G. Thomas, E. Vasseur,2021,"Recording Lameness in Dairy Cattle",2021 ICAR meeting Summary of the Invention [Problem to be solved by the invention]

[0008] Preventing lameness helps optimize milk production, improves conception rates and animal welfare, reduces treatment costs and antibiotic use, and therefore reduces stress levels in both cows and dairy farmers. It is therefore an object of the present disclosure to provide a system and method for objectively detecting lameness in cows and / or determining the extent of lameness based on cow imaging, preferably exclusively based on cow imaging, preferably 3D imaging obtained from above the cow. [Means for solving the problem]

[0009] One aspect of the present disclosure thus relates to a method for detecting lameness in an animal, typically a bovine subject, such as a bovine subject of a known breed, comprising obtaining at least one image, preferably a 3D image, of the bovine subject, preferably of the back of the bovine subject, where it is previously known that the back of the bovine subject conveys a great deal of information about that particular animal.

[0010] The method may further comprise the step of extracting data from said at least one image, preferably data relating to the topology and / or topography of the back of the bovine subject, e.g. extracting data representing the spine of the bovine subject, which may for example be represented by a line of maxima along the back of the bovine subject.

[0011] At least one curvature of at least one preselected portion of the spine may then be determined. Lameness may then be detected by correlating said curvature with at least one reference value, such as at least one predefined threshold. Additionally or alternatively, the degree of lameness may be determined based on, for example, a difference between the curvature and a corresponding reference value.

[0012] Alternatively and / or additionally, lameness in a bovine subject can be detected by extracting data relating to the topology of the back of the bovine subject in the form of contour points from said at least one 3D image and detecting lameness in the bovine subject and / or determining the extent of lameness by correlating said contour points to a reference model.

[0013] The inventors understand that lameness of a bovine subject correlates with the 3D shape, i.e., topology and / or topography, of the bovine subject's back. That is, parameters such as the age, lactation, etc. of the bovine subject may be excluded from the estimation. This allows for detection of lameness of a bovine subject based only on 3D images acquired from above, such that the topology of the bovine subject's back can be extracted from the 3D images. This also allows the disclosed system to be installed in a barn, cowshed, etc., and images of the bovine subject acquired from above, for example, while the bovine subject is eating or passing through a lock, e.g., to or from milking.

[0014] In the disclosed approach, it has been shown that all that is required is a reference model and / or simply one or more curvature thresholds, possibly for a particular breed of bovine subject and / or for a particular individual animal. It has also been shown that a very limited number of topology predictors extracted from the acquired 3D images is sufficient to accurately detect lameness and / or lameness degree of a bovine subject. This eliminates the need for subjective scoring of each animal in the herd. This further allows for the health of bovine subjects to be continuously monitored on a daily basis, and even multiple times each day.

[0015] In particular, the inventors have realised that the shape and / or contour points of the spine extracted from the 3D image data of the back of a bovine subject are highly correlated with lameness in bovine subjects, particularly dairy cows. Data can therefore advantageously be extracted from said at least one 3D image in the form of a disparate number of contour points. The contour points can for example be extracted relative to a line of maximum along the back of the bovine subject, typically defined as the spine of the bovine subject defining a longitudinal direction in the 3D image of the bovine subject. In some cases, only data representative of the spine needs to be extracted to detect lameness and / or the degree of lameness in the bovine subject.

[0016] The present disclosure further relates to a system for detecting lameness in a bovine subject, the system comprising an imaging system configured to acquire at least one 3D image of the back of the bovine subject, and a processing unit configured to acquire at least one 3D image of the back of the bovine subject, extract data representative of the spine of the bovine subject, determine at least one curvature of at least one pre-selected portion of the spine, and detect lameness in the bovine subject by correlating said curvature to at least one pre-defined threshold.

[0017] The disclosed system and method for lameness detection can completely eliminate the need to treat each animal individually and subjectively in the process of detecting and assessing lameness, allowing each individual animal in a large herd to be frequently monitored. For example, a bovine subject may walk through a lock / narrow corridor that connects a shed rest area with a feeding and / or milking area. The corridor may be narrow enough to allow one animal to pass at a time. One or more images of the bovine subject's back may then be acquired by one or more cameras placed above the corridor. The camera may be any type of camera that provides 3D information, such as, but not limited to, a time-of-flight (ToF) camera, a stereo camera, a structured light camera, a light field camera, or a combination thereof. The imaging system may include a 2D camera and a depth sensor. The imaging system may be configured to acquire a topographic image.

[0018] As also disclosed in the commonly owned WO2017 / 001538, such a camera may be installed above the feeding area and used to capture 3D images of the bovine subjects, with which individual bovine subjects can be identified so that each bovine subject can also be monitored for lameness in the pen.

[0019] The ability to allow the bovine subject to walk while the images are acquired allows frequent monitoring of the lameness of the bovine subject, as this can be done without disturbing the bovine subject's daily routine. This frequent monitoring of the lameness of the bovine subject further allows farmers to collect statistical data on the lameness and / or lameness severity of the bovine subjects in their herd. This data can be used in mathematical modeling of the bovine subject to distinguish between short-term and long-term fluctuations and changes. Short-term fluctuations can be due to the swelling and / or feeding of the bovine subject and / or bloating of the bovine subject. Long-term changes, however, reflect changes in the weight of the bovine subject, for example, as follows: - changes in muscle and fat mass, and / or - distribution, and / or - skeletal growth in animals that are not fully grown, and / or - in the case of pregnant animals, the development of the foetus, and / or - In pregnant animals, the event of parturition.

[0020] Another aspect of the present disclosure relates to artificial intelligence based detection of lameness, i.e., models are trained to detect lameness, preferably determine the degree of lameness, for example based on expert labeling data and / or labeling of actual bovine subjects.

[0021] One embodiment relates to a method for detecting lameness in a bovine subject, the method comprising acquiring a plurality of range images of the back of at least one bovine subject. The range images are preferably acquired from above the at least one bovine subject. Image processing can then be applied to the range images to generate at least one sequence of 3D images of a pre-selected bovine subject. Artificial intelligence (AI) is then utilized to model the at least one sequence of 3D images against a trained reference model to detect lameness and / or determine the degree of lameness of the pre-selected bovine subject.

[0022] A further embodiment relates to a system for detecting lameness in a bovine subject, comprising an imaging system configured to acquire a plurality of range images of the back of the bovine subject, and a processing unit configured to perform the disclosed method.

[0023] That is, for example, veterinary expertise can be brought virtually into the barn and integrated into a vision system that acquires range images from above, for example, as the bovine subjects walk to the milking so that they can be continuously viewed.

[0024] The disclosed methods may advantageously be computer-implemented.The present disclosure also relates to a system for detection of lameness in a bovine subject, comprising a computer-readable storage device for storing instructions that, when executed by a processor, perform the methods disclosed herein.

[0025] The disclosed methods may be implemented as one or more separate computer programs, but may also be implemented as a plug-in to an existing service running on a device or online.

[0026] The present disclosure therefore also relates to a computer program having instructions that, when executed by a computing device or system, cause the computing device or system to perform the detection of lameness as disclosed herein. A computer program in this context is broadly contemplated and includes, for example, a program to be run on a PC or software designed to run on a smartphone, tablet computer or other mobile device. Computer programs and mobile applications include software that is free and software that must be purchased, and also software that is distributed via a distribution software platform or runs on a server.

[0027] The present disclosure is described in more detail below with reference to the drawings, which are exemplary and intended to illustrate some of the features of the present methods and systems and are not intended to be limiting on the disclosure. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 shows contour lines of the back of a bovine subject corresponding to height loss relative to the vertebral level. [Diagram 2] FIG. 1 is a schematic diagram of an embodiment of a process for generating a contour map of the back of a bovine subject. [Diagram 3]Schematic diagram of the driving of animals through passages / locks / narrow corridors from one shed area to another on the way to or returning from milking. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Topographical methods As noted above, one embodiment relates to a method for detecting lameness in a bovine subject, the method comprising: - acquiring at least one 3D image of the back of a bovine subject; - extracting data representative of the spine of a bovine subject; - determining at least one curvature of at least one preselected portion of the spine; - detecting lameness and / or determining the degree of lameness in a bovine subject by correlating said curvature with at least one reference value, such as at least one predefined threshold value; and / or - extracting data relating to the topology of the back of the bovine subject from said at least one 3D image in the form of contour points, - detecting lameness and / or determining the extent of lameness in a bovine subject by correlating said contour points against a reference model.

[0030] The extracted data may comprise at least 50 data points, preferably at least 100 data points. The reference model may comprise information on the topology of the back and the corresponding degree of lameness, for example in relation to the breed of said bovine subject.

[0031] The ordinary meaning of the term "topography" is the study and description of the physical features or shape of an area, for example its hills, valleys, or rivers, or the representation of these features on a map, i.e. as commonly used with topography. In this case, the topography of the back of a bovine subject is therefore understood as the shape of the back of the bovine subject, i.e. the 3D shape along with any "hills" and "valleys" that appear on the back of the bovine subject.

[0032] The term "topology" as used herein refers to a particular body area, structure, or part, in this case the anatomy of a bovine subject, typically the back. The anatomy of a bovine subject is the structure of the subject's body, e.g., the outer shape of the body.

[0033] The usual procedure for subjectively assessing and scoring lameness by manual observation of individual cows is that if the cow stands and walks with a flat back posture and movements are smooth and fluid with normal gait, the cow is not lame, but if the cow stands with a flat back posture but develops a hunched posture while walking, the cow is considered to be moderately lame. The Sprecher scoring system continues up to level 5, where the cow has extreme hunching while standing and walking and obvious joint stiffness with obvious head shaking when the sore foot / hoof contacts the ground.

[0034] However, based on large-scale 3D imaging data acquisition from five dairy herds, the inventors realize that it is meaningless to talk about a cow having a "flat back", because all cows have a curved back, and the degree of curvature varies over the longitudinal extension of the back. The inventors have also surprisingly discovered that there is variability in the variation of curvature over the back. In order to detect subtle signs of lameness, it is therefore important to have appropriate reference values ​​and / or appropriate reference models, possibly specific to individual herds, breeds and / or bovine subjects.

[0035] Thus, the reference model may comprise information of the topology of the back and the corresponding degree of lameness, for example in relation to the breed of said bovine subject. Additionally and / or alternatively, the reference values ​​and / or the reference model may be specific to said bovine subject. Using the possible image acquisition set-ups outlined herein, the reference values ​​and / or the reference model may advantageously be generated from pre-acquired 3D images of the back of said bovine subject, e.g. acquired over a time period of at least 2 days, more preferably at least 5 days, even more preferably at least 1 week, even more preferably at least 2 weeks, most preferably at least 4 weeks.

[0036] The curvature of the spine, or only a preselected portion of the spine, is preferably defined only as the vertical curvature of said preselected portion of the spine, i.e. the rate of vertical change along the longitudinal extension of the cow, i.e. between head and tail. The curvature of the preselected portion of the spine is then preferably determined by calculating the inclination of said preselected portion of the spine.

[0037] The inventors have realized that the posterior portion of the spine is relatively important for detecting and assessing lameness by looking exclusively at at least one preselected portion of the spine that is the posterior portion of the spine, i.e., toward the tail of the bovine subject. The posterior portion may be defined as the portion between the caudal portion of the spine and the mid-portion of the spine.

[0038] The inventor further understands that a lot of information is hidden in the local curvature of the spine of a bovine subject. Thus, at least a portion of the spine can be divided into subsections, and the curvature of each subsection is determined, such as at least 2, 3, 5, 7, preferably at least 8 or 9 or at least 10 subsections. At that point, a reference value can be given to each subsection, or at least a number of said subsections. Then, further, the detection of lameness can be correlated with a predefined difference between the curvature and the corresponding reference value. Furthermore, the degree of lameness can be correlated with the amount of difference between the curvature and the corresponding reference value.

[0039] In a preferred embodiment, the extracted data used for correlation against the reference model comprises a disparate number of contour points, e.g. a disparate number of contour points defining the vertebrae, for ease of calculation, e.g. less than 1000 contour points, preferably less than 500 contour points, even more preferably less than 250 contour points, most preferably 100 contour points or less, e.g. 50 or only 25 or only 12 contour points.

[0040] It can be shown that, if appropriately selected, as few as 100 or 50 contour points extracted from a 3D image of a bovine subject's back may be sufficient to detect lameness in a bovine subject.

[0041] The bovine subject's spine is preferably present in at least one 3D image of the bovine subject's back. The bovine subject's spine can be defined as a through-going line of local height maxima, as illustrated in Figure 1. Thus, the bovine subject's spine can be used to define the longitudinal direction in the 3D image of the bovine subject.

[0042] In one embodiment, data is extracted from the 3D image by contouring the back of the bovine subject, thereby generating a contour line relative to the spine. The contour line may be based on the relative decrease in height relative to the height of the spine at a given location along the spine, i.e., the contour line connects contour points of equal elevation. Thus, a single contour point of the contour line may be defined as a predefined decrease in height on the back of the bovine subject relative to the height of the spine, where the decrease in height relative to a point on the spine of the bovine subject is found along a line perpendicular to the spine, as exemplarily shown in FIG. 2.

[0043] As mentioned above, the inventors appreciate that lameness, and optionally the degree of lameness, of a bovine subject may be predicted with only a limited number of data points extracted from the 3D image. In one embodiment, the extracted data comprises less than 20 contour points, such as less than 10 contour points, such as between 1 and 10 contour points, such as less than 7, 6 or 5 contour points, such as 4 contour points, selected from the spine, and / or less than 10 contour lines, such as less than 7, 6 or 5 contour points, such as between 1 and 10 contour lines, such as less than 20 contour points, such as less than 10 contour points, such as less than 7, 6 or 5 contour points, such as between 1 and 10 contour points, such as 4 contour points, selected from each of the three contour lines, such as less than 10 contour points, such as less than 7, 6 or 5 contour points, such as between 1 and 10 contour points, such as 4 contour points, relative to the spine. The contour lines are preferably generated at discrete intervals at a height of 15 cm or less, preferably 10 cm or less, relative to the height of the spine. For example, contour lines at 2.5 cm, 5 cm and 10 cm relative to the spine.

[0044] Understanding that lameness detection is possible based on 3D imaging of the back of a bovine subject allows for one or more images to be acquired while the bovine subject is moving. Thus, in one embodiment of the present disclosure, the bovine subject is moving during acquisition of said at least one 3D image. Multiple 3D images may be acquired from the bovine subject. Thus, said at least one 3D image is preferably based on multiple 3D images, which may be acquired while the bovine subject is moving. Thus, data analysis may be based on a median image of two or more images.

[0045] An example of a contour plot is shown in Figure 1, where each line represents a contour map of a cow's back. A central vertical line is formed by the local maximum in height in the image and defines the spine of the bovine subject. In one embodiment, the contour plot is lines and / or points along the back of the bovine subject that all correspond to a particular amount of height reduction relative to the spine in that particular area.

[0046] One possible strategy for developing such a contour map is shown in FIG. 2, where the body of the bovine subject is represented by an ellipse 20. The spine is seen along the back of the bovine subject as the highest part of the central area of ​​the back, i.e. the local height maximum, going along the midline from the neck to the tail. The position of the spine, i.e. the spinal axis, is represented in FIG. 2 by a dashed line 21. A given number of points of interest are selected along the spinal axis 21. In FIG. 2, four points are selected, labelled 22a-d, and are marked with crosses along the spine. From the selected point of the spine, e.g. 22a, a line is drawn perpendicular to the axis of the spine, this perpendicular axis being labelled 23a in FIG. 2. From the point of interest on the spine, i.e. 22a, the perpendicular line 23a is followed along one direction towards the edge of the bovine subject, e.g. towards the right of the bovine subject. When a height reduction of X cm relative to the spine height at 22a is reached, this point is entered into the database, where X refers to a real number. This process is then repeated moving along 23a towards the other side of the bovine subject, which may therefore be the left of the bovine subject, and the same level of height reduction of X cm is entered for this other side of the spine. This process is repeated for all points of interest 22b-d along the spine 21 and along the respective perpendicular lines 23b-d. Once all points representing a given height reduction of X cm relative to the points of interest 22a-d along the spine axis 21 have been located, a line is fitted to best describe the location of said points. This fitted line now represents a contour of the relative height reduction of X cm relative to each individual point along the spine. If more contours for different values ​​of height reduction are desired, the process may be repeated for other values ​​of relative height reduction relative to the height of each point along the spine. Alternatively, all relevant contour points along a given perpendicular line can be viewed before proceeding to a line perpendicular to the next point of interest along the spine. These relevant contours can be, for example, Xcm, Ycm, Zcm, and Tcm, where X, Y, Z, and T refer to real numbers. In Figure 1, for example, the values ​​of X, Y, Z, and T are 3, 5, 10, and 15 cm, respectively, each resulting in a respective contour line.

[0047] The diagram in Fig. 2 is a simplified diagram for the purpose of illustrating the described strategy for developing a contour map in the simplest possible way. Therefore, the bovine subject is shown as an ellipse only for ease of explanation. In other words, the body shape of the bovine subject is not considered as an ellipse by the strategy for developing a contour map. In a correct data handling process, such as in Fig. 1, the spine is not necessarily a perfect straight line, since the bovine subject has an irregular shape and is not an ellipse. Therefore, the perpendicular line of a given point along the spine can be estimated to the point of interest based on several adjacent points on the spine. These points can be, for example, fitted to a straight line, and the perpendicular line is determined based on this fitted line. Therefore, a best estimate of the perpendicular line to a given point of the irregular line is made when dealing with correct data.

[0048] The contour map may therefore be based on a random number of points and a best fit to these points. In this way, the contour map reflects the decrease in height relative to the spine at every location along the back of the bovine subject, rather than a contour map with the same fixed reference for all points of the contour map. Thus, in this embodiment, a single contour point of the contour line reflects a particular height decrease in the back relative to the height of the spine along a line perpendicular to the bovine subject's spine that passes through said contour point, so that the base of the contour map is not the final height relative to a given fixed point, but rather every contour point is calculated relative to each of their separate reference points along the bovine subject's spine. This further means that the contouring of the bovine subject's back can occur along the length of the bovine subject's back, in other words along the bovine subject's spine. In a preferred embodiment, a line connecting each point of the back topography is drawn as a best fit to the points representing a given height reduction, so that the fitted line reflects a best fit for the contour line.

[0049] In an embodiment of the present disclosure, multiple images are acquired of the back of the bovine subject. This may be multiple images of the bovine subject while walking under the camera or multiple images of the bovine subject standing still. If the bovine subject is moving, images may be required as long as the bovine subject has any part of its body within the imaging frame of the camera. Alternatively, images may be acquired for a period of time during which the bovine subject has its entire body within the imaging frame of the camera. Yet another alternative is that the camera may acquire images for a given, pre-set period of time or the camera may require a pre-set number of images.

[0050] In a preferred embodiment, the data analysis is based on a median image of two or more images. Thus, all or part of the acquired images should be combined to generate an average of the bovine subject's back shape. The term "median image" refers to an average image generated as an average of the bovine subject's back topography and / or anatomical information in all collected images. One advantage of using such a median image is that the movement of the bovine subject's back walking during image acquisition can be smoothed in the median or average image so that walking-induced fluctuations in the topography can be eliminated.

[0051] In an embodiment, the bovine subject stands still during image acquisition. Preferably, the method is compatible with both moving and stationary animals. Preferably, the imaging system is configured to acquire images while the bovine subject is walking. In an embodiment, the imaging system is configured to acquire images while the bovine subject is standing still.

[0052] In a preferred embodiment, acquisition of at least one image of the back of the bovine subject is practiced at least once, and preferably multiple times, per day to capture time and situation specific variations.

[0053] Preferably, all acquired data is stored in a database. The data can then be accessed later and it is further possible to plot the time-dependent plot of the development of spinal curvature and / or spinal subsection curvature, which may aid the process of finding unhealthy animals. This method may therefore be combined with methods for pattern recognition and / or machine learning to perform early stage detection of lameness. In an embodiment, the process of monitoring and modeling the curvature and / or contour changes of the bovine subject is based on a large number of measurements acquired over an extended period of time, such as days, preferably weeks, and more preferably months. Preferably, these data also include multiple data points per day, every day over the acquisition time, and in a preferred embodiment, this data is supported by a time schedule of the feeding and / or milking events of the bovine subject. Since the process of expansion and feed intake is strongly dependent on the time since the last feeding, this data strongly supports the modeling of short-term fluctuations when this data is accompanied by data on the amount of time since the last feeding and / or milking.

[0054] When assessing the timely development of a cow, it may be interesting to assess short-term and long-term variations separately. Preferably, data history is used to generate a model for splitting the curvature variation at least into long-term and short-term fluctuation contributions. More preferably, measurements are used to mathematically model the short-term and long-term fluctuations. The model may thus sub-divide the variation of the curvature into short-term fluctuations and long-term changes. In an embodiment of the present disclosure, the short-term fluctuations of the curvature are the changes during the day or on a day-to-day basis. These short-term fluctuations may be due to the distension of the intestinal system and / or the feeding content and / or bloating of the bovine subject. Long-term changes of the curvature may be considered as gradual changes over a period of at least several days, preferably over weeks, more preferably over months. These long-term variations of the curvature may be key to determine early indications of lameness.

[0055] Spatial and Temporal Correlation Methods The approach relies on recoding time-series 3D data of a bovine subject to detect lameness in the bovine subject based on how the bovine subject looks and how it moves, and utilizing a trained AI model to detect lameness.

[0056] One embodiment relates to a method for detecting lameness in a bovine subject comprising acquiring a plurality of range images of the back of at least one bovine subject, the range images being acquired from above the at least one bovine subject; image processing the range images to generate at least one sequence of 3D images of a pre-selected bovine subject; and detecting lameness and / or determining the extent of lameness of the pre-selected bovine subject by utilizing artificial intelligence (AI) to model the at least one sequence of 3D images against a trained reference model.

[0057] The range images may be directly acquired 3D images or may be 2D image data including depth information from an RGB 2D camera combined with depth information from an infrared sensor such as a time-of-flight system, as known for example from the Kinect system. The 2D image data combined with the depth information can be processed to become a 3D image.

[0058] The plurality of range images may be a time series of range images and / or the sequence of 3D images of the pre-selected bovine subject may be a time series of 3D images.

[0059] The output of the disclosed approach may result from labeling the pre-selected bovine subjects as "no lameness", "possible lameness", or "moderate lameness", "lameness" or "severe lameness", i.e., after AI modeling. Additionally, after detecting lameness, a labeling step of the pre-selected bovine subjects as "lameness" using the degree of lameness as an integer between 1 and 5, such as the Sprecher score, to determine the degree of lameness.

[0060] To provide good temporal correlation, range images are preferably acquired at a high frequency, i.e., range images may be acquired at a frame rate of at least 10, or 15 fps, preferably at least 20 frames per second (20 fps), more preferably at least 25 fps, or even at least 30, 40 or at least 50 fps.

[0061] Since it may happen that the animal stops during the passage of the imaging unit, it would be sufficient for the bovine subject to actually move in order to trigger lameness detection. Thus, typically, the sequence of 3D images selected from the acquired range images includes the preselected bovine subject moving forward. Accordingly, it may be advantageous for lameness detection that the selected sequence of 3D images for use in modeling includes the preselected bovine subject moving forward a minimum distance of at least 1 meter, or at least 2 m, preferably at least 3 m or 4 m, more preferably at least 5 m, and most preferably at least 7 m, between 5 m and 10 m. Similarly, the sequence of 3D images selected for modeling includes the preselected bovine subject during a minimum period of at least 1 second, or at least 2 seconds, preferably at least 3 seconds, more preferably at least 5 seconds, and most preferably at least 7 or 10 seconds, between 5 seconds and 10 seconds.

[0062] As noted above, range image acquisition typically includes acquisition of infrared (IR) 2D image data to provide depth information. However, the inventors appreciate that where the IR data is 2D, this additional 2D IR image data may be an extra modality that is fed into the AI ​​model to enhance detection of lameness. Thus, image processing may include generating at least one sequence of 2D IR images of the back of a pre-selected bovine subject that corresponds to at least one sequence of 3D images.

[0063] Preferably, the range images are acquired from above a corridor through which a number of bovine subjects pass, e.g. at least once a day, as shown in e.g. Fig. 3. Identifying the preselected bovine subject accordingly, using at least one ID reader located near one or both ends of the corridor. In fact, this identification may be part of selecting the preselected animal, since the time instance of the identification readout gives a reference to a time series of range images, such that a corresponding sequence of 3D images can be generated for the identified bovine subject to be the preselected bovine subject. Thus, the sequence of 3D images of the preselected bovine subject preferably corresponds to a temporal (windowed) selection of a number of range images selected such that only the identified / preselected bovine subject is present in the sequence of 3D images.

[0064] The image processing step preferably includes background removal to separate the bovine subject from its surroundings in the range image, which may be part of the segmentation process, for example, since known background elements may be segmented, identified and removed from the 3D image data.

[0065] Image processing preferably also includes perspective correction of the range and / or 3D images to provide a good reference coordinate system for the 3D setup, which can be done by image processing methods known in the art.

[0066] The image processing advantageously includes a step of segmenting at least a portion of the acquired range images, for example using instance segmentation to select bovine subjects among the segmented instances and generate at least one sequence of said 3D images of the preselected bovine subjects. The instance segmentation can be performed using Mask-RCNN (Mask Region Convolutional Neural Network), an open source model that is a variant of deep neural networks and can detect objects in images and generate high-quality segmentation masks for each instance. That is, the instance segmentation can provide a mask that describes / contains / defines relevant pixels for each bovine subject in the acquired range images. There are other variants of instance segmentation neural networks, for example Cascade R-CNN and Mask2Former, that can also be applied here. In that respect, the segmented bovine animal instance located closest to the ID reader and at the time of the corresponding ID reading can be associated with the identity of the segmented bovine subject, leading to the selection of the preselected bovine subject.

[0067] The disclosed technique may further include generating a point cloud of the preselected bovine subject based on the range images and image processing, in particular segmentation, e.g. the point cloud can be generated based on a mask from the instance segmentation. The depth information can then turn the point cloud into a real 3D point cloud with 3D coordinates to generate at least one sequence of 3D images with point cloud coordinates representing the preselected bovine subject in the images, i.e. a sequence of 3D images with labeled point cloud data. This can then be combined with the labeled 2D infrared data. Perspective correction can also be applied here. These data with several modalities can be input into a trained machine learning model, e.g. a neural network, e.g. a convolutional neural network, to detect and / or characterize lameness in the bovine subject.

[0068] Thus, the sequence of 3D images used as input to the AI ​​model includes 3D point cloud data that may represent the movement of a pre-selected bovine subject in said sequence, and the sequence of 3D images used as input to the AI ​​model may further include 2D IR data of the pre-selected bovine subject.

[0069] The (trained) reference model may include information of the topology of the back of a bovine subject relative to the degree of lameness, e.g. for a breed of said preselected bovine subject. The trained reference model is preferably trained to include spatial correlation of lameness in a sequence of 3D images and temporal correlation of lameness in a sequence of 3D images. The trained reference model may be specific to said preselected bovine subject, e.g. specific to the breed of said bovine subject.

[0070] The trained reference model may be trained with the degree of lameness as an output result. The trained model may be trained using a supervised learning approach, where the AI ​​model is trained using labeled data, where the labeled data is obtained from a trained veterinarian determining the degree of lameness of a bovine subject determined by the trained veterinarian to be lame. The trained reference model may be trained using a combination of self-supervised learning and semi-supervised learning, where a self-supervised learning approach is provided to pre-train the AI ​​model with unlabeled data, and then the pre-trained AI model is fine-tuned with a relatively small set of labeled data using semi-supervised learning.

[0071] The trained reference model may be trained using machine learning, a neural network, a recurrent neural network, a convolutional neural network, or any combination thereof.

[0072] The AI ​​model may be selected from the group consisting of machine learning, neural networks, recurrent neural networks, convolutional neural networks, or any combination thereof.

[0073] The actual modeling preferably includes both spatial and temporal correlations of lameness in a sequence of 3D images with a trained reference model. That is, the AI ​​modeling becomes a kind of video prediction or sequence prediction model capable of recognizing patterns of movement of a bovine subject resulting from lameness when the subject is imaged from above while moving, typically moving forward. Examples of sequence prediction models are readily available multi-scale 3D Resnets such as ST-3DGMR (Spatial-Temporal 3D Grouped Multiscale ResNet Network), or ViViT (video vision transformer), where spatio-temporal tokens from a sequence of 3D images of a bovine subject can be encoded by a series of transformer layers. Sequence prediction models typically allow temporal and spatial relationships to be transformed separately to handle large amounts of data in video prediction.

[0074] The spatial and temporal modeling approach further relates to a system for detecting lameness in a bovine subject, comprising an imaging system configured to acquire a plurality of range images of the back of the bovine subject, and a processing unit configured to execute the disclosed method. The imaging system may be configured to acquire said range images while the bovine subject is standing and / or walking in a lock and / or passage. The imaging system may be configured to acquire said range images from above the bovine subject, thereby imaging said bovine subject from a top perspective. The imaging system may be configured to start acquiring range images when triggered by at least one bovine subject approaching and / or entering a lock / passage. The imaging unit may comprise at least one RGB camera and a depth sensor, such as an infrared sensor, for providing 2D infrared images. The imaging system may be configured to acquire range images at a frame rate of at least 15 frames per second, preferably at least 20 frames per second, more preferably at least 25 frames per second.

[0075] The system may be equipped with one or more ID readers for reading the identification numbers of bovine subjects entering and / or exiting the locks / passages.

[0076] Predictive analytics One of the advantages of an automated bovine monitoring system that can be installed above animal tracks is that many animals can be monitored over long periods of time to generate a large amount of 3D image data of the moving animals, potentially captured over weeks, months and even years. This can be used for very early predictive detection of lameness, since bovine subjects detected as visibly lame by the disclosed techniques or by a professional such as a veterinarian can be used as input.

[0077] In one embodiment, only the most recent image data in the dataset, or only relatively old cattle with obvious lameness detected manually, e.g. data from the last week, 2 weeks, or preferably 3 weeks, more preferably 4 weeks, most preferably 2 months, are labeled by the expert. This labeled dataset, in which each bovine subject is evaluated by the expert and categorized as lame or not lame, or by the degree of lameness, can then be used as output result for training a model based on previous data of the same animal, e.g. data that is at least 1 month, or at least 2 months, or at least 3 months, preferably at least 4 months, more preferably at least 5 months, most preferably at least 6 months, earlier. For example, the dataset may include data acquired over a 12-month time period. Data from 10-12 months is labeled by the expert, or the animals are manually evaluated and used as output result for training a model based on data from 1-3 or 1-6 months, i.e. data acquired at least 4-6 months earlier than the one evaluated by the expert. This may lead to the discovery of early predictors of lameness, i.e. predictors that are not visible or known to the experts, thereby significantly improving the model.

[0078] aisle In one embodiment, the bovine subject is walking during image acquisition. The 3D images can therefore be acquired while the bovine subject is moving, for example walking in a straight line. Being able to detect lameness information of moving animals is a major advantage, as it allows lameness detection and monitoring of a large number of animals in a line, thus eliminating the need to handle each animal individually. When animals are walking from a rest area of ​​the shed to a feeding and / or milking area, or in the opposite direction, the bovine subject may be passing through a narrow corridor, thereby allowing only one animal to pass at a time. By placing the imaging system above this narrow corridor, the bovine subject walks through the area under the camera, which can then capture an image of each individual animal as it passes through the corridor. A major advantage of this is that when viewed from the 3D camera from above, the bovine subjects do not overlap each other's field of view. Thus, when the bovine subjects are driven or led, for example, from a rest area to a feeding and / or milking area or in the opposite direction, they pass through a narrow corridor and through the frame area of ​​the 3D camera. Another advantage of a narrow corridor is that it is possible to obtain a good control of the flow of the animals.

[0079] An example of this process of guiding bovine subjects through a narrow corridor is shown in FIG. 3, where the bold line 1 indicates the boundary of the area in which the bovine subjects can move. The boundary may be a fence or wall or similar containment mechanism. Each animal is shown as an oval in FIG. 3 to simplify the illustration. The arrows connected to each animal (the oval in the illustration) indicate the direction in which the bovine subjects are moving. The bovine subjects 2 on the left are bovine subjects that have already passed through the narrow corridor, while the bovine subjects 3 standing just below the 3D camera 4 are the bovine subjects that are in the narrow corridor and are currently being imaged by the 3D camera 4. The bovine subjects 5 in the back have just entered the narrow corridor and will soon be imaged as they pass the area just below the 3D camera 4, and therefore when the animal 5 reaches the current position of the animal 3. The bovine subjects 6 on the right all represent bovine subjects that are still waiting to enter the narrow corridor and will eventually enter the area on the left side of the corridor.

[0080] Imaging unit The imaging unit comprises one or more cameras, preferably cameras adapted for range imaging, such as range cameras. There may be several imaging units distributed along the bovine subject passage so as to be able to image the bovine subject over a longer distance while moving. Examples of range imaging techniques applicable here are time-of-flight, stereo triangulation, structured light, light field imaging, e.g. 4D light field imaging, etc. The imaging unit may be configured to acquire topographic images. Each range camera may be provided with a depth sensor and a 2D camera, such as an RGB camera, for example known from the Kinect system. The imaging unit may comprise at least one infrared (IR) sensor, such as an IR camera, preferably providing 2D IR data. Depth data may then be acquired using the IR signal.

[0081] The imaging unit is preferably configured to capture images over a period of time at multiple frames per second (fps) capture such that the output is a video sequence with more or less, for example at least 5, 10 or 15 fps, preferably at least 20 fps, most preferably at least 25 fps or at least 30 fps, preferably both RGB, depth and IR, a high fps such that the video sequence includes both 3D data and 2D IR data.

[0082] The time period for video acquisition may be triggered by the detection of a bovine subject near the imaging unit, and the video acquisition continues while one or more bovine subjects pass by. Subsequent image processing, for example by segmentation, can separate the data so that a video sequence of each bovine subject can be extracted. The disclosed approach is preferably configured such that a bovine subject is imaged during a time period of at least 1 second, preferably at least 2 seconds, more preferably at least 3 seconds, most preferably at least 5 seconds, or even 8 or 10 seconds, for example between 5 and 10 seconds, while passing through the corridor / imaging unit. The bovine subject is preferably moving in the video sequence such that the movement dynamics of the bovine subject are recorded in the video sequence.

[0083] Alternatively or supplementally, the disclosed technique may be configured such that a single bovine subject is imaged over a predefined minimum distance to ensure there is movement, preferably ambulatory movement, in the image, for example a minimum distance of 2 m, or at least 3 m, preferably at least 5 m, most preferably at least 7 or even at least 10 m, for example between 5 and 10 meters.

[0084] An imaging system typically includes at least one optical device, e.g., a camera, and in order to keep the at least one optical device clean, the device may eventually have to be washed and / or possibly protected from dirt in the shed environment. One possible solution is to use a protective cover that keeps dirt away from the actual optical device. In a preferred embodiment, the at least one optical device is directly protected by a protective cover to prevent dirt from covering the functional parts of the optical device. In yet another embodiment, the cover of the at least one optical detector is washed using an automated cleaning system, such as an automatic windshield wiper. Thus, if the optical device cover is becoming too dirty to capture the desired data, the cover may be automatically cleaned by running a cleaning system, such as an automatic windshield wiper.

[0085] Another approach to keep the at least one optical detector clean is, for example, by having the at least one optical detector protected by a gate or shutter that opens for a short period of time when acquiring data. Thus, the detector is exposed for a very short amount of time when collecting data by the at least one optical detector. Thus, in one embodiment, the at least one optical detector is protected by a gate or shutter that opens for a short period of time when acquiring data. Thus, the gate or shutter opens for a short period of time for image acquisition and then closes again to avoid the optical device from getting dirty. In the latter case, the optical device includes both a protective cover and a gate / shutter, and the gate / shutter thus prevents the cover of the optical device from getting dirty too quickly. To optimize the time that the optical detector is exposed to collect data, and therefore the time that the shutter opens, the shutter or gate of the system may include another feedback system to determine when the bovine subject is standing in the right position for data acquisition. This feedback system may be an independent system based on a sensor placed next to the 3D camera. The position of the bovine subject may be determined from another detector and / or sensor that is not covered by the shutter or gate, which determines when the bovine subject is in a proper position for imaging. When this occurs, the shutter opens, a range image is taken, and the shutter immediately closes thereafter. It may also be possible to acquire a series of images before the shutter closes.

[0086] In yet another embodiment, the at least one optical detector includes a cleaning alert system that detects when the sensor needs to be cleaned, and notifies the user through said alert system, such as by turning on a light, making a noise, or sending a wireless signal to a computer. This is particularly useful when the device does not have an automated cleaning system, or when thorough cleaning of the device is not sufficient. In this case, the alert system makes the user aware that further cleaning of the device or the cover of the device is required. The user may then manually clean the device or activate the automated cleaning system. The cleaning process may therefore be automatic or manual. If the automated cleaning process is sufficient, it may not be necessary to notify the user, and this step may be omitted as long as the system can efficiently clean itself.

[0087] identification Knowing the identity of a particular imaged animal is an advantage for further data analysis and monitoring of the individual animal. Identification may be manual, for example by reading the identification number of the ear tag of the bovine subject, or by pattern recognition of the bovine subject, by a known, sequential sequence of the bovine subject, by electronic identification means such as radio frequency ID (RFID), or any other identification means. The bovine subject may be identified prior to image acquisition or after image acquisition.

[0088] When using subject-specific references, it may be necessary to know the identity of the bovine subject in the image. However, as shown in WO2017 / 001538, it is possible to (uniquely) identify an animal in a population of known animals based on an image of the back of the bovine subject. Thus, the disclosed technique may further comprise identifying the bovine subject based on said at least one 3D image.

[0089] Identification of the animals may not be necessary to detect lameness. However, knowing the identity of a particular imaged animal is an advantage for further data analysis and monitoring of the individual animals. Identification may be manual, for example by reading the identification number of the animal's ear tag, or by pattern recognition of the animals, by a known, sequential sequence of the animals, by electronic identification means such as radio frequency ID (RFID), or any other identification means. Identification (ID) readers, for example for reading (RFID) ear tags, are typically placed near the corridors, i.e. at the entrances and / or exits of the corridors through which the animals are expected to pass one by one. It should be noted that the entrances may be the exits or vice versa, depending on which way the animals pass. Thus, the animals may be identified prior to, during, and / or after image acquisition.

[0090] However, as shown in WO2017 / 001538, it is possible to (uniquely) identify an animal in a population of known animals based on an image of the animal's back. Thus, the disclosed technique may further comprise identifying the animal based on said at least one 3D image.

[0091] The system may also comprise an ID tag. Such an ID tag may be connected to the animal to be identified. The ID tag may be a visual and / or electronic ID tag. The electronic ID tag may be an electronic ear tag and / or an electronic ID tag attached to the animal, such as on a collar. An animal may be marked with one or more ID tags, such as at least one visual ID tag and / or at least one electronic ID tag. One example is at least one visual ear ID tag combined with at least one electronic ID tag on a collar. Another example is at least one visual ear ID tag combined with at least one electronic ear ID.

[0092] Segmentation A number of bovine subjects may be passing simultaneously and the acquired 3D images may include a number of animals. Segmentation may therefore be applied advantageously to isolate individual bovine subjects in the images, for example in the form of semantic segmentation to group pixels in a semantically meaningful way such that pixels belonging to different objects are grouped separately, but preferably in the form of instance segmentation such that a label is assigned to each object in the images, such that instance segmentation can be provided for individual bovine subjects in the 3D images, preferably in a sequence of 3D images.

[0093] Instance segmentation typically involves object detection of all objects in an image to classify individual objects and localize each object instance, e.g., using bounding boxes, and then segmenting each instance to classify each pixel into a fixed set of categories without distinguishing between object instances. In this approach, it is paramount to classify individual bovine subjects, such that a sequence of 3D images containing multiple bovine subjects is segmented to separate the individual bovine subjects into separate instances.

[0094] One example of instance segmentation is the known Mask-RCNN (Mask Region Convolutional Neural Network, a variant of a deep neural network that detects objects in images and generates high-quality segmentation masks for each instance), which can provide a mask describing the relevant pixels for each bovine subject in a sequence of 3D images. There are other variants of instance segmentation neural networks, such as Cascade R-CNN and Mask2Former, which can also be applied here.

[0095] variety As used herein, the bovine subject may preferably be a bovine subject, including both cows and bulls, whether grown or newborn.The breed of the bovine subject may thus be selected from the group consisting of Jersey, Holstein, Holstein Friesian cattle population, Holstein Swartbond cattle population, German Holstein Schwarzbund cattle population, US Holstein cattle population, red and white spotted Holstein, German Holstein Schwarzbund cattle population, Red Danish population, Finnish Ayrshire population, Swedish Red and White population, Danish Holstein population, Swedish Red and White population and Nordic Red population.

[0096] (Example) Example 1 Data from over 1000 dairy cows from five different herds was acquired over a period of time in the form of 3D images, with each cow being imaged multiple times during that time period, as shown by the ratio between the number of cows and the number of images in Table 1 below.

[0097] [Table 1]

[0098] After each milking, a 3D image of the cow's back was acquired (Figure 1). The highest point on the cow's back is the spine. 100 points are placed evenly along the spine. The 90 points towards the end of the cow's back are used to generate 9 slopes. The slopes are from points 1 to 10 (slope 1), 11 to 20 (slope 2), 21 to 30 (slope 3), 31 to 40 (slope 4), 41 to 50 (slope 5), 51 to 60 (slope 6), 61 to 70 (slope 7), 71 to 80 (slope 8) and 81 to 90 (slope 9). Negative slopes indicate a decrease from the start to the end of the 10 points, while positive ones indicate an increase from the start to the end.

[0099] The more extreme one or more of the slopes are compared to normal, and the greater the number of extreme slopes, the higher the probability that the cow is lame.

[0100] [Table 2]

[0101] In Table 2, the average slope and standard deviation for each slope are presented for the five herds. In general, slopes 1 to 3 are negative, slopes 4 and 5 are positive, and slopes 6 to 9 are negative again. This indicates that the cows generally have curved backs. The standard deviation is smallest for slopes 4 and 5, indicating that the cows have a more uniform curve in the mid section of their backs compared to the rear and front of their backs. Also, the large standard deviation indicates that there is generally a very large difference between the cows in terms of back curvature.

[0102] [Table 3]

[0103] Table 3 shows the correlations between the nine slopes. The correlations are always high between adjacent slopes. Also, the correlations between slopes 1 through 3 and slopes 7 through 9 are actually negative indicating that the curvature of the front of the cow is often unrelated to the curvature of the rear end of the cow, and is often negatively related.

[0104] [Table 4]

[0105] Table 4 shows the cow subject variability, residual variability, and total variability on a scale of 1 to 9, as well as the repeatability, defined as the cow subject variability divided by the total variability. A repeatability close to 0 indicates no relationship between the two measures for the same cow, and a repeatability of 1 indicates that the exact same thing is measured every time a cow is enrolled. In this example, a repeatability between 0.42 and 0.51 is observed. This indicates that there is substantial information associated with the particular cow being enrolled, and that this can be used to predict lameness.

[0106] The objective but rough lameness threshold is the slope of the posterior part of the spine, i.e. the slope of the spine from the tail to the middle of the spine. Based on the picture in Figure 1, the slope between the height of the spine (point 1) and the height at the middle of the spine (point 50) was calculated for all cows. The lameness threshold was defined to be 30 for this slope, which corresponds to a 7.5 cm height difference of the spine over the posterior part of the back. This is summarised in Table 5 below.

[0107] [Table 5]

[0108] That is, about 1% of cows have a steep, posterior spine and can be considered lame. This is an example of a global reference threshold for objectively detecting lameness in bovine subjects. However, it is also an example of a very rough threshold that discards most of the information of the back topology that is available for a particular cow.

[0109] Example 2 A sequence of range images from over 1000 dairy cows from five different herds is acquired over a period of time. A sequence of 3D images of each cow walking in or through the aisles can then be generated. The herd is still manually supervised as usual, and some of the cows imaged are eventually observed to be lame. This information can be fed into the data for the creation of the trained model. Modeling the acquired image data with the trained model may reveal additional cows suspected of lameness, which can be further investigated, i.e., the trained model can be continually improved by constantly feeding the model with new verified lameness data. At some point, the model is good enough to be trained on very early data, i.e., data acquired months before lameness is detected in the cow. In that way, the disclosed techniques can be used for early prediction of lameness.

[0110] References Christen A.-M., C. Egger-Danner, N. Capion, N. Charfeddine, J. Cole, G. Cramer, A. Fiedler, T. Fjeldaas, N. Gengler, M. Haskell, B. Heringstad, M. Holzhauer, G. de Jong, A. Koeck, J. Kofler, K. Mueller, J. Pryce, Å. M. Sogstad, KF Stock, G. Thomas, E. Vasseur,2021,"Recording Lameness in Dairy Cattle",2021 ICAR meeting

[0111] item 1. A method for detecting lameness in a bovine subject, comprising: - acquiring at least one 3D image of the back of a bovine subject; - extracting data representative of the spine of a bovine subject; - determining at least one curvature of at least one preselected portion of the spine; and - detecting lameness and / or determining the degree of lameness in a bovine subject by correlating said curvature with at least one reference value, such as at least one predefined threshold value; and / or - extracting data relating to the topology of the back of the bovine subject from said at least one 3D image in the form of contour points, - detecting lameness and / or determining the extent of lameness in a bovine subject by correlating said contour points against a reference model.

[0112] 2. The method according to item 1, wherein the extracted data comprises at least 50 data points, preferably at least 100 data points.

[0113] 3. The method according to item 1 or 2, wherein the reference model comprises information on the topology of the back and the corresponding degree of lameness, e.g. in relation to the breed of the bovine subject.

[0114] 4. The method according to any one of items 1 to 3, wherein the reference value and / or reference model is specific to the bovine subject.

[0115] 5. The method according to any one of items 1 to 4, wherein the reference value and / or reference model is specific to the breed of the bovine subject.

[0116] 6. The method according to any one of items 1 to 5, wherein the reference values ​​and / or reference models are specific to the bovine subject and are generated from a previous 3D image of the back of the bovine subject.

[0117] 7. The method of any one of the preceding items 6, wherein the previous 3D images of the bovine subject are acquired over a time period of at least 2 days, more preferably at least 5 days, even more preferably at least 1 week, even more preferably at least 2 weeks, and most preferably at least 4 weeks.

[0118] 8. The method according to any one of items 1 to 7, wherein the curvature of the preselected portion of the spine is defined solely as the vertical curvature of said preselected portion of the spine.

[0119] 9. The method of any one of items 1 to 8, wherein the curvature of a preselected portion of the spine is determined by calculating the inclination of said preselected portion of the spine.

[0120] 10. The method of any one of items 1 to 9, wherein the at least one preselected portion of the spine is a posterior portion of the spine (towards the tail of the bovine subject).

[0121] 11. The method of any one of items 1 to 10, wherein at least a portion of the spine is divided into subsections and the curvature of each subsection is determined.

[0122] 12. The method of any one of the preceding items 11, wherein at least a portion of the spine is divided into at least 2, 3, 5, 7, or at least 10 subsections.

[0123] 13. The method according to item 11 or 12, wherein a reference value is given for each subsection.

[0124] 14. The method according to any one of items 1 to 13, wherein the detection of lameness is correlated with a predefined difference between the curvature and a corresponding reference value.

[0125] 15. The method according to any one of items 1 to 14, wherein the degree of lameness is correlated with the difference between the curvature and a corresponding reference value.

[0126] 16. The method according to any one of items 1 to 15, wherein the contour points are extracted relative to a line of maxima along the back of the animal.

[0127] 17. The method according to any one of items 1 to 16, wherein the spine is defined as the maximal breakthrough line.

[0128] 18. The method according to any one of items 1 to 17, comprising identifying a bovine subject based on the at least one 3D image.

[0129] 19. The method according to any one of items 1 to 18, comprising identifying the bovine subject based on RFID technology.

[0130] 20. The method according to any one of items 1 to 19, wherein the bovine subject is in motion during acquisition of the at least one 3D image.

[0131] 21. The method of any one of items 1 to 20, wherein the at least one 3D image is based on a plurality of 3D images acquired while the bovine animal is in motion.

[0132] 22. The method according to any one of items 1 to 21, wherein a plurality of 3D images are acquired of the bovine subject.

[0133] 23. The method according to any one of items 1 to 22, wherein the at least one 3D image is based on a plurality of 3D images acquired while the bovine subject is in motion.

[0134] 24. The method according to any one of items 1 to 23, wherein the data analysis is based on a median image of two or more images.

[0135] 25. The method according to any one of items 1 to 24, wherein the bovine subjects are either adult or newborn animals, including both cows and bulls.

[0136] 26. The method of any one of items 1 to 25, wherein the threshold is defined based on the breed of the bovine subject, the breed being selected from the group consisting of Jersey, Friesian cattle population, Holstein Swartbond cattle population, German Holstein Schwarzbund cattle population, US Holstein cattle population, Red and White Holstein, German Holstein Schwarzbund cattle population, Red Danish population, Finnish Ayrshire population, Swedish Red and White population, Danish Holstein population, Swedish Red and White population and Nordic Red population.

[0137] 27. A system for detecting lameness in a bovine subject, comprising: - an imaging system configured to acquire at least one 3D image of the back of a bovine subject; a processing unit, the processing unit comprising: - acquiring at least one 3D image of the back of a bovine subject; - extracting data representative of the spine of a bovine subject; - determining at least one curvature of at least one preselected portion of the spine; - detecting lameness in a bovine subject by correlating said curvature to at least one predefined threshold value; A system configured to:

[0138] 28. The system according to item 27, configured to acquire the at least one 3D image while a bovine subject is standing and / or walking within the lock.

[0139] 29. The system of item 27 or 28, configured such that the at least one 3D image is acquired from above the bovine subject, thereby imaging the bovine subject from an upward perspective.

[0140] 30. The system of any one of items 27 to 29, configured to acquire the at least one 3D image upon triggering by the bovine subject approaching and / or entering a lock.

[0141] 31. The system of any one of items 27 to 30, wherein the processing unit is configured to perform the method of any one of items 1 to 26. [Explanation of symbols]

[0142] 1 Thick line 2,3,5,6 Animals, bovine subjects 4. 3D Camera 20. Oval 21 Dashed line, vertebral axis 22a, 22b, 22c, 22d points 23a,23b,23c,23d line, right angle axis

Claims

1. 1. A method for detecting lameness in a bovine subject, comprising: - acquiring a plurality of range images of a back of at least one bovine subject, said range images being acquired from above said at least one bovine subject; - image processing said range images to generate at least one sequence of 3D images of a pre-selected bovine subject; - detecting lameness and / or determining the degree of lameness in said pre-selected bovine subject by modeling at least one sequence of said 3D images against a trained reference model using artificial intelligence (AI); The method includes:

2. The method of claim 1 , wherein the plurality of range images are a time series of range images and / or the sequence of 3D images of the pre-selected bovine subject are a time series of 3D images.

3. 3. The method of claim 1 or 2, comprising labelling the preselected bovine subjects as "no lameness", "potential lameness" or "moderate lameness" after modelling, or as "lameness" or "severe lameness".

4. 4. The method of claim 1, further comprising the step of labelling the preselected bovine subject as "lame" after detection of lameness, with the degree of lameness as an integer between 1 and 5, thereby determining the degree of lameness.

5. 5. The method according to any one of claims 1 to 4, wherein the range images are acquired at a frame rate of at least 20 frames per second (20 fps), preferably at least 25 fps.

6. The method of claim 1 , wherein the sequence of 3D images comprises a sequence of 3D images of the pre-selected bovine subject moving forward.

7. 7. The method of claim 1, wherein the sequence of 3D images comprises a sequence of 3D images of the pre-selected bovine subject advancing a minimum distance of at least 3 meters.

8. 8. The method of any one of claims 1 to 7, wherein said sequence of 3D images comprises a sequence of 3D images of said preselected bovine subject for a minimum period of at least 3 seconds, preferably at least 5 seconds.

9. The method of claim 1 , wherein acquiring the range image comprises acquiring infrared (IR) 2D image data.

10. 10. The method of claim 1, wherein the image processing comprises generating at least one sequence of 2D IR images of the back of the preselected bovine subject corresponding to at least one sequence of 3D images.

11. 11. The method of any one of claims 1 to 10, wherein the range images are acquired from above an aisle through which a plurality of bovine subjects are passing, such as an aisle that is passed through at least once a day.

12. 12. The method of claim 11, further comprising identifying the preselected bovine subject with at least one ID reader located at one or both ends of the corridor.

13. 13. The method of claim 1, wherein the image processing comprises background removal to isolate the bovine subject from its surroundings in the range image.

14. The method according to claim 1 , wherein the image processing comprises a viewpoint correction of the range image and / or the 3D image.

15. 15. The method of claim 1, wherein the sequence of 3D images of the pre-selected bovine subject corresponds to a temporal (windowed) selection of the plurality of range images.

16. 16. The method of any one of claims 1 to 15, wherein the sequence of 3D images of the preselected bovine subject corresponds to a temporal (windowed) selection of the plurality of range images selected such that only the preselected bovine subject is present in the sequence of 3D images.

17. 17. The method of any one of claims 1 to 16, wherein the image processing comprises the step of segmenting at least a portion of the acquired range images using instance segmentation to select a bovine subject among the segmented instances and generate at least one sequence of 3D images of the pre-selected bovine subject.

18. The method of claim 17 , wherein the instance segmentation is provided using Mask-RCNN.

19. 19. The method of claim 17 or 18, wherein the segmented bovine instance is located in close proximity to an ID reader and a corresponding ID read result is associated with the identity of the preselected bovine subject.

20. 20. The method of claim 1, comprising generating a point cloud of the pre-selected bovine subject based on the range images and the image processing.

21. 21. The method of any one of claims 1 to 20, wherein said at least one sequence of 3D images comprises coordinates of a cloud of points representing said preselected bovine subject in images.

22. 22. The method of claim 20 or 21, comprising adding depth data to the point cloud data and viewpoint correcting the point cloud data based on range and / or depth data in the range images.

23. 23. The method of any one of claims 1 to 22, wherein the sequence of 3D images used as input to the AI ​​model comprises 3D point cloud data representing the movement of the pre-selected bovine subject in the sequence.

24. 24. The method of claim 23, wherein the sequence of 3D images used as input to the AI ​​model further includes 2D IR data of the pre-selected bovine subject.

25. 25. The method according to any one of claims 1 to 24, wherein the reference model comprises information on the topology of the bovine subject's back and the relative degree of lameness, for example for the preselected breed of bovine subject.

26. 26. The method of claim 1, wherein the trained reference model is trained with degree of lameness as an output result.

27. 27. The method of any one of claims 1 to 26, wherein the trained model is trained using a supervised learning technique and the AI ​​model is trained using labeled data, the labeled data being obtained from a trained veterinarian assessing the degree of lameness in a bovine subject assessed by the trained veterinarian to be lame.

28. 28. The method of claim 1, wherein the trained reference model is trained to include spatial correlation of the lameness in the sequence of 3D images and temporal correlation of the lameness in the sequence of 3D images.

29. 29. The method of any one of claims 1 to 28, wherein the trained reference model is trained using a combination of self-supervised learning and semi-supervised learning, where a self-supervised learning technique is provided to pre-train the AI ​​model on unlabeled data, and then the pre-trained AI model is fine-tuned on a relatively small set of labeled data using semi-supervised learning.

30. 30. The method of any one of claims 1 to 29, wherein the trained reference model is trained with detection of lameness as an output result, and training data is obtained for a time period before the lameness is detected as an output result and manually verified by an expert.

31. 31. The method of claim 30, wherein the time period is at least 1 month, or at least 2 months, or at least 3 months, or at least 4 months.

32. 32. The method of claim 1, wherein the trained reference model is trained using machine learning, a neural network, a recurrent neural network, a convolutional neural network, or any combination thereof.

33. 33. The method of any one of claims 1 to 32, wherein the AI ​​model is selected from the group consisting of machine learning, neural networks, recurrent neural networks, convolutional neural networks, or any combination thereof.

34. 34. The method of any one of claims 1 to 33, wherein the trained reference model is specific to the preselected bovine subject.

35. 35. The method of any one of claims 1 to 34, wherein the modeling includes both spatial and temporal correlation of lameness in the sequence of 3D images with the trained reference model.

36. 36. The method of claim 35, wherein the sequence prediction model is multi-scale 3D Resnet or ViViT.

37. 37. The method of any one of claims 1 to 36, wherein the reference value and / or the reference model is specific to the breed of the bovine subject.

38. 38. The method of any one of claims 1 to 37, comprising identifying the bovine subject based on at least one 3D image.

39. 39. The method of any one of claims 1 to 38, comprising identifying the bovine subject based on RFID technology.

40. 40. The method of any one of claims 1 to 39, wherein the breed of the bovine subject is selected from the group consisting of Jersey, Friesian cattle population, Holstein Swartbond cattle population, German Holstein Schwarzbund cattle population, US Holstein cattle population, Red and White Holstein, German Holstein Schwarzbund cattle population, Red Danish population, Finnish Ayrshire population, Swedish Red and White population, Danish Holstein population, Swedish Red and White population and Nordic Red population.

41. 1. A system for detecting lameness in a bovine subject, comprising: - an imaging system configured to acquire a plurality of range images of a back of a bovine subject; - a processing unit configured to carry out the method according to any one of claims 1 to 40; A system comprising:

42. 42. The system of claim 41, configured to acquire the range images while the bovine subject is standing and / or walking within a lock and / or passageway.

43. 43. The system of claim 41 or 42, configured such that the range images are acquired from above the bovine subject, thereby imaging the bovine subject from an upward perspective.

44. 44. The system of any one of claims 41 to 43, configured to initiate acquisition of said range images when triggered by at least one bovine subject approaching and / or entering a lock.

45. 45. The system of any one of claims 41 to 44, wherein the imaging system is configured to acquire the range images at a frame rate of at least 15 frames per second, preferably at least 20 frames per second, more preferably at least 25 frames per second.

46. 46. ​​The system of any one of claims 41 to 45, comprising one or more ID readers for reading identification numbers of bovine subjects entering and / or exiting the locks and / or passageways.

47. 47. The system of any one of claims 41 to 46, wherein the imaging system comprises at least one RGB camera and a depth sensor, such as an infrared sensor for providing a 2D infrared image.

Images