An apparatus for aiding in evaluating a gait of a quadruped

Abstract

An apparatus (100) for aiding in evaluating the gait of a quadruped (2) is disclosed. The apparatus comprises an image capturing device (4) for capturing a video segment (6) of a quadruped in motion, said video segment represents a series of time separated images (8), wherein said video segment is having a duration which spans at least one gait cycle of the quadruped; and a data processing device (10). The data processing device (10) is being configured, from said series of images (8) of said video segment (6), to extract motion data (12) indicative of one or more predetermined biometric characteristics of the gait cycle(s) of said quadruped, and the data processing device (10) is being configured, from said motion data (12), to determine a gait rhythm of said gait cycle(s). Moreover, the data processing device (10) is being configured to generate an audio segment (14) comprising a plurality of audio signal pulses (16) corresponding to said gait rhythm of said gait cycle(s). Finally, the data processing device (10) is being configured to mix said audio segment (14) with said video segment (6), thereby generating a combined audio / video segment (18). The apparatus of the invention is suitable for training a human being in recognizing a lameness of a quadruped by watching a gait of that quadruped, while at the same time listening to the audio signal pulses (16) generated by the data processing device.

Description

[0001] An apparatus for aiding in evaluating a gait of a quadruped

[0002] Field of the invention

[0003] The present invention relates to the field of veterinary science and in particular to technology for evaluating the gait of a quadruped.

[0004] Accordingly, in a first aspect the present invention relates to an apparatus for evaluating the gait of a quadruped.

[0005] In a second aspect the present invention relates to a use of the apparatus of the first aspect of the present invention as an aid in evaluating the gait of a quadruped.

[0006] In a third aspect the present invention relates to a method for evaluating the gait of a quadruped.

[0007] In a fourth aspect the present invention relates to a computer program product.

[0008] Background of the invention

[0009] Lameness is a relatively frequently occurring physical disorder in quadrupeds such as horses, dogs, cats, cattle and swine. In case a condition of lameness is not detected at an early stage there is a risk that the lameness will worsen over time. A worsening of a lameness condition in a quadruped will increase medical expenses and additionally often also add emotional pain to the owner of the animal.

[0010] Within the field of veterinary science lameness in quadrupeds has traditionally been diagnosed by a veterinarian by watching the gait of the walking or running animal and paying special attention to any asymmetry in the movements performed by the animal. In case an asymmetry can be detected, the animal is likely lame, whereas the opposite is true in case no asymmetry can be detected.

[0011] However, human vision is limited when it comes to detecting asymmetries in cyclic movements having a frequency in the range of the frequency of foot beats of a walking or running quadruped.

[0012] In recent years, more technical solutions have been developed which aids in detection of lameness in a quadruped.

[0013] One of these technologies involves providing the quadruped with sensors of a sensor system at various positions. Such sensor systems are capable of sensing various biometric parameters of the animal during movement. On the basis of such sensed biometric parameters a data processor will process the data obtained in order to diagnose a potential presence of a lameness in the quadruped. Other technologies relate to image capture and image analysis of a quadruped in motion. Accordingly, these technologies make use of computer power to analyze, from a captured video of the moving quadruped, the movement pattern of specific positions or anatomic elements of the animal and on the basis of this analysis, assess whether the movement of the animal is symmetric (no lameness being present) or is asymmetric (potential lameness being present).

[0014] Some of these latter technologies utilizing image analysis are even available for smartphones or tablets as an app which the owner of the quadruped can download and install and use without the necessity of consulting a veterinarian. Thereby the owner of the animal may film the gait of the animal and after image analysis the app will tell whether or not and optionally with which severity the animal is lame.

[0015] Such a technology is known from US 2015 / 0080765 Al, which describes a system that utilizes computerized depth perception to auto-matically scan physical objects, e.g. a quadruped such as a sport horse, over time thereby enabling a detailed analysis of its movement, and changes thereof over time, without the need for attaching sensors to the body of the horse, or requiring a force plate or high speed expensive cameras is disclosed herein.

[0016] Another known system is known from US 2016 / 0073614 Al, which method of diagnosing lameness in quadrupeds utilizing computer vision and a computerized depth perception system to scan quadrupeds, such as sport horses, over time. The method enables a detailed analysis of the quadruped's movement, and changes thereof over time without the need for attaching sensors to the body of the horse or requiring force plates or expensive high-speed cameras. A processing system receives the input of the movement data and utilizes it to determination the severity of lameness signals of the animal.

[0017] However, a non-professional person, i.e. a non-veterinarian person may not fully comprehend how such diagnosis of such image analysis systems is arrived at, and because of this, a nonveterinarian person may not be fully confident that the analysis result, or diagnosis, provided with such systems, really is correct and is based on objective facts.

[0018] Moreover, just being presented with a diagnostic result without being mentally involved in the process leading to such diagnostic result, will deprive a non-veterinarian person from being trained in assessing presence of lameness in a quadruped solely based on viewing the gait of that quadruped.

[0019] Accordingly, a need persists for technology which will aid in training a non-veterinarian in assessing presence of lameness in a quadruped based on viewing the gait of the quadruped.

[0020] Brief description of the invention

[0021] This objective is fulfilled according to the present invention in its various aspects. Accordingly, the present invention relates in a first aspect to an apparatus for aiding in evaluating the gait of a quadruped, wherein said apparatus comprises:

[0022] -an image capturing device for capturing a video segment of a quadruped in motion, said video segment represents a series of time separated images, wherein said video segment is having a duration which spans at least one gait cycle of the quadruped;

[0023] -a data processing device; wherein said data processing device is being configured, from said series of images of said video segment, to extract motion data indicative of one or more predetermined biometric characteristics of the gait cycle(s) of said quadruped; wherein said data processing device is being configured, from said motion data, to determine a gait rhythm of said gait cycle(s); wherein said data processing device is being configured to generate an audio segment comprising a plurality of audio signal pulses corresponding to said gait rhythm of said gait cycle(s); wherein said data processing device is being configured to mix said audio segment with said video segment, thereby generating a combined audio / video segment.

[0024] In a second aspect the present invention provides a use of an apparatus according to the first aspect of the invention as an aid in evaluating the gait of a quadruped.

[0025] In a third aspect the present invention provides a method for evaluating the gait of a quadruped, wherein said method comprises: i) capturing a video segment of a quadruped in motion, wherein said video segment comprises a series of time separated images, wherein said video segment is having a duration which spans at least one gait cycle of the quadruped; ii) from said video segment obtained in step i), extracting motion data indicative of one or more predetermined biometric characteristics of the gait cycle of said quadruped; iii) from said motion data obtained in step ii), determining a gait rhythm of said gait cycle; iv) generating an audio segment comprising a plurality of audio signal pulses corresponding to said gait rhythm of said gait cycle, as determined in step iii); v) mixing said audio segment obtained in step iv) with said video segment obtained in step i); thereby generating a combined audio / video segment; vi) visually and audible reproducing said combined audio / video segment obtained in step v).

[0026] In a fourth aspect the present invention relates to a computer program product which, when being loaded on a computer, is being configured to perform steps ii) - vi) of the method of the third aspect of the present invention. The present invention in its various aspects provides an aid in evaluating the gait of a quadruped, for example the symmetry of a gait of a quadruped, such as a horse, a dog, a cat, cattle or swine.

[0027] This aid relies on the features of the present invention which allow watching a video playback of one or more gait cycles of a quadruped while at the same time listening to an automatically generated audio representation of the rhythm of the gait cycle.

[0028] That is to say, it has been found that the human auditory system is rather excellent in perceiving and recognizing rhythmic and arrhythmic sound patterns comprising audio pulses lying within the frequency of steps of walking or running quadrupeds, such as horses, dogs, cats, cattle or swine.

[0029] Thereby, watching a video playback of one or more gait cycles of a quadruped, or watching such video real-time or near real-time, or in augmented reality, while at the same time listening to an audio representation of the rhythm of the gait cycle, will teach and train a human being, professional veterinarian or not, to identify symmetry or lack of symmetry in a gait cycle of that quadruped.

[0030] Brief description of the figures

[0031] Fig. la and lb are diagrams which illustrate symmetry differences in a quadruped’s movement depending on whether the quadruped is lame or not lame.

[0032] Fig. 2 is a diagram illustrating a representation of an embodiment of an apparatus according to the first aspect of the present invention.

[0033] Fig. 3 is a diagram illustrating the process of converting a video segment as captured by an image capturing device into a combined audio / video segment according to the present invention.

[0034] Fig. 4 is a diagram illustrating the working principle of the present invention.

[0035] Detailed description of the invention

[0036] The first aspect of the present invention

[0037] In a first aspect the present invention relates to an apparatus 100 for aiding in evaluating the gait of a quadruped 2, wherein said apparatus comprises:

[0038] -an image capturing device 4 for capturing a video segment 6 of a quadruped in motion, said video segment represents a series of time separated images 8, wherein said video segment is having a duration which spans at least one gait cycle of the quadruped;

[0039] -a data processing device 10; wherein said data processing device 10 is being configured, from said series of images 8 of said video segment 6, to extract motion data 12 indicative of one or more predetermined biometric characteristics of the gait cycle(s) of said quadruped; wherein said data processing device 10 is being configured, from said motion data 12, to determine a gait rhythm of said gait cycle(s); wherein said data processing device 10 is being configured to generate an audio segment 14 comprising a plurality of audio signal pulses 16 corresponding to said gait rhythm of said gait cycle(s); wherein said data processing device 10 is being configured to mix said audio segment 14 with said video segment 6, thereby generating a combined audio / video segment 18.

[0040] Accordingly, in the first aspect of the present invention an apparatus is provided which is able to capture a video segment of a quadruped performing one or more gait cycles. The apparatus is furthermore configured to analyze the captured video segment so as to determine a gait rhythm of the gait cycles(s) and finally the apparatus is configured to generate an audio segment comprising a plurality of audio signal pulses and mix this audio segment with the captured video segment so as to obtain a combined audio / video segment comprising these audio signal pulses.

[0041] As the audio signal pulses are corresponding to the gait rhythm of the quadruped, a human being, when watching and listening to the combined audio / video segment, will be trained in visually identifying any lameness of the quadruped by listening to the audio signal pulses of the combined audio video segment, while at the same time watching the video segment of the combined audio / video segment.

[0042] In the present description and in the appended claims the term “gait cycle” shall be construed to mean one cycle of movement of the four legs during the movement of the quadruped.

[0043] In the present description and in the appended claims the term “gait rhythm of a gait cycle” may be construed to mean one or more recurring movements of one or more anatomic elements or areas of the quadruped within a gait cycle. In this way, a front leg’s impact with the ground defines a gait rhythm of two “beats per gait cycle”, whereas a leg’s impact with the ground defines a gait rhythm of four “beats per gait cycle”.

[0044] In the present description and in the appended claims the term “motion data” may be construed to mean data defining the movement of one or more anatomic elements or areas of the quadruped.

[0045] In the present description and in the appended claims the term “audio signal pulses” shall be construed to mean audio signals having at least portions or transients which are separated in time. In one embodiment of the apparatus 100 according to the first aspect of the present invention the apparatus further comprises a display 20 and an audio reproducing device 22, such as loudspeaker or a headphone, for visual and audible reproducing said combined audio / video segment 18.

[0046] This embodiment allows for using the apparatus 100 itself for visual and audible reproducing the combined audio / video segment 18.

[0047] In one embodiment of the apparatus 100 according to the first aspect of the present invention the apparatus is being configured for visually and audibly reproducing said combined audio / video segment 18 in real-time or in near real-time, relative to the time of capturing said video segment 6 by said image capturing device 4.

[0048] Hereby, training of a human being in identifying lameness of the quadruped can be performed in a “live”, non-playback situation.

[0049] In the present description and in the appended claims, the term “near real-time” shall be construed to mean real time with the exception of the necessary time lag due to the data processing being performed. Such lag may be in the order of 200 ms (milliseconds) or less, such as 100 ms or less, for example 50 ms or less, e.g. 25 ms or less, 10 ms or less, such as 5 ms or less; or 1 ms or less.

[0050] In one embodiment of the apparatus 100 according to the first aspect of the present invention the apparatus comprises a data storage 24 configured for storing said video segment 6 captured by said image capturing device 4 and / or configured for storing said combined audio / video segment 18 generated by said data processing device 10.

[0051] Providing the apparatus with a data storage enables replay at a later stage of the combined audio / video segment 18 which has been stored on the data storage 24.

[0052] In some embodiments of the apparatus according to the first aspect of the present invention the apparatus is configured to allow playback of the combined audio / video segment 18 at a slower frame rate, compared to the frame rate at which the video segment 6 was originally captured by the image capturing device 4.

[0053] In one embodiment of the apparatus 100 according to the first aspect of the present invention the audio segment 14 generated by said data processing device 10 comprises a silent background layer and an audio layer comprising said individual audio signal pulses 16; or alternatively, the audio segment 14 generated by said data processing device 10 comprises a non-silent background layer and an added audio layer comprising said audio segment 14 comprising said individual audio signal pulses 16.

[0054] Some of the audio segments may comprise a continuous audio background. What is important is that some parts of the audio segment are separated in time to the extent that these time separated parts are audible as audio signal pulses upon being played back.

[0055] In one embodiment of the apparatus 100 according to the first aspect of the present invention the audio signal pulses 16 of said audio segment 14 of the combined video / audio segment 18 independently are having a duration of 5 - 1000 ms (milliseconds), such as 10 - 900 ms, for example 15 - 800 ms, e.g. 20 - 750 ms, such as 25 - 700 ms, for example 50 - 650 ms, such as 75 - 600 ms, such as 100 - 550 ms, for example 150 - 500 ms, e.g. 200 - 450 ms, such as 250 - 400 ms or 300 - 350 ms.

[0056] These durations serve the intended purpose of enabling a human being in recognizing a beat pattern of the audio segment.

[0057] In one embodiment of the apparatus 100 according to the first aspect of the present invention the motion data 12, indicative of said predetermined biometric characteristics of the gait cycle, relates to one or more of the following: vertical displacement of the trunk, such as measured by the parameter MinDiff and / or MaxDiff; limb position, velocity, flight pattern, deceleration and / or acceleration of one or more limbs or anatomic elements of the quadruped.

[0058] These biometric characteristics are well-suited as a basis for analyzing a gait cycle of a quadruped.

[0059] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured to determine said gait rhythm in a form comprising two, three or four “beats” per gait cycle, wherein said “beat” optionally represents cyclic movements of one or more anatomic elements of the quadruped 2 during its movement.

[0060] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured to generate said audio segment 14 in such a way that it comprises a beat pattern comprising two, three or four of said audio pulses per gait cycle in correspondence with said gait rhythm.

[0061] Such two, three or four “beats” per gait cycle correspond to the rhythm of gait types such as walk, trot, canter, lope, pace, gallop or tbit.

[0062] In one embodiment of the apparatus 100 according to the first aspect of the present invention it is a condition that in the combined audio / video segment 18, the audio segment 14 is being synchronized with said video segment 6 in such a way that said audio signal pulses 16 of said audio segment 14 are being synchronized with a cyclic movement of one more limbs or anatomic elements of the quadruped 2, such as based on each foot’s impact with the ground.

[0063] Hereby it is easier to identify any lameness upon listening to the audio signal pulses.

[0064] In one embodiment of the apparatus 100 according to the first aspect of the present invention the duration of said video segment 6 and the duration of said audio segment 14 are identical, or alternatively the duration of said video segment 6 is longer than the duration of said audio segment 14.

[0065] It is preferred but not necessary that the audio segment 14 is having the same duration as the video segment 6.

[0066] In one embodiment of the apparatus 100 according to the first aspect of the present invention the video segment 6 and said combined video / audio segment 18 span a plurality of gait cycles, such as 2 - 1000 or more gait cycles, for example 5 - 950 gait cycles, such as 10 - 900 gait cycles, for example 15 - 850 gait cycles, such as 20 - 800 gait cycles, e.g. 25 - 750 gait cycles, for example 50 - 700 gait cycles, such as 100 - 650 gait cycles, for example 150 - 600 gait cycles, for example 200 - 550 gait cycles, such as 250 - 500 gait cycles, e.g. 300 - 450 gait cycles or 350 - 400 gait cycles.

[0067] Hereby a relatively long span of time may be allocated for the human being to train his or her ability to recognize a lameness of the quadruped.

[0068] In one embodiment of the apparatus 100 according to the first aspect of the present invention, one or more of said one or more predetermined biometric characteristics of the gait cycle relate to an intensity of movement of one or more of the four legs of the quadruped; and said data processing device 10 is being configured to determine, from said motion data 12 and in respect of one or more specific gait cycles of said video segment 6, preferably in respect of each gait cycle of said video segment 6, a variation in the intensity of movement of the four legs of the quadruped within the same gait cycle, or said data processing device 10 is being configured to determine, from said motion data 12 and in respect of two or more gait cycles of said video segment 6, a variation in the intensity of movement of the four legs of the quadruped within said two or more gait cycles.

[0069] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured to generate said audio segment 14 comprising said plurality of audio signal pulses 16 in such a way that the intensity and / or the frequency and / or timbre and / or another sound expression characteristics of said audio signal pulses 16 will vary commensurate with the variation in the intensity of movement of the four legs of the quadruped within the same gait cycle, or within said two or more gait cycles, as the case may be.

[0070] Hereby the apparatus may even be useful to train a human being in recognizing the severity of a lameness of the quadruped.

[0071] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured to generate said audio segment 14 comprising said plurality of audio signal pulses 16 in such a way that said audio signal pulses 16 is representing the sound of a horse hoof hitting the ground, such as in the form of a recorded sound of a horse hoof hitting the ground, or in the form of an artificially generated sound resembling the sound of a horse hoof hitting the ground; or wherein said data processing device 10 is being configured to generate said audio segment 14 comprising said plurality of audio signal pulses 16 in such a way that said audio signal pulses 16 comprise sounds having an increased frequency or a decreased frequency over the duration of each audio signal pulses 16.

[0072] Such types of sounds have been found useful in the training process of a human being in recognizing lameness of a quadruped.

[0073] In one embodiment of the apparatus 100 according to the first aspect of the present invention the frame rate of said video segment 6 and / or of said combined video / audio segment 18 is selected from the ranges of 1 - 1,000 fps (frames per second), such as 12 - 900 fps, e.g. 24 - 800 fps, for examples 36 - 600, for example 60 - 500 fps, such as 120 - 400 fps or 240 - 300 fps.

[0074] These frame rates have proved sufficiently high for performing a proper image analysis of a moving quadruped.

[0075] It is preferred that the frame rate of the combined audio / video segment 18 is the same as the frame rate of the video segment 6.

[0076] In some alternative embodiments it is preferred to configure the data processing device 10 to perform a subdivision of the video segment 6 into a subdivided video segment comprising more images 8 than the original video segment 6, such as by interpolation of pairs of two consecutive images of the originally captured video segment 6; and optionally additionally configure the data processing device 10 to extract the motion data 12 from this subdivided video segment; and configure the data processing device 10 to subsequently employ this subdivided video segment as the basis for generating the combined audio / video segment 18.

[0077] In one embodiment of the apparatus 100 according to the first aspect of the present invention the apparatus is configured for evaluating the gait of a horse, a dog, a cat, cattle or swine.

[0078] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 furthermore is being configured, from said motion data 12, to extract one or more gait movement parameters 26 which may be indicative of lameness of the quadruped 2.

[0079] Hereby, when a human being is using the apparatus of the first aspect of the present invention to train his or her ability to recognize a lameness or movement asymmetry of the quadruped, the human being may additionally be provided with an objective assessment, made by that apparatus, and relating the presence and / or severity of a lameness or movement asymmetry of the quadruped.

[0080] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured to extract one or more of said one or more gait movement parameters 26 dynamically over the span of said video segment 6.

[0081] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured to add a visual representation of one or more of said gait movement parameters in such a way that said combined audio / video segment comprises a visual representation 28 of one or more of said one or more gait movement parameters 26.

[0082] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured, based on said series of time separated images 8 of said video segment 6 to analyze said images and to identify key features / key points thereof, relating to specific anatomic elements of the quadruped, and wherein said data processing device 10 is being configured to determine, when going from one image 8 to the next of said video segment 6, in which way such key features / key points move in relation the quadruped during its movement. Accordingly, such identification of key features / key points form the basis of the analysis made on the images 8 of the video segment 6.

[0083] In one embodiment of the apparatus 100 according to the first aspect of the present invention the key features / key points relate to one or more of the following anatomic elements: head, neck, withers, croup, pelvis, back, front leg(s), such as elbow, carpus, front fetlock, front foot / hoof of a front leg; back legs, such as stifle, hock, hind fetlock, hind foot / hoof of a back leg.

[0084] These anatomic elements have proven very suitable for the intended purpose of determining motion data indicative of one or more predetermined biometric characteristics of the gait cycle(s) of said quadruped.

[0085] In one embodiment of the apparatus 100 according to the first aspect of the present invention the analysis being performed by said data processing device 10 is being based on patch features, such as colour distinction, edge detection, morphology, entropy of image patches making up such images 8.

[0086] In one embodiment of the apparatus 100 according to the first aspect of the present invention the apparatus comprises a data storage 24, wherein said data storage comprises a representation of learning images 30, and wherein said data processing device 10 is configured to perform said analysis by consulting said representation of learning images 30, wherein one or more of said learning images of said representation of learning images comprises a real image or an artificially generated image, such as a computer generated image, of said species of quadruped 2 or a part of said species of quadruped at different postures; and wherein one or more of said images of said representation of learning images 30 has / have been allocated with information correlating said image with information relating to anatomical details depicted by said image.

[0087] In one embodiment of the apparatus 100 according to the first aspect of the present invention the data processing device 10 is being configured for employing deep learning algorithms and / or artificial intelligence and / or neural network in analyzing and in identifying said key features / key points of the series of images of said video segment 6.

[0088] In one embodiment of the apparatus 100 according to the first aspect of the present invention the apparatus is being in the form of a mobile device, such as a smartphone or a tablet or the like; or is in the form of an augmented reality device, such as augmented reality glasses or goggles.

[0089] In one embodiment of the apparatus 100 according to the first aspect of the present invention the apparatus 100 further comprises a separate receiving device for receiving electromagnetic radiation, wherein said mobile device 100 furthermore comprises transmitting means for relaying, by transmission of electromagnetic radiation, signals, representing said audio segment 14 to said receiving device.

[0090] This embodiment allows that the separate receiving device can be carried by a rider who is riding the quadruped which is in the form of a horse. Hereby the rider may receive and hear the audio signal pulses 16 of the horse when riding that horse in order to train the rider in recognizing movement characteristics, movement symmetry or asymmetry or lameness of the horse while riding that horse.

[0091] In preferred embodiments of the apparatus 100 according to the first aspect of the present invention, the apparatus 100 further comprises an interface, such as a graphical user interface (GUI) for enabling a user to provide instructions to said apparatus relating to the operation of that apparatus.

[0092] The second aspect of the present invention

[0093] In a second aspect the present invention provides a use of an apparatus 100 according to the first aspect of the present invention in evaluating the gait of a quadruped 2.

[0094] The third aspect of the present invention

[0095] In a third aspect the present invention provides a method for evaluating the gait of a quadruped 2, wherein said method comprises: i) capturing a video segment 6 of a quadruped in motion, wherein said video segment comprises a series of time separated images 8, wherein said video segment is having a duration which spans at least one gait cycle of the quadruped; ii) from said video segment 6 obtained in step i), extracting motion data 12 indicative of one or more predetermined biometric characteristics of the gait cycle of said quadruped; iii) from said motion data 12 obtained in step ii), determining a gait rhythm of said gait cycle; iv) generating an audio segment 14 comprising a plurality of audio signal pulses 16 corresponding to said gait rhythm of said gait cycle as determined in step iii); v) mixing said audio segment 16 obtained in step iv) with said video segment 6 obtained in step i); thereby generating a combined audio / video segment 18; vi) visually and audible reproducing said combined audio / video segment 18 obtained in step v).

[0096] In one embodiment of the method according to the third aspect of the present invention, the method furthermore comprises the following steps: iia) from said motion data 12 obtained in step i), extracting one or more gait movement parameters 26 which may be indicative of lameness of the quadruped; via) in step vi) visually reproducing one or more of said one or more gait movement parameters 26 obtained in step iia). In one embodiment of the method according to the third aspect of the present invention, the method is performed while the quadruped 2 is performing a movement in a gait type selected from the group comprising: walk, trot, canter, lope, pace, gallop or tbit. In one embodiment of the method according to the third aspect of the present invention, the quadruped is being a horse, a dog, a cat, cattle or swine.

[0097] In one embodiment of the method according to the third aspect of the present invention, the quadruped is a horse, and the horse is moving without a rider or with a rider.

[0098] In one embodiment of the method according to the third aspect of the present invention, the method is performed by using an apparatus 100 according to the first aspect of the present invention.

[0099] The fourth aspect of the present invention

[0100] In a fourth aspect the present invention relates to a computer program product which, when being loaded on a computer, is being configured to perform steps ii) - vi) of the method of the third aspect of the present invention.

[0101] In one embodiment of the fourth aspect, the computer program product is being in the form of an app to a mobile device, such as a smartphone or a tablet, or an augmented reality device.

[0102] In order to better illustrate the present invention, reference is now made to the figures, in which Fig. la and lb are diagrams which illustrate symmetry differences in a quadruped’s movement depending on whether the quadruped is lame or not lame.

[0103] Fig. la and lb show cartesian coordinate systems illustrating vertical movement of the head of a horse during a trot gait cycle.

[0104] The x-axis in Fig. la and lb represents time and the y-axis represents the vertical position of the head of the horse. In both Fig. la and lb the curves are averaged over several observed gait cycles of a horse.

[0105] In Fig. la and lb the terms LF midstance and RF midstance, respectively, refer to a posture of the horse, where the lower part of the left front leg is essentially vertically oriented, or where the lower part of the front right leg is essentially vertically oriented, respectively.

[0106] In Fig. la it is seen that the vertical movement of the head of the horse is symmetrical, irrespective of the position of the foot of the left or right front leg.

[0107] Accordingly, it can be concluded that the horse having a vertical displacement of the head as seen in Fig. la does not have any (detectable) lameness in the front legs.

[0108] In contrast, Fig. lb shows an asymmetry in the vertical movement of the head of the horse.

[0109] It is seen in Fig. lb that when the left foot is in a vertical orientation (LF midstance) the head is not as low as when the right foot is in a vertical orientation (RF midstance).

[0110] Likewise, as seen in Fig. lb, in the posture of the horse at a time midway between the LF midstance and the RF midstance, the head of the horse is having a lower vertical position, compared to the posture of the horse at a time midway between the RF midstance and the LF midstance. This asymmetry is measured by the parameters MinDiff and MaxDiff, respectively as illustrated in Fig. lb.

[0111] The presence of parameter values MinDiff and / or MaxDiff, respectively, which are 0 is an indication that the horse is having an asymmetry or a lameness on one of the front legs. In the present situation corresponding to Fig. lb, the horse is having a lameness on the left front leg. One standard for some veterinarians stipulates that MinDiff must be at least ± 6 mm for the vertical movement of the head in order for the gait to qualify as being representing a lameness on a front leg.

[0112] Fig. 2 is a diagram illustrating a representation of an embodiment of an apparatus according to the first aspect of the present invention.

[0113] Fig. 2 shows an apparatus 100. The apparatus 100 is for aiding in evaluating the gait of a quadruped 2. The apparatus comprises an image capturing device 4, a data processing device 10 and a data storage 24. The image capturing device 4 is for capturing a video segment 6 of a quadruped in motion, where the video segment represents a series of time separated images 8 as illustrated by the dashed square 6,8 which have been stored in the data storage 24. The video segment is having a duration which spans at least one gait cycle of the quadruped.

[0114] The data processing device 10 is configured, from the series of images 8 of the video segment, which has been captured by the image capturing device 4, to extract motion data 12 (denoted by the dashed square 12) indicative of one or more predetermined biometric characteristics of the gait cycle of said quadruped.

[0115] From said motion data 12 the data processing device 10 is configured to determine a gait rhythm of said gait cycle of the quadruped.

[0116] The data processing device 10 is further configured to generate an audio segment 14 comprising a plurality of audio signal pulses 16 corresponding to the gait rhythm of the gait cycle. This is illustrated in Fig. 2 as the dashed square 14,16 within the data processing device 10.

[0117] Finally, the data processing device 10 is configured to mix the audio segment 14 with the video segment 6. Hereby the data processing device generates a combined audio / video segment as illustrated as the dashed square 18 within the data processing device 10.

[0118] It is seen in Fig. 2 that the combined audio / video segment 18 is also stored in the data storage 24. This allows storing a generated combined audio / video segment 18 with the view to perform a play-back of the combined audio / video segment 18 at a later time.

[0119] However, in the embodiment illustrated in Fig. 2 the apparatus 100 also comprises a display 20 and an audio reproducing device 22 in the form of a loudspeaker or a headphone. Hereby visual and audible reproducing the combined audio / video segment 18 can be performed by the apparatus 100 itself. In some embodiments the visually and audibly reproduction of the combined audio / video segment 18 can be performed in real-time or in near real-time, relative to the time of capturing said video segment 6 by the image capturing device 4.

[0120] In the process of analyzing the time separated images 8 of the video segment 6 captured by the image capturing device 4, the data processing device 10 may be configured to identify key features or key points thereof, relating to specific anatomic elements of the quadruped. In this process, the data processing device 10 may determine, when going from one image 8 to the next of the video segment 6, in which way these key features / key points move in relation to the quadruped during its movement.

[0121] Such key features / key points may be based on patch features, such as colour distinction, edge detection, morphology, entropy of image patches making up such images 8.

[0122] In the embodiment illustrated in Fig. 2, the data storage 24 comprises a representation of learning images 30, wherein one or more of the learning images comprises a real image or an artificially generated image, such as a computer-generated image, of a specific species of quadruped 2 or a part of such a quadruped at different postures. The one or more of the images of the representation of learning images 30 may have been allocated with information correlating the image with information relating to anatomical details depicted by said image.

[0123] In this way, the data processing device 10 may be configured for employing deep learning algorithms and / or artificial intelligence and / or neural network in analyzing and in identifying said key features / key points of the series of images of said video segment 6.

[0124] The data processing device 10 of the apparatus of Fig. 2 is also configured to extract one or more gait movement parameters 26 from the motion data 12. Such gait movement parameters 26 are parameters which may be indicative of asymmetry or lameness of the quadruped 2 and indicative of the severity of the asymmetry or lameness of the quadruped. The data processing device 10 may be configured to extract one or more of these one or more gait movement parameters 26 dynamically over the span of said video segment 6.

[0125] Finally, the data processing device 10 is configured to add a visual representation of one or more of said gait movement parameters 26 in such a way that said combined audio / video segment comprises a visual representation 28 of one or more of said one or more gait movement parameters 26.

[0126] As seen in Fig. 2 the one or more gait movement parameters 26 and the representation 28 thereof are being stored on the data storage 24.

[0127] It should be noted that the technology for obtaining gait movement parameters from analyzing motion data originating from a video of a gait of a quadruped is well-known in the art.

[0128] Examples of such systems are provided by the companies Qualisys (Goteborg, Sweden), Simi Reality Motion Systems (Unterschleissheim, Germany) and Sleip (Stockholm, Sweden).

[0129] Reference is also made to the following publication, disclosing such technology: Identifying Lameness in Horses through Deep Learning, SAC ’21 : Proceedings of the 36thAnnual ACM Symposium on Applied Computing, March 2021, pages 976 - 985. Using the apparatus illustrated in Fig. 2, the video and audio signals may be processed in a way as illustrated in Fig. 3.

[0130] Fig. 3 is a diagram illustrating the process of converting a video segment as captured by an image capturing device into a combined audio / video segment according to the present invention.

[0131] Fig. 3 shows (upper part) that the video segment 6 of gait cycles of a horse and which has been captured by an image capturing device 4 comprises a series of time separated images 8.

[0132] Lower part of Fig. 3 shows that during processing of the video segment by using the apparatus 100 according to the first aspect of the present invention, a combined audio / video segment 18 is generated.

[0133] In the combined audio / video segment 18 the original video segment 6 has been combined with an audio segment 14.

[0134] Moreover, as seen in lower part of Fig. 3, a visual representation 28 of one or more gait movement parameters 26 has additionally been mixed into the combined audio / video segment 18 (as illustrated by the curve in Fig. 3). The visual representation 28 of the one or more gait movement parameters 26 have been obtained from analyzing the motion data 12 by the data processing device 10.

[0135] Fig. 4 is a diagram illustrating the working principle of the present invention.

[0136] In the upper part of Fig. 4 are illustrated excerpts from a video segment 6 of a trotting horse 2 which has been captured by an image capturing device 4. The excerpts of the video segment 6 comprises 10 images numbered No. 1 through No. 10.

[0137] The images No. 1 - 10 as seen in Fig. 4 are not necessarily consecutive images of the video segment 6. Rather, the images 1 through 10 have been selected and excepted from the video segment 6 for the purpose of illustrating the principle of the present invention according to this example.

[0138] It is seen in the images in the upper part of Fig. 4, that in the images No. 3 and 7 the right front (RF) leg is in its midstance position, and in image No. 2, 6 and 10 the right foot (RF) is just about to hit the ground.

[0139] Likewise, it is seen that in the upper part of Fig. 4, that in the images No. 1, 5, and 9 the left front leg (LF) is in its midstance position, and in images No. 4 and 8 the left foot (LF) is just about to hit the ground.

[0140] Fig. 4 also comprises a lower part showing a graph illustrating the vertical movement of the head of the horse during a couple of trot gait cycles in the same way as explained in respect of Fig. la and lb above.

[0141] This vertical movement of the horse has been determined from motion data which has been extracted from the images 8 of the video segment 6 captured by the image capturing device 4 of the apparatus 100 of the present invention. In Fig. 4 the postures of the horse in the upper part of Fig. 4 during the gait cycles have been correlated with the vertical movement of the head of the horse (lower part of Fig. 4) during the gait cycles.

[0142] It is seen that the amplitude of the sinusoidal curve of lower part of Fig. 4 is lower (i.e. higher variation in the amplitude) when the sound foreleg is in contact with the ground (Positions A on the sinusoidal curve), whereas the opposite is true when the lame foreleg is in contact with the ground (Positions B on the sinusoidal curve).

[0143] Similarly, the impact intensity of a leg with the ground will be higher when the sound foreleg hits the ground in its descending movement (positions a on the sinusoidal curve), compared to the situations, where the lame foreleg impacts the ground (positions b on the sinusoidal curve).

[0144] The principle of the present invention is to perform a video analysis of a video segment 6 of a gait cycle of a quadruped 2 and extract motion data indicative of one or more biometric characteristics of the gait cycle, to determine a gait rhythm of the gait cycle, to generate an audio segment 14 comprising audio signal pulses 16 corresponding to that gait rhythm, and to mix that audio segment 14 with that originally captured video segment 6.

[0145] In Fig. 4 this latter audio part of the invention is illustrated by the loudspeaker symbols appearing in the lower part of Fig. 4. Each occurrence of a loudspeaker symbol symbolizes a “beat” (foot’s impact with the ground) of the gait cycle, and the change in size of the loudspeaker symbols, symbolizes a change of characteristics of the audio signal pulses 16 produces by the loudspeaker, such as varying volume of the audio signal pulses commensurate with the symmetry or asymmetry of the gait cycle.

[0146] The present invention is highly useful in training a human being in identifying the presence of a lameness in a quadruped, such as a horse.

[0147] It should be understood that all features and achievements discussed above and in the appended claims in relation to one aspect of the present invention and embodiments thereof apply equally well to the other aspects of the present invention and embodiments thereof.

[0148] List of reference numerals

[0149] 2 Quadruped

[0150] 4 Image capturing device

[0151] 6 Video segment 8 Image of video segment

[0152] 10 Data processing device

[0153] 12 Motion data extracted from series of images

[0154] 14 Audio segment

[0155] 16 Audio signal pulse 18 Combined audio / video segment

[0156] 20 Display

[0157] 22 Audio reproducing device

[0158] 24 Data storage

[0159] 26 Gait movement parameter 28 Visual representation of gait movement parameter.

[0160] 30 Learning images

[0161] 100 Apparatus

Claims

Claims1. An apparatus (100) for aiding in evaluating the gait of a quadruped (2), wherein said apparatus comprises:-an image capturing device (4) for capturing a video segment (6) of a quadruped in motion, said video segment represents a series of time separated images (8), wherein said video segment is having a duration which spans at least one gait cycle of the quadruped;-a data processing device (10); wherein said data processing device (10) is being configured, from said series of images (8) of said video segment (6), to extract motion data (12) indicative of one or more predetermined biometric characteristics of the gait cycle(s) of said quadruped; wherein said data processing device (10) is being configured, from said motion data (12), to determine a gait rhythm of said gait cycle(s); wherein said data processing device (10) is being configured to generate an audio segment (14) comprising a plurality of audio signal pulses (16) corresponding to said gait rhythm of said gait cycle(s); wherein said data processing device (10) is being configured to mix said audio segment (14) with said video segment (6), thereby generating a combined audio / video segment (18).

2. An apparatus (100) according to claim 1, wherein said apparatus further comprises a display (20) and an audio reproducing device (22), such as loudspeaker or a headphone, for visual and audible reproducing said combined audio / video segment (18).

3. An apparatus (100) according to claim 2, wherein said apparatus is being configured to visually and audibly reproduce said combined audio / video segment (18) in real-time or in near real-time, relative to the time of capturing said video segment (6) by said image capturing device (4).

4. An apparatus (100) according to any of the preceding claims, wherein said apparatus comprises a data storage (24) configured for storing said video segment (6) captured by said image capturing device (4) and / or configured for storing said combined audio / video segment (18) generated by said data processing device (10).

5. An apparatus (100) according to any of the preceding claims, wherein said audio segment (14) generated by said data processing device (10) comprises a silent background layer and an audio layer comprising said individual audio signal pulses (16); or wherein said audio segment (14) generated by said data processing device (10) comprises a non-silent background layer and an added audio layer comprising said audio segment (14) comprising said individual audio signal pulses (16).

6. An apparatus (100) according to any of the preceding claims, wherein said audio signal pulses (16) of said audio segment (14) of said combined video / audio segment (18)independently are having a duration of 5 - 1000 ms (milliseconds), such as 10 - 900 ms, for example 15 - 800 ms, e.g. 20 - 750 ms, such as 25 - 700 ms, for example 50 - 650 ms, such as 75 - 600 ms, such as 100 - 550 ms, for example 150 - 500 ms, e.g. 200 - 450 ms, such as 250 - 400 ms or 300 - 350 ms.

7. An apparatus (100) according to any of the preceding claims, wherein said motion data (12), indicative of said predetermined biometric characteristics of the gait cycle, relates to one or more of the following: vertical displacement of the trunk, such as measured by the parameter MinDiff and / or MaxDiff; limb position, velocity, flight pattern, deceleration and / or acceleration of one or more limbs or anatomic elements of the quadruped.

8. An apparatus (100) according to any of the preceding claims, wherein said data processing device (10) is being configured to determine said gait rhythm in a form comprising two, three or four “beats” per gait cycle, wherein said “beat” optionally represents cyclic movements of one or more anatomic elements of the quadruped (2) during its movement.

9. An apparatus (100) according to claim 8, wherein said data processing device (10) is being configured to generate said audio segment (14) in such a way that it comprises a beat pattern comprising two, three or four of said audio pulses per gait cycle in correspondence with said gait rhythm.

10. An apparatus (100) according to any of the preceding claims, wherein in the combined audio / video segment (18), the audio segment (14) is being synchronized with said video segment (6) in such a way that said audio signal pulses (16) of said audio segment (14) are being synchronized with a cyclic movement of one more limbs or anatomic elements of the quadruped (2), such as based on each foot’s impact with the ground.

11. An apparatus (100) according to any of the preceding claims, wherein the duration of said video segment (6) and the duration of said audio segment (14) are identical, or wherein the duration of said video segment (6) is longer than the duration of said audio segment (14).

12. An apparatus (100) according to any of the preceding claims, wherein said video segment (6) and said combined video / audio segment (18) span a plurality of gait cycles, such as 2 - 1000 or more gait cycles, for example 5 - 950 gait cycles, such as 10 - 900 gait cycles, for example 15 - 850 gait cycles, such as 20 - 800 gait cycles, e.g. 25 - 750 gait cycles, for example 50 - 700 gait cycles, such as 100 - 650 gait cycles, for example 150 - 600 gait cycles, for example 200 - 550 gait cycles, such as 250 - 500 gait cycles, e.g. 300 - 450 gait cycles or 350 - 400 gait cycles.

13. An apparatus (100) according to any of the preceding claims, wherein one or more of said one or more predetermined biometric characteristics of the gait cycle relate to an intensity of movement of one or more of the four legs of the quadruped; and wherein said data processing device (10) is being configured to determine, from said motion data (12) and in respect of one or more specific gait cycles of said video segment (6), preferably in respect of each gait cycle of said video segment (6), a variation in the intensity of movement of the four legs of the quadruped within the same gait cycle, or wherein said data processing device (10) is being configured to determine, from said motion data (12) and in respect of two or more gait cyclesof said video segment (6), a variation in the intensity of movement of the four legs of the quadruped within said two or more gait cycles.

14. An apparatus (100) according to claim 13, wherein said data processing device (10) is being configured to generate said audio segment (14) comprising said plurality of audio signal pulses (16) in such a way that the intensity and / or the frequency and / or timbre and / or another sound expression characteristics of said audio signal pulses (16) will vary commensurate with the variation in the intensity of movement of the four legs of the quadruped within the same gait cycle, or within said two or more gait cycles, as the case may be.

15. An apparatus (100) according to claim 14, wherein said data processing device (10) is being configured to generate said audio segment (14) comprising said plurality of audio signal pulses (16) in such a way that said audio signal pulses (16) is representing the sound of a horse hoof hitting the ground, such as in the form of a recorded sound of a horse hoof hitting the ground, or in the form of an artificially generated sound resembling the sound of a horse hoof hitting the ground; or wherein said data processing device (10) is being configured to generate said audio segment (14) comprising said plurality of audio signal pulses (16) in such a way that said audio signal pulses (16) comprise sounds having an increased frequency or a decreased frequency over the duration of each audio signal pulses (16).

16. An apparatus (100) according to any of the preceding claims, wherein the frame rate of said video segment (6) and / or of said combined video / audio segment (18) is selected from the ranges of 1 - 1,000 fps (frames per second), such as 12 - 900 fps, e.g. 24 - 800 fps, for examples 36 - 600, for example 60 - 500 fps, such as 120 - 400 fps or 240 - 300 fps.

17. An apparatus (100) according to any of the preceding claims, wherein the apparatus is configured for evaluating the gait of a horse, a dog, a cat, cattle or swine.

18. An apparatus (100) according to any of the preceding claims, wherein said data processing device (10) furthermore is being configured, from said motion data (12), to extract one or more gait movement parameters (26) which may be indicative of asymmetry or lameness of the quadruped (2).

19. An apparatus (100) according to claim 18, wherein said data processing device (10) is being configured to extract one or more of said one or more gait movement parameters (26) dynamically over the span of said video segment (6).

20. An apparatus (100) according to claim 18 or 19, wherein said data processing device (10) is being configured to add a visual representation of one or more of said gait movement parameters (26) in such a way that said combined audio / video segment comprises a visual representation (28) of one or more of said one or more gait movement parameters (26).

21. An apparatus (100) according to any of the preceding claims, wherein said data processing device (10) is being configured, based on said series of time separated images (8) of said video segment (6) to analyze said images and to identify key features / key points thereof, relating to specific anatomic elements of the quadruped, and wherein said data processing device (10) is being configured to determine, when going from one image (8) to the next ofsaid video segment (6), in which way such key features / key points move in relation the quadruped during its movement.

22. An apparatus (100) according to claim 21, wherein said key features / key points relate to one or more of the following anatomic elements: head, neck, withers, croup, pelvis, back, front leg(s); such as elbow, carpus, front fetlock, front foot / hoof of front leg; back legs, such as stifle, hock, hind fetlock, hind foot / hoof of back leg.

23. An apparatus (100) according to claim 21 or 22, wherein said analysis being performed by said data processing device (10) is being based on patch features, such as colour distinction, edge detection, morphology, entropy of image patches making up such images (8).

24. An apparatus (100) according to any of the claims 21 - 23, wherein said apparatus comprises a data storage (24), wherein said data storage comprises a representation of learning images (30), and wherein said data processing device (10) is configured to perform said analysis by consulting said representation of learning images (30), wherein one or more of said learning images of said representation of learning images comprises a real image or an artificially generated image, such as a computer generated image, of said species of quadruped (2) or a part of said species of quadruped at different postures; and wherein one or more of said images of said representation of learning images (30) has / have been allocated with information correlating said image with information relating to anatomical details depicted by said image.

25. An apparatus (100) according to any of the claims 21 - 24, wherein said data processing device (10) is being configured for employing deep learning algorithms and / or artificial intelligence and / or neural network in analyzing and in identifying said key features / key points of the series of images of said video segment (6).

26. An apparatus (100) according to any of the preceding claims, wherein said apparatus is being in the form of a mobile device, such as a smartphone or a tablet or the like; or is in the form of an augmented reality device, such as augmented reality glasses or goggles.

27. An apparatus (100) according to claim 26 further comprises a separate receiving device for receiving electromagnetic radiation, wherein said mobile device (100) furthermore comprises transmitting means for relaying, by transmission of electromagnetic radiation, signals, representing said audio segment (14) to said receiving device, when said receiving device is being carried by a rider who is riding said quadruped which is in the form of a horse, thereby allowing said rider to receive and hear said audio signal pulses (16) of the horse while riding that horse.

28. Use of an apparatus (100) according to any of the preceding claims as an aid in evaluating the gait of a quadruped (2).

29. A method for evaluating the gait of a quadruped (2), wherein said method comprises: i) capturing a video segment (6) of a quadruped in motion, wherein said video segment comprises a series of time separated images (8), wherein said video segment is having a duration which spans at least one gait cycle of the quadruped;ii) from said video segment (6) obtained in step i), extracting motion data (12) indicative of one or more predetermined biometric characteristics of the gait cycle of said quadruped; iii) from said motion data (12) obtained in step ii), determining a gait rhythm of said gait cycle; iv) generating an audio segment (14) comprising a plurality of audio signal pulses (16) corresponding to said gait rhythm of said gait cycle as determined in step iii); v) mixing said audio segment (16) obtained in step iv) with said video segment (6) obtained in step i); thereby generating a combined audio / video segment (18); vi) visually and audible reproducing said combined audio / video segment (18) obtained in step v).

30. A method according to claim 29, wherein step ii) furthermore comprises the following steps: iia) from said motion data (12) obtained in step i), extracting one or more gait movement parameters (26) which may be indicative of lameness of the quadruped; via) in step vi) visually reproducing one or more of said one or more gait movement parameters (26) obtained in step iia).

31. A method according to any of the claims 29 - 30, wherein said method is performed while the quadruped (2) is performing a movement in a gait type selected from the group comprising: walk, trot, canter, lope, pace, gallop or tbit.

32. A method according to any of the claims 29 - 31, wherein said quadruped is being a horse, a dog, a cat, cattle or swine.

33. A method according to claim 32, wherein the quadruped is a horse, and wherein the horse is moving without a rider or with a rider.

34. A method according to any of the claims 29 - 33, wherein said method is performed by using an apparatus (100) according to any of the claims 1 - 27.

35. A computer program product which, when being loaded on a computer, is being configured to perform steps ii) - vi) of the method of any of the claims 29 - 34.

36. A computer program product according to claim 35, wherein said computer program product is being in the form of an app to a mobile device, such as a smartphone, a tablet or an augmented reality device.

Images