Scoring method of dairy cow gait based on machine vision

Abstract

The present invention provides a scoring method for cow gait based on machine vision, which belongs to the field of animal husbandry. The method comprises: obtaining a walking video of a cow; extracting characteristic parts in each frame image of the walking video of the cow and determining characteristic parameters corresponding to the characteristic parts; calculating the judgment parameter value of each scoring item in the cow gait based on the characteristic parameters corresponding to the characteristic parts; determining the judgment state of the scoring item according to the judgment parameter value of each scoring item in the cow gait; scoring the judgment states of all scoring items according to the scoring conditions in the cow gait scoring rules to obtain the cow gait score. The present invention integrates the judgment states of multiple scoring items to improve the accuracy of cow scoring.

Description

Technical Field

[0001] The invention relates to the field of animal husbandry, and in particular to a cow gait scoring method based on machine vision. Background Art

[0002] At present, the detection methods of lameness in dairy cows are mainly divided into contact and non-contact methods. Domestic and foreign scholars have conducted a lot of research on lameness detection by analyzing the behavioral characteristics of dairy cows. In previous studies, the detection of cow arch back was usually the main focus, but the arch back characteristics have individual differences, and different cow body conditions have a greater impact on the arch back; for cow gait detection, leg extraction is relatively vague, and some studies can only determine the position of cow hoof landing by manual marking or combining with pressure pads, and cannot achieve accurate automatic positioning. Cow lameness is mainly caused by behavioral changes caused by hoof disease. When cows walk, the hoof's load-bearing capacity decreases due to pain. The cow's gait changes and the corresponding changes in the head and neck slope and the degree of arch back can intuitively reflect the degree of lameness. The gait of lame cows can be detected by machine vision technology. Because the cow's hoof is relatively small in size and the shape characteristics are not obvious, it is difficult to locate it. Previous studies mainly focused on manual marking, which cannot achieve automatic detection. In addition, in the detection of cow lameness, it is usually judged whether the cow is lame or not based on the arch back situation or lameness detection.

[0003] In the prior art, Belgian scholars Poursaberi et al. used image analysis technology to identify lameness in dairy cows at an early stage. This study took into account the pause of the cow in front of the camera and the inconsistency of the cow's walking, pre-processed the original collected data, roughly located the moving cows by background elimination, and identified the moving cows based on the statistical analysis method of binary grayscale images. In order to extract the cow's features from the background, the foreground magnification method was used to separate the cow image and the background, and the images of the cow's back when standing and walking were obtained. By comparing with the back spline curve of normal cows, the back spline curve of lame cows was significantly different. Therefore, the back posture information can detect cow trekking. In order to study how hoof disease affects the gait of dairy cows, Flower et al. used a camera to record the gait data of 48 high-yielding Holstein cows in a 40m long measurement channel, extracted 6 kinematic gait features through motion analysis software, and drew a spatial-temporal relationship distribution map of the toes and hooves of healthy dairy cows and dairy cows with hoof lesions. Through analysis, it was found that compared with dairy cows with hoof disease, healthy dairy cows walked faster, had a shorter stride duration, and a longer stride; dairy cows completed double-toe support by alternating support between the ipsilateral hoof and the contralateral hoof; during three-toe support, healthy dairy cows only spent 18% of their stride time on three toes, while dairy cows with hoof disease more than doubled this data to reduce the load on the affected toes, accounting for 42% of the stride time. It can be seen from this that there are differences in the support duration and swing duration of dairy cows with hoof lesions, which can be used as indicators for detecting lameness. Kangxi et al. used computer vision technology to automatically locate the position of dairy cow hooves, and identified and classified lameness by analyzing the movement trajectory of the ipsilateral hoof when the dairy cows walked. Song Huaibo extracted the contour line connecting the head, neck and back from the sequence images of dairy cows, calculated the slope of the fitted straight line, and classified 18 dairy cows for lameness identification, with 94% of the cows correctly classified. Jiang et al. used a binormal distribution model to obtain the pixel area of ​​the target dairy cow and designed a background statistical model to determine the degree of dairy cow lameness. The results showed that this method can be used to detect dairy cow lameness.

[0004] Since the 1980s, a lot of research has been done on the detection of lameness using force sensor technology, mainly using force plates and pressure-sensitive pads to develop an automatic lameness identification system for dairy cows. Telezhenko first described the spatiotemporal variables of cow footprints to evaluate the gait pattern of non-lame cows. Variables such as stride length, step length, step width, tracking distance, step angle, and gait asymmetry were selected. By comparing the coefficient of variation of each variable, it was found that the variation between measurements was small, indicating that cows maintained the same average gait pattern for at least 3 weeks. Further analysis of the spatiotemporal characteristics of 32 dairy cows showed that the gait speed was positively correlated with stride length, step length, and tracking distance. Therefore, when dairy cows are healthy, their gait is relatively stable, but when lameness occurs, the stability will be destroyed, resulting in abnormal gait. Van De Gucht et al. selected 9 non-lame cows, 11 mildly lame cows, and 12 severely lame cows, and used pressure-sensitive pads to measure the time variables related to the cows' toe and hoof landing, weight bearing, and lifting. Comparing the variables under different scores, it was found that the proportion of the time when the toe of lame cows touched the ground and lifted was longer, while the proportion of the time when they were fully weight-bearing was shorter. The time when the toe touched the ground and lifted increased with the increase of lameness. This shows that in order to relieve the pain of the toe, the cow will carefully put the toe on the ground, and the speed of lifting the toe will also be slower. Therefore, the gait characteristics of cows in the swinging stage can be used as a basis for detecting lameness.

[0005] In the research on lameness detection of dairy cows based on machine vision technology, the current research objects are relatively single, mostly focusing on single detection and analysis of cows' head shaking, arched back, and lameness, which cannot accurately determine the lameness of dairy cows. Moreover, the detection results are usually whether there is lameness behavior, and it is impossible to accurately judge the degree of lameness of dairy cows. In the research on lameness detection of dairy cows based on force sensor technology, the hoof data of dairy cows can be collected comprehensively and intuitively, including the time of contact with the ground, force analysis, etc. However, this solution is not easy to implement in actual use and is limited to laboratory conditions. The equipment based on force sensor technology has strict requirements on the experimental environment during use, and the environment of ordinary farms cannot meet the standards of laboratories; the equipment price is not cost-effective, and deploying the equipment in farms will cause huge cost increases for farms. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a method for scoring cow gait based on machine vision, so as to at least solve the accuracy problem of the above-mentioned cow scoring in engineering applications.

[0007] In order to achieve the above object, the first aspect of the present invention provides a method for scoring cow gait based on machine vision, the method comprising:

[0008] Get videos of cows walking;

[0009] Extracting characteristic parts in each frame of the cow's walking video and determining characteristic parameters corresponding to the characteristic parts;

[0010] Calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic parts;

[0011] Determining a determination state of each scoring item according to a determination parameter value of the scoring item in the cow's gait;

[0012] The judgment status of all scoring items is scored according to the scoring conditions in the gait scoring rules of the dairy cow to obtain the gait score of the dairy cow.

[0013] Optionally, the judgment parameter value of the scoring item includes: the judgment parameter value of the head shaking condition; the characteristic parts include the cow skull and the end of the cervical vertebra;

[0014] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0015] Determine the corresponding coordinates of the cow skull and the end of the cervical vertebrae in each frame of the image;

[0016] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0017] According to the corresponding coordinates of the cow skull and the end of the cervical vertebra in each frame of the image, the Euclidean distance between the cow skull and the end of the cervical vertebra in each frame of the image is calculated;

[0018] The Euclidean distances between the cow skull and the end of the cervical vertebrae in all frame images are sorted in time order to obtain a regression equation;

[0019] The regression equation is used to determine the judgment parameter value of the head swing situation.

[0020] Optionally, the judgment parameter values ​​of the scoring items include: judgment parameter values ​​of the arched back condition; the characteristic parts include the neck, back and hip angle;

[0021] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0022] Determine the corresponding coordinates of the neck, back and hip angles in each frame of the image;

[0023] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0024] Calculate the cervical vertebrae curvature in each frame of image according to the corresponding coordinates of the neck, back and hip angle in each frame of image; the cervical vertebrae curvature is the angle between the coordinate line connecting the neck and hip angle and the coordinate line connecting the back and hip angle;

[0025] Arrange the cervical vertebrae curvatures in all frame images in chronological order to determine the relationship between the cervical vertebrae curvature and time;

[0026] The relationship between the cervical vertebra arc and time is used to determine the judgment parameter value of the kyphosis condition.

[0027] Optionally, the judgment parameter values ​​of the kyphosis condition include the extreme difference value of the spinal curvature and the average value of the spinal curvature;

[0028] Determining the determination state of each scoring item according to the determination parameter value of each scoring item in the cow gait includes:

[0029] The judgment state of the hunchback condition is determined according to the comparison relationship between the extreme difference value of the spinal curvature and the average value of the spinal curvature and the corresponding threshold value.

[0030] Optionally, the judgment parameter value of the scoring item includes: a judgment parameter value of a gait feature; the characteristic part includes a hoof of a dairy cow;

[0031] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0032] Determine the corresponding coordinates of each cow hoof in each frame image based on the reference coordinate system; the coordinates include an X-axis coordinate value and a Z-axis coordinate value; the Z-axis coordinate value is a frame index; the X-axis coordinate value is a distance that the cow hoof moves horizontally in the frame image;

[0033] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0034] Establish a corresponding relationship diagram between the X-axis coordinate values ​​of all cattle hooves and the frame index;

[0035] Obtain the time difference of the falling edge in the corresponding relationship diagram between the X-axis coordinate values ​​of all cow hooves and the frame index; determine the footstep sequence of the cow's walking process according to the time difference of the falling edge;

[0036] When it is determined that the step sequence of the cow's walking process is normal, the step length of each step corresponding to each cow's hoof is determined according to the corresponding relationship diagram, and the average step length of each cow's hoof in the detection period is calculated according to the step length of each step corresponding to each cow's hoof, so as to obtain the average step length of all cow's hoofs in the detection period;

[0037] The minimum value of the average stride lengths of all the cow hooves in the detection period is determined, and the difference between the minimum value and the average stride lengths of all the cow hooves in the detection period is calculated as the determination parameter value of the gait feature.

[0038] Optionally, determining the step length of each step corresponding to each cow hoof according to the corresponding relationship diagram includes:

[0039] Get all the X-axis coordinate values ​​corresponding to the falling edge interval in the corresponding relationship diagram of each cow hoof;

[0040] Calculate the average value of the X-axis coordinate values ​​corresponding to the previous adjacent falling edge interval of the current falling edge interval, and calculate the average value of the X-axis coordinate values ​​corresponding to the current falling edge interval;

[0041] The difference between the average value of the X-axis coordinate values ​​corresponding to the last adjacent falling edge interval of the current falling edge interval and the average value of the X-axis coordinate values ​​corresponding to the current falling edge interval is used as the step size corresponding to the current falling edge interval.

[0042] Optionally, the calculation of the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points further includes:

[0043] When it is determined that the step sequence of the cow's walking process is normal, the time required for each step corresponding to each cow's hoof is determined according to the corresponding relationship diagram; the difference between the minimum value and the average step length of all cow's hooves in the detection period or the time required for each step corresponding to each cow's hoof is used as the judgment parameter value of the gait feature.

[0044] Optionally, determining the time required for each step corresponding to each cow hoof according to the corresponding relationship diagram includes:

[0045] Obtain the average frame index corresponding to the next falling edge and the average frame index corresponding to the previous falling edge in each step in the corresponding relationship diagram of each cow hoof;

[0046] Calculate the frame index difference between the average frame index corresponding to the next falling edge and the average frame index corresponding to the previous falling edge in each step;

[0047] The product of the frame index difference and the frame rate of the cow's walking video is taken as the time required for each step corresponding to each cow hoof.

[0048] Optionally, the judgment parameter value of the scoring item includes: the judgment parameter value of the load condition; the characteristic part includes the hoof of the dairy cow;

[0049] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0050] Determine the Y-axis coordinate corresponding to each cow hoof in each frame image; the Y-axis coordinate value is the distance the cow hoof moves in the vertical direction in the frame image;

[0051] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0052] According to the Y-axis coordinates corresponding to each hoof in all frame images, the average value of the axis coordinates of each hoof is calculated;

[0053] Compare the average values ​​of the axis coordinates of all the hooves; determine the smallest of the average values ​​of the axis coordinates of all the hooves as the first parameter, and the average values ​​of the axis coordinates of the remaining hooves as the second parameter;

[0054] The difference between the second parameter and the first parameter is used as a judgment parameter value of the load condition.

[0055] Optionally, the determination status of the scoring items includes: the determination result status of the head shaking condition, the determination result status of the weight bearing condition, the determination result status of the hunched back condition and the determination result status of the gait characteristics;

[0056] The determination status of all scoring items is determined according to the scoring conditions in the gait scoring rules of the dairy cow to obtain the gait score of the dairy cow, including:

[0057] The gait score of the cow is obtained based on the result of the head condition and the result of the weight condition; or

[0058] Obtaining a gait score of the cow according to the judgment result state of the head shaking condition and the judgment result state of the gait characteristics; or

[0059] The gait score of the dairy cow is obtained according to the judgment result state of the head shaking condition, the judgment result state of the gait characteristics and the judgment result state of the arched back condition.

[0060] The present invention also provides a computer-readable storage medium on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned cow gait scoring method based on machine vision.

[0061] Through the above technical scheme, the judgment status of all scoring items is scored according to the scoring conditions in the gait scoring rules of the cow to obtain the gait score of the cow; and the judgment status of the scoring item is determined by calculating the judgment parameter value of each scoring item in the cow's gait, and the judgment status of multiple scoring items is integrated to improve the accuracy of the cow's scoring.

[0062] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0064] Figure 1 This is a flow chart of a method for scoring cow gait based on machine vision provided by an embodiment of the present invention;

[0065] Figure 2 is a schematic diagram of scoring using specific scoring conditions provided by an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of a cow video collection layout provided by an embodiment of the present invention;

[0067] Figure 4 This is a cow head shaking situation analysis diagram provided by an embodiment of the present invention;

[0068] Figure 5 It is a spinal curvature change diagram provided by an embodiment of the present invention;

[0069] Figure 6 It is a stereoscopic diagram of the movement trajectory of a cow's hoof provided by an embodiment of the present invention;

[0070] Figure 7 1 is a relationship diagram between an x ​​value of a cow's hoof and a frame index provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0072] like Figure 1 As shown, in order to achieve the above-mentioned purpose, the first aspect of the present invention provides a method for scoring cow gait based on machine vision, the method comprising:

[0073] Acquire a walking video of a dairy cow; extract characteristic parts in each frame image of the walking video of the dairy cow and determine characteristic parameters corresponding to the characteristic parts; calculate the judgment parameter value of each scoring item in the dairy cow's gait based on the characteristic parameters corresponding to the characteristic parts; determine the judgment state of the scoring item according to the judgment parameter value of each scoring item in the dairy cow's gait; score the judgment states of all scoring items according to the scoring conditions in the dairy cow's gait scoring rules to obtain the dairy cow's gait score.

[0074] Use a camera to capture video of the cow's walking process. The capture method is as follows: Figure 3As shown. Preferably, a video of the cow walking on the way to the milking parlor is collected. The road surface of this section is hard and not easy to slip, so the cow's walking video can be well collected, showing the cow's normal gait. By installing a fence, only a single cow is allowed to pass through at a time, ensuring that only a single cow passes through the camera acquisition area. Use a deep learning algorithm to identify the characteristic points of the cow: head, neck, back, tail, tibia, knee, and hoof, record the coordinates of the characteristic points in each frame of the video, analyze the changes in the coordinates of each characteristic point of the cow during the walking process, reflect the changes in the cow's head shaking, arched back, and gait characteristics during the walking process, and use the five-point scoring system widely used in the dairy farming industry to correspond to each characteristic point to give a gait score.

[0075] Specifically, the technical roadmap of the present invention is as follows: Figure 2 As shown in the figure, the input of the system is a video of a cow walking captured by a camera. The key points of the cow, including the head, neck, back, tail, and hoof, are extracted through the pre-trained DeepLabCut deep convolutional network, and the key coordinate values ​​and confidence levels of each frame are obtained. The head shaking of the cow is reflected by calculating the length of the line connecting the head and the neck for regression analysis. The arched back of the cow is reflected by the change in the angle between the neck, back, and tail within the range time. The movement of the cow is reflected by the change in the x-coordinate of the cow's hoof within the range time. The weight-bearing condition of the cow is reflected by the size relationship of the y-coordinate of the cow's hoof within the range time. The above four situations are comprehensively judged according to the gait scoring rules of the cow to obtain the gait score of the cow.

[0076] The key point extraction of the video is implemented using the DeepLabCut deep convolutional network. This network model is an open source software package for posture estimation developed by Mackenzie Mathis from EPEL Lausanne Institute of Technology. It can obtain relatively accurate performance with a small amount of data annotation (50-200 pictures). The model training selects unobstructed cow walking videos. The video must be able to clearly identify the key points mentioned above with the naked eye. The kemeans algorithm is used to extract 20 frames from each video (depending on the number of training videos, ensuring that the final number of extracted frames is about 200 frames). After that, the extracted 20 frames are manually screened to ensure that the key points in each frame are clearly visible. Pictures that do not meet the requirements are replaced with adjacent frames. The data annotation software provided by the model is used to annotate the feature points in the selected pictures. The annotation information includes the name and position of the feature points. The training set and the test set are randomly divided according to the proportion to conduct the first model training. The model obtained after training is tested, and the feature points that are not accurately detected are re-labeled and trained to finally obtain an accurate model. Use the trained model to detect the video, export the detection information, and obtain the feature points (that is, the feature parts, in this embodiment, the center point of the feature parts is preferably used as the feature point) and the feature parameters corresponding to the feature parts about the frame index; wherein, the feature parameters corresponding to the feature parts include coordinate information and confidence; the coordinate information and confidence are obtained through the trained DeepLabCut deep convolutional network. The coordinates of the feature points (the coordinate values ​​of each feature point are derived from the positions of each key point in each frame in the video image) include the distance that the feature point moves in the X-axis direction in the frame image, the distance that the feature point moves in the Y-axis direction in the frame image (the distance moved vertically) and the Z-axis related to the time attribute (preferably calibrated by the frame index). The above confidence is used to indicate the reliability of the coordinate value of the point, and points with low confidence are not adopted.

[0077] Optionally, the judgment parameter value of the scoring item includes: the judgment parameter value of the head shaking condition; the characteristic parts include the cow skull and the end of the cervical vertebra;

[0078] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0079] Determine the corresponding coordinates of the cow skull and the end of the cervical vertebrae in each frame of the image;

[0080] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0081] According to the corresponding coordinates of the cow skull and the end of the cervical vertebra in each frame of the image, the Euclidean distance between the cow skull and the end of the cervical vertebra in each frame of the image is calculated;

[0082] The Euclidean distances between the cow skull and the end of the cervical vertebrae in all frame images are sorted in time order to obtain a regression equation;

[0083] The regression equation is used to determine the judgment parameter value of the head swing situation.

[0084] Specifically, the shaking of the head during the process. The most obvious feature of a cow's head shaking is the rhythmic up and down shaking of the cow's head during walking, which can be analyzed by analyzing the movement trajectory of the cow's head. During the cow's walking process, the movement of the head can be divided into horizontal forward movement on the x-axis and vertical up and down movement on the y-axis. The movement trajectory on the x-axis is a straight line, and it is impossible to accurately judge the movement state of the cow's head. The movement trajectory on the y-axis can be approximated to a regular waveform. During the normal walking process of the cow, the height of the cow's head will also have a slight up and down movement trajectory, so it is also impossible to judge by the movement trajectory of the cow's head on the y-axis.

[0085] Therefore, it is proposed to reflect the cow's head shaking situation by changing the length of the line connecting the cow's head and neck. The length of the line connecting the cow's head and neck is defined as the Euclidean distance between the two points, and the calculation formula is shown in formula (1). The calculation formula of the Euclidean distance Dis is:

[0086]

[0087] where x h With y h Represents the x and y coordinates of the cow's head, x n With y n Represents the x and y coordinates of the cow's neck. By analyzing the obtained distance, it can be found that the distance of a normal cow during walking does not change significantly as the cow walks, while the distance of a lame cow changes regularly with walking, and the change range is large. The change trend of cow Dis can be predicted and analyzed through time series.

[0088] Define the frame index as coordinate y, the Euclidean distance Dis as x, and use the least squares method to find the regression equation:

[0089]

[0090] in,

[0091]

[0092] The distance of a normal cow is relatively stable during walking, with a small change, and the trend line is relatively gentle, that is, the calculated b value is close to 0. However, the distance of a lame cow changes greatly during walking, showing an increasing trend, and the calculated b value is larger. The threshold is set. The b value (that is, the judgment parameter value of the swing head condition) is greater than the threshold Determine whether there is a head swing, otherwise there is no head swing.

[0093] like Figure 4 As shown in the figure, the dotted line is the relationship between the head and neck length and the frame index during the movement of the cow, and the straight line is the change trend of the head and neck length. This cow is a normal cow. It can be seen that the change trend of the head and neck length of the cow during walking is horizontal. The calculated b value (that is, the judgment parameter value of the head shaking situation) is approximately equal to 0.0053, indicating that the change of the head and neck length of the cow during walking tends to be stable. Therefore, it can be inferred that the cow does not shake its head during walking.

[0094] Optionally, the judgment parameter values ​​of the scoring items include: judgment parameter values ​​of the arched back condition; the characteristic parts include the neck, back and hip angle;

[0095] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0096] Determine the corresponding coordinates of the neck, back and hip angles in each frame of the image;

[0097] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0098] Calculate the cervical vertebrae curvature in each frame of image according to the corresponding coordinates of the neck, back and hip angle in each frame of image; the cervical vertebrae curvature is the angle between the coordinate line connecting the neck and hip angle and the coordinate line connecting the back and hip angle;

[0099] Arrange the cervical vertebrae curvatures in all frame images in chronological order to determine the relationship between the cervical vertebrae curvature and time;

[0100] The relationship between the cervical vertebra arc and time is used to determine the judgment parameter value of the kyphosis condition.

[0101] Specifically, the cow's arched back condition is reflected by the cervical radian. The cervical radian is defined as the angle between the three points of the neck, back and hip angle. The cosine value of the angle θ can be obtained by formula (4);

[0102]

[0103] Among them, x n With y n Represents the x and y coordinates of the neck, x b With y b Indicates the x and y coordinates of the back, x t With y t Represents the x and y coordinates of the hip angle. The angle of the angle can be obtained through the cosine value of the angle. Figure 5 (a) shows the change of the spine curvature of a normal dairy cow during walking. Figure 5 (b) shows the change of the spinal curvature of lame cows during walking. The horizontal axis in the figure is the frame index, and the vertical axis is the spinal curvature. It can be seen that the spinal curvature changes all the time as the cow walks. The spinal curvature of normal cows changes relatively little and the curvature is larger, while the spinal curvature of lame cows changes greatly and the curvature is lower than that of normal cows.

[0104] Optionally, the judgment parameter values ​​of the kyphosis condition include the extreme difference value of the spinal curvature and the average value of the spinal curvature;

[0105] Determining the determination state of each scoring item according to the determination parameter value of each scoring item in the cow gait includes:

[0106] The judgment state of the hunchback condition is determined according to the comparison relationship between the extreme difference value of the spinal curvature and the average value of the spinal curvature and the corresponding threshold value.

[0107] Specifically, the judgment of the hunchback condition of the cow is defined as a comprehensive evaluation of the extreme value of the cow's spine curvature and the average value of the spine curvature. First, the extreme value of the cow's spine curvature is calculated, and the calculation formula is shown in Formula 5:

[0108]

[0109] The range is the extreme value, which is obtained by subtracting the sum of the three values ​​with the smallest spinal curvature from the sum of the three values ​​with the largest spinal curvature and dividing the sum by three. The extreme value is used to reflect the change in the spinal curvature of the cow during walking. The larger the change, the more serious the hunchback condition of the cow during walking.

[0110] The average value of the cow's spine curvature is calculated as shown in Formula 6:

[0111]

[0112] The average value of the spinal curvature of a normal cow is larger than the average value of the spinal curvature of a lame cow. Therefore, it can be judged by combining the average value of the spinal curvature of the cow. The more severe the cow's arched back is, the more severe the cow's lameness is, and the smaller the average value of the spinal curvature is. Combined with the extreme value of the spinal curvature of the cow during walking, it can well reflect the back characteristics of the cow during walking and stationary, thereby judging the arched back condition of the cow. The above-mentioned determination state of the arched back condition is determined based on the comparison relationship between the extreme value of the spinal curvature and the average value of the spinal curvature and the corresponding threshold, including: when the extreme value of the spinal curvature of the cow is greater than the threshold, and the average value is lower than the threshold, it can be judged that the cow has an arched back. Obviously, when the extreme value of the spinal curvature of the cow is less than or equal to the threshold, or the average value is greater than or equal to the threshold, it can be judged that the cow does not have an arched back.

[0113] Optionally, the judgment parameter value of the scoring item includes: a judgment parameter value of a gait feature; the characteristic part includes a hoof of a dairy cow;

[0114] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0115] Determine the corresponding coordinates of each hoof in each frame of image based on the reference coordinate system; the coordinates include an X-axis coordinate value (the horizontal movement distance in each frame of image) and a Z-axis coordinate value (a time reference); the Z-axis coordinate value is a frame index;

[0116] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0117] Establish a corresponding relationship diagram between the X-axis coordinate values ​​of all cow hooves and the frame index; the X-axis coordinate value is the distance that the cow hooves move horizontally in the frame image;

[0118] Obtain the time difference of the falling edge in the corresponding relationship diagram between the X-axis coordinate values ​​of all cow hooves and the frame index; determine the footstep sequence of the cow's walking process according to the time difference of the falling edge;

[0119] When it is determined that the step sequence of the cow's walking process is normal, the step length of each step corresponding to each cow's hoof is determined according to the corresponding relationship diagram, and the average step length of each cow's hoof in the detection period is calculated according to the step length of each step corresponding to each cow's hoof, so as to obtain the average step length of all cow's hoofs in the detection period;

[0120] The minimum value of the average stride lengths of all the cow hooves in the detection period is determined, and the difference between the minimum value and the average stride lengths of all the cow hooves in the detection period is calculated as the determination parameter value of the gait feature.

[0121] Specifically, the movement of a cow is a continuous process, which includes objects and their position changes over time. The position includes x-coordinates and y-coordinates, which can be expressed as the change of the x-coordinates and y-coordinates of the object over time t through three-dimensional coordinates. The relationship between the position of the cow's hoof and time is as follows: Figure 6 As shown in the figure, the x-axis is the x-coordinate of the cow's hoof, the y-axis is the y-coordinate of the cow's hoof, and the z-axis is the frame index. Although the three-dimensional coordinates of the cow's movement can intuitively reflect the entire movement process of the cow's hoof, it is not conducive to subsequent data analysis. Therefore, a dimensionality reduction analysis method is proposed to divide the cow's movement process into the relationship between the x-coordinate and time and the relationship between the y-coordinate and time, so as to analyze the cow's movement more intuitively and accurately.

[0122] The movement of dairy cows includes three characteristics: stride, gait, and step frequency. When a normal dairy cow walks, the order of footsteps is LH-LF-RH-RF (where L, R, H, and F represent left, right, back, and front, respectively), with a larger stride and a stable and faster step frequency. However, lame dairy cows will experience different degrees of changes during their walking, such as a smaller stride and a slower step frequency of one hoof. By extracting the hoof movement trajectory of dairy cows, we can Figure 7 As shown in the figure, it is a relationship diagram between the x value of the cow's hoof and the frame index. The horizontal axis in the figure is the frame index sequence, the vertical axis is the X-axis coordinate value, and lff-x-data, lbf-x-data, rff-x-data and rbf-x-data represent the cow's right hind foot, left hind foot, right front foot and left front corner respectively. The curve of the x coordinate with respect to the frame index during the cow's walking process is approximately a step wave. Taking the gray line in the figure as an example, the line represents the motion trajectory of the cow's left hind leg during the movement process. Each falling edge in the figure indicates that the cow has taken a step forward. There are four falling edges in the figure, indicating that the cow has taken four steps in this video. The straight part of the trajectory indicates that the cow's hoof is in the ground stage during the cow's walking process, and the hoof has not changed in displacement at the current stage. The time difference of the falling edges of the four hooves can be used to determine the sequence of the cow's footsteps during the walking process.

[0123] First, find the first falling edge that appears in the figure. The first falling edge that appears is the falling edge of the gray line, which is the left hind hoof. The second falling edge that follows is the left front hoof, followed by the right hind hoof, and finally the right front hoof. It can be obtained that the footstep sequence of the cow during one movement is left hind-left front-right hind-right front, which is consistent with the normal walking sequence of the cow. It can be judged that the footstep sequence of the cow during walking is normal.

[0124] Optionally, determining the step length of each step corresponding to each cow hoof according to the corresponding relationship diagram includes:

[0125] Get all the X-axis coordinate values ​​corresponding to the falling edge interval in the corresponding relationship diagram of each cow hoof;

[0126] Calculate the average value of the X-axis coordinate values ​​corresponding to the previous adjacent falling edge interval of the current falling edge interval, and calculate the average value of the X-axis coordinate values ​​corresponding to the current falling edge interval;

[0127] The difference between the average value of the X-axis coordinate values ​​corresponding to the last adjacent falling edge interval of the current falling edge interval and the average value of the X-axis coordinate values ​​corresponding to the current falling edge interval is used as the step size corresponding to the current falling edge interval.

[0128] Optionally, the calculation of the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points further includes:

[0129] When it is determined that the step sequence of the cow's walking process is normal, the time required for each step corresponding to each cow's hoof is determined according to the corresponding relationship diagram; the difference between the minimum value and the average step length of all cow's hooves in the detection period or the time required for each step corresponding to each cow's hoof is used as the judgment parameter value of the gait feature.

[0130] Preferably, determining the time required for each step corresponding to each cow hoof according to the corresponding relationship diagram includes:

[0131] Obtain the average frame index corresponding to the next falling edge and the average frame index corresponding to the previous falling edge in each step in the corresponding relationship diagram of each cow hoof;

[0132] Calculate the frame index difference between the average frame index corresponding to the next falling edge and the average frame index corresponding to the previous falling edge in each step;

[0133] The product of the frame index difference and the frame rate of the cow's walking video is taken as the time required for each step corresponding to each cow hoof.

[0134] Specifically, the stride length of a cow is defined as the distance generated by a hoof in each step, which can be equal to the X-axis coordinate value from one falling edge to the next falling edge in the figure. The calculation formula is shown in Formula 7. The difference dis of the X-axis coordinate values ​​is equal to the average value of the X-axis coordinate values ​​in the previous interval minus the average value of the X-axis coordinate values ​​in the next interval.

[0135]

[0136] where x i is the X-axis coordinate value of each frame in the previous cycle, y iis the X-axis coordinate value of each frame in the next cycle, and n and m are the frames in the corresponding cycle. When a normal cow walks, the walking stride length of each hoof is consistent, while the stride length of the lame leg of a lame cow becomes shorter during walking. The stride length of each hoof of the cow during the entire walking process is calculated, and the average value during the walking process is calculated:

[0137]

[0138] Where disLH, disLF, disRH, and disRF represent the step lengths of walking in the cycle. Then, the hoof with the shortest average step length and avg dis Compare and if the difference is less than the threshold value Φ, there is no lameness. If it is greater than the threshold value ΦTrue, there is a risk of lameness. The calculation is shown in formula 9.

[0139]

[0140] Furthermore, in the analysis of the cow's stride frequency, the cow's stride time is defined as the time interval required for the cow to take one step, which is calculated as follows:

[0141] t i =(avgt i+1 -avgt i )×fps·i=1,…,m (10)

[0142] Among them, avgt i+1 The average frame index avgt for the cow's hoof from lifting to falling in the next cycle i The average frame index of the cow's hoof from lifting to falling in the current cycle. The time from the cow lifting its foot to the cow lifting its foot again can be obtained by multiplying the difference of the frame index by the frame rate of the video. If the cow walks normally, the cow touches the bottom decisively during the walking process, showing a fast and crisp movement speed. If the cow's pace is slow during the movement, there may be a risk of lameness. By setting a threshold, if the pace is lower than the threshold, it is determined that there is a risk of lameness.

[0143] Optionally, the judgment parameter value of the scoring item includes: the judgment parameter value of the load condition; the characteristic part includes the hoof of the dairy cow;

[0144] The determining of the characteristic parameters corresponding to the characteristic parts includes:

[0145] Determine the Y-axis coordinate corresponding to each cow hoof in each frame image; the Y-axis coordinate value is the distance the cow hoof moves in the vertical direction in the frame image;

[0146] The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic points includes:

[0147] According to the Y-axis coordinates corresponding to each hoof in all frame images, the average value of the axis coordinates of each hoof is calculated;

[0148] Compare the average values ​​of the axis coordinates of all the hooves; determine the smallest of the average values ​​of the axis coordinates of all the hooves as the first parameter, and the average values ​​of the axis coordinates of the remaining hooves as the second parameter;

[0149] The difference between the second parameter and the first parameter is used as a judgment parameter value of the load condition.

[0150] Specifically, when walking, severely lame cows are different from other cows in that they are unwilling to walk and prefer to lie still. During walking, they cannot put weight on the ground due to serious hoof problems. This is reflected in the data that the average value of the y coordinate of the lame foot is higher than the average value of the y coordinate of other normal feet. The detailed calculation process is as follows:

[0151]

[0152] In formula 11, n is the number of frames in the video, y is i For the y coordinate value in each frame, calculate the average y coordinates of the four hooves of the cow. Then, take the smallest avg value as min y , except min y In addition, calculate the average avg of the avg values ​​of the other three hooves y , see formula 10

[0153]

[0154] Subtract min from the average value avgy y The value m is obtained. If the value of m is less than the threshold Φ, it can be determined that all four hooves of the cow can touch the ground. Otherwise, one hoof cannot touch the ground.

[0155] Optionally, the determination status of the scoring items includes: the determination result status of the head shaking condition, the determination result status of the weight bearing condition, the determination result status of the hunched back condition and the determination result status of the gait characteristics;

[0156] The determination status of all scoring items is determined according to the scoring conditions in the gait scoring rules of the dairy cow to obtain the gait score of the dairy cow, including:

[0157] The gait score of the cow is obtained based on the result of the head condition and the result of the weight condition; or

[0158] Obtaining a gait score of the cow according to the judgment result state of the head shaking condition and the judgment result state of the gait characteristics; or

[0159] The gait score of the dairy cow is obtained according to the judgment result state of the head shaking condition, the judgment result state of the gait characteristics and the judgment result state of the arched back condition.

[0160] Specifically, Figure 2 As shown in Table 1, the 5-point scoring system for dairy cows in the actual production process follows Table 1 (where the back is shown by the arched back, the walking state is shown by the symmetry, stride length and frequency; the head is shown by the head swing, and the load condition is shown by the y coordinate and frame index). A dairy cow with a gait score of 1 is a normal dairy cow. The cow has a straight back when standing and walking, no signs of uneven load between the two toes, no head swing when walking, and a normal gait; a dairy cow with a gait score of 2 is slightly lame and walks almost normally. The cow has a straight back when standing and walks. When the cow is lame, it shows a hunched back, no head swinging when walking, and the gait may be slightly uneven. Some cows walk with short strides, but there is no obvious sign of lameness; cows with a score of 3 show moderate lameness, showing a shortened stride of one or more limbs, the cows hunch their backs when standing and walking, and no signs of head swinging when walking; lame cows with a score of 4 have one or more limbs that are lame, with a hunched back when standing and walking and obvious head swinging when walking; severely lame cows with a score of 5 show an unwillingness to bear weight on the affected limbs on the basis of a score of 4.

[0161] Table 1 Cow walking status score table

[0162]

[0163] Based on this system and the results obtained from the analysis, a 5-point scoring system for dairy cow gait can be completed according to the following scoring process.

[0164] First, determine whether the cow has a head-shaking phenomenon. If the cow shakes its head while walking, the cow's gait score is 4 points or above, and then judge whether it can bear weight. If it can barely bear weight, it will be 4 points. If one hoof cannot touch the ground, it will be 5 points. If the cow does not have a head-shaking phenomenon, it will be judged by gait characteristics (if the cow does not have a head-shaking phenomenon, it will be judged by gait characteristics. Gait characteristics include gait symmetry and walking step length and frequency. When the symmetry changes or the step length and frequency of a single hoof change, it is judged that the cow's movement has changed, and the cow's gait score can be judged as 3 points.), if the gait is asymmetrical, it will be 3 points. Finally, it is judged by whether it has an arched back. If it has an arched back, it will be 2 points, otherwise it will be 1 point.

[0165] This application implements a five-point scoring system for dairy cows based on video footage shot by a camera, which saves costs while providing a more comprehensive analysis of possible problems with dairy cows; it takes into account the analysis of the cows' head shaking, arched back, and gait, which is consistent with the manual scoring process, that is, it can give a score and also provide a basis for the score. It saves costs while providing a more comprehensive evaluation of dairy cows. Among them, the movement of dairy cows in three-dimensional space is split into the relationship between the x coordinate and time t in two-dimensional space and the relationship between the y coordinate and time t for analysis, which more intuitively analyzes the trajectory of dairy cows during movement and improves the operability of the scoring system.

[0166] The present invention also provides a computer-readable storage medium on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned cow gait scoring method based on machine vision.

[0167] Those skilled in the art will understand that all or part of the steps in the method for implementing the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions for making a single-chip microcomputer, a chip or a processor (processor) perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0168] The optional embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, the technical scheme of the embodiments of the present invention can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0169] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A cow gait scoring method based on machine vision, characterized in that: The method comprises: Get videos of cows walking; Extracting characteristic parts in each frame of the cow's walking video and determining characteristic parameters corresponding to the characteristic parts; Calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic parts; According to the judgment parameter value of each scoring item in the cow gait, the judgment state of the scoring item is determined; wherein, The judgment parameter values ​​of the scoring items include: the judgment parameter value of the head shaking condition; the characteristic parts include the cow skull and the end of the cervical vertebra; The determining of the characteristic parameters corresponding to the characteristic parts includes: Determine the corresponding coordinates of the cow skull and the end of the cervical vertebrae in each frame of the image; The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic parts includes: According to the corresponding coordinates of the cow skull and the end of the cervical vertebra in each frame of the image, the Euclidean distance between the cow skull and the end of the cervical vertebra in each frame of the image is calculated; The Euclidean distances between the cow skull and the end of the cervical vertebrae in all frame images are sorted in time order to obtain a regression equation; Using the regression equation to determine the judgment parameter value of the head swing situation; The judgment status of the scoring items includes: the judgment result status of the head shaking condition, the judgment result status of the weight bearing condition, the judgment result status of the arched back condition and the judgment result status of the gait characteristics; the gait score of the cow is obtained according to the judgment result status of the head shaking condition and the judgment result status of the weight bearing condition; or Obtaining a gait score of the cow according to the judgment result state of the head shaking condition and the judgment result state of the gait characteristics; or The gait score of the dairy cow is obtained according to the judgment result status of the head shaking condition, the judgment result status of the gait characteristics and the judgment result status of the arched back condition.

2. The method for scoring cow gait based on machine vision according to claim 1, characterized in that: The judgment parameter values ​​of the scoring items include: the judgment parameter values ​​of the arched back; the characteristic parts include: the neck, the back and the hip angle; The determining of the characteristic parameters corresponding to the characteristic parts includes: Determine the corresponding coordinates of the neck, back and hip angles in each frame of the image; The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic parts includes: Calculate the cervical vertebrae curvature in each frame of image according to the corresponding coordinates of the neck, back and hip angle in each frame of image; the cervical vertebrae curvature is the angle between the coordinate line connecting the neck and hip angle and the coordinate line connecting the back and hip angle; Arrange the cervical vertebrae curvatures in all frame images in chronological order to determine the relationship between the cervical vertebrae curvature and time; The relationship between the cervical vertebra arc and time is used to determine the judgment parameter value of the kyphosis condition.

3. The method for scoring cow gait based on machine vision according to claim 2, characterized in that: The determination parameter values ​​of the kyphosis condition include the extreme difference value of the spinal curvature and the average value of the spinal curvature; Determining the determination state of each scoring item according to the determination parameter value of each scoring item in the cow gait includes: The judgment state of the hunchback condition is determined according to the comparison relationship between the extreme difference value of the spinal curvature and the average value of the spinal curvature and the corresponding threshold value.

4. The method for scoring cow gait based on machine vision according to claim 1, characterized in that: The judgment parameter values ​​of the scoring items include: judgment parameter values ​​of gait characteristics; the characteristic parts include the cow's hoof; The determining of the characteristic parameters corresponding to the characteristic parts includes: Determine the corresponding coordinates of each cow hoof in each frame image based on the reference coordinate system; the coordinates include an X-axis coordinate value and a Z-axis coordinate value; the Z-axis coordinate value is a frame index; the X-axis coordinate value is a distance that the cow hoof moves horizontally in the frame image; The step of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic parts includes: Establish a corresponding relationship diagram between the X-axis coordinate values ​​of all cattle hooves and the frame index; Obtain the time difference of the falling edge in the corresponding relationship diagram between the X-axis coordinate values ​​of all cow hooves and the frame index; determine the footstep sequence of the cow's walking process according to the time difference of the falling edge; When it is determined that the step sequence of the cow's walking process is normal, the step length of each step corresponding to each cow's hoof is determined according to the corresponding relationship diagram, and the average step length of each cow's hoof in the detection period is calculated according to the step length of each step corresponding to each cow's hoof, so as to obtain the average step length of all cow's hoofs in the detection period; The minimum value of the average stride lengths of all the cow hooves in the detection period is determined, and the difference between the minimum value and the average stride lengths of all the cow hooves in the detection period is calculated as the determination parameter value of the gait feature.

5. The method for scoring cow gait based on machine vision according to claim 4, characterized in that: Determining the step length of each step corresponding to each cow hoof according to the corresponding relationship diagram includes: Get all the X-axis coordinate values ​​corresponding to the falling edge interval in the corresponding relationship diagram of each cow hoof; Calculate the average value of the X-axis coordinate values ​​corresponding to the previous adjacent falling edge interval of the current falling edge interval, and calculate the average value of the X-axis coordinate values ​​corresponding to the current falling edge interval; The difference between the average value of the X-axis coordinate values ​​corresponding to the last adjacent falling edge interval of the current falling edge interval and the average value of the X-axis coordinate values ​​corresponding to the current falling edge interval is used as the step size corresponding to the current falling edge interval.

6. The method for scoring cow gait based on machine vision according to claim 4 or 5, characterized in that: The method of calculating the determination parameter value of each scoring item in the cow's gait based on the characteristic parameters corresponding to the characteristic parts also includes: When it is determined that the step sequence of the cow's walking process is normal, the time required for each step corresponding to each cow's hoof is determined according to the corresponding relationship diagram; the difference between the minimum value and the average step length of all cow's hooves in the detection period or the time required for each step corresponding to each cow's hoof is used as the judgment parameter value of the gait feature.

7. The method for scoring cow gait based on machine vision according to claim 6, characterized in that: Determining the time required for each step corresponding to each cow hoof according to the corresponding relationship diagram includes: Obtain the average frame index corresponding to the next falling edge and the average frame index corresponding to the previous falling edge in each step in the corresponding relationship diagram of each cow hoof; Calculate the frame index difference between the average frame index corresponding to the next falling edge and the average frame index corresponding to the previous falling edge in each step; The product of the frame index difference and the frame rate of the cow's walking video is taken as the time required for each step corresponding to each cow hoof.

8. The method for scoring cow gait based on machine vision according to claim 1, characterized in that: The judgment parameter values ​​of the scoring items include: the judgment parameter value of the load condition; the characteristic parts include the cow's hoof; The determining of the characteristic parameters corresponding to the characteristic parts includes: Determine the Y-axis coordinate corresponding to each cow hoof in each frame image; the Y-axis coordinate is the distance the cow hoof moves in the vertical direction in the frame image; the determination parameter value of each scoring item in the cow gait is calculated based on the characteristic parameters corresponding to the characteristic parts, including: According to the Y-axis coordinates corresponding to each hoof in all frame images, the average value of the axis coordinates of each hoof is calculated; Compare the average values ​​of the axis coordinates of all the hooves; determine the smallest of the average values ​​of the axis coordinates of all the hooves as the first parameter, and the average values ​​of the axis coordinates of the remaining hooves as the second parameter; The difference between the second parameter and the first parameter is used as a judgment parameter value of the load condition.

9. A computer-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the method for scoring cow gait based on machine vision as described in any one of claims 1 to 8.

Images