Livestock body condition scoring method, device, electronic device and storage medium
The problem of missing detection and missing frames in cow body condition scores was solved through point cloud image registration method, and high accuracy scores for different livestocks were achieved. It is suitable for cows and other livestocks, reducing the influence of human factors.
Patent Information
- Application Number
- CN202111422109.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-26
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-11-26
AI Technical Summary
The existing cow body condition scoring methods have problems with missing detection and missing frame tracking, resulting in reduced stability and are not suitable for other livestock.
Using the registration method based on point cloud images, the target point cloud images of livestock to be scored and the point cloud image template for registration are obtained. The body condition score is determined through the first registration result and the body ruler feature template to improve the automation of scores. and accuracy.
It improves the stability and accuracy of body condition scores, can be applied to different types of livestock, and reduces human interference.
Smart Images

Figure CN114266811B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a livestock body condition scoring method, device, electronic equipment and storage medium. Background Art
[0002] With the rapid development of animal husbandry, the study of the body condition of livestock is of vital importance. For example, for dairy cows, good body condition is a sign of high production and health of dairy cows. Cow Body Condition Scoring (BCS) is usually used to quantify the body condition of dairy cows. Normally, the body condition score of dairy cows adopts a 5-point system with a difference of 0.25. The higher the body condition score of the dairy cow, the fatter the dairy cow, and vice versa. When the score of the dairy cow is higher than 4.0 or lower than 2.5, it indicates that the dairy cow has serious health problems. Therefore, the body condition score of dairy cows can provide a reference for the management of cattle breeding, which is of great significance.
[0003] At present, the body condition score of dairy cows is mainly determined by a modeling method based on deep images. In this method, RGB-D videos are usually used to extract key frames in the video, and an improved single-stage multi-box prediction model (SingleShot MultiBox Detector, SSD) is used in the key frames to predict the cow's hip area. Then, the kernel correlation filter algorithm (KCF) is used to track the cow's hip area to obtain key frames that meet the requirements. Then, features are extracted from the RGB-D key frames of the cow's hip area and input into the classification model to finally obtain the cow's body condition score.
[0004] In the above method, since the body condition score of dairy cows is determined by combining SSD and KCF, SSD may have missed detection problems due to the influence of its own model structure, and KCF may have missed frame tracking problems due to the missed detection problems caused by SSD, which will reduce the stability of the method and further affect the accuracy of the body condition score of dairy cows. Moreover, since dairy cows are very different from other livestock, the body condition scoring method applicable to dairy cows in the prior art is not well applicable to other livestock. Summary of the invention
[0005] The present invention provides a livestock body condition scoring method, device, electronic equipment and storage medium, which are used to solve the defects in the prior art.
[0006] The present invention provides a livestock body condition scoring method, comprising:
[0007] Acquire a target point cloud image of the livestock to be scored;
[0008] Registering the target point cloud image with each key part point cloud image template set to obtain a first registration result;
[0009] The body condition score of the livestock to be scored is determined based on the first registration result and the standard body condition score corresponding to the point cloud image template set of each key part.
[0010] According to a livestock body condition scoring method provided by the present invention, the body condition score of the livestock to be scored is determined based on the first registration result and the standard body condition score corresponding to the point cloud image template set of each key part, including:
[0011] If the first registration result is successful registration, the body condition score of the livestock to be scored is determined based on the first standard body condition score corresponding to the first key part point cloud image template set; the first key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image;
[0012] If the first registration result is that the registration is unsuccessful, the target body size features of the livestock to be scored are determined based on the target point cloud image, the target body size features are registered with each body size feature template set to obtain a second registration result, and the body condition score of the livestock to be scored is determined based on the first registration result, the second registration result and the standard body condition scores corresponding to each body size feature template set.
[0013] According to a livestock body condition scoring method provided by the present invention, the unsuccessful registration includes partial successful registration and failed registration;
[0014] Accordingly, determining the body condition score of the livestock to be scored based on the first registration result, the second registration result, and the standard body condition score corresponding to each body size feature template set includes:
[0015] If the first registration result is a partial registration success, and the second registration result is a registration success, then determining the body condition score of the livestock to be scored based on the average of the second standard body condition score corresponding to the second key part point cloud image template set and the third standard body condition score corresponding to the target body size feature template set;
[0016] The second key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image part, and the target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
[0017] According to a livestock body condition scoring method provided by the present invention, the target body size features include multiple body size feature templates, each of the body size feature template sets includes multiple body size feature templates, and the multiple body size feature templates correspond to the target body size features one by one;
[0018] Accordingly, registering the target body size feature with each body size feature template set to obtain a second registration result includes:
[0019] For any body size feature template set in the body size feature template sets, calculate a relative error between each body size feature template in the body size feature template set and a corresponding target body size feature, and determine a registration result corresponding to the body size feature template set based on the relative error;
[0020] The second registration result is determined based on the registration results corresponding to each body size feature template set.
[0021] According to a livestock body condition scoring method provided by the present invention, the body condition score of the livestock to be scored is determined based on the first standard body condition score corresponding to the first key part point cloud image template set, including:
[0022] Determining the target body size feature of the livestock to be scored based on the target point cloud image, and registering the target body size feature with each body size feature template set to obtain a second registration result;
[0023] The body condition score of the livestock to be scored is determined based on the first standard body condition score, the second registration result, and the standard body condition scores corresponding to each body size feature template set.
[0024] According to a livestock body condition scoring method provided by the present invention, the body condition score of the livestock to be scored is determined based on the first standard body condition score, the second registration result, and the standard body condition score corresponding to each body size feature template set, including:
[0025] If the second registration result is successful registration, and the difference between the first standard body condition score and the third standard body condition score corresponding to the target body size feature template set is within a preset range, then the body condition score of the livestock to be scored is determined based on the average value of the first standard body condition score and the third standard body condition score;
[0026] The target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
[0027] A livestock body condition scoring method according to the present invention further includes: obtaining target basic information of the livestock to be scored;
[0028] Accordingly, registering the target point cloud image with each key part point cloud image template set to obtain a first registration result includes:
[0029] Determine the error value corresponding to the target posture information based on the target point cloud registration model corresponding to the target basic information;
[0030] Determining the first registration result based on the error value corresponding to the target posture information;
[0031] Among them, the target pose information is the pose information between the target point cloud image and the target key part point cloud image template, and the target key part point cloud image template is the key part point cloud image template in the key part point cloud image template set corresponding to the target basic information; the target point cloud registration model is trained by the bulldozer distance loss function between the template features of the target key part point cloud image template and the corresponding template estimation features, and the template estimation features are estimated based on the sample pose information, and the sample pose information is the pose information between the sample point cloud image of the sample livestock corresponding to the target basic information and the target key part point cloud image template.
[0032] According to a livestock body condition scoring method provided by the present invention, the template estimation feature is estimated based on the following method:
[0033] Extracting features from each region in the sample point cloud image and the target key part point cloud image template to obtain regional features of each region and template features of the target key part point cloud image template;
[0034] Calculating the distance between the template feature and the regional features of each region, and selecting a preset number of regional features with smaller distances as target regional features;
[0035] Splicing the template feature with the target area feature to obtain a splicing feature, and determining the sample pose information based on the splicing feature;
[0036] The template estimation feature is determined based on the target area feature and the sample pose information.
[0037] According to a livestock body condition scoring method provided by the present invention, the step of obtaining a target point cloud image of the livestock to be scored comprises:
[0038] Acquire local point cloud images of the livestock to be scored at multiple viewing angles;
[0039] Inputting the local point cloud images under the multiple viewing angles into a three-dimensional reconstruction model to obtain the target point cloud image output by the three-dimensional reconstruction model;
[0040] The three-dimensional reconstruction model is trained based on sample local point cloud images from multiple perspectives.
[0041] According to a livestock body condition scoring method provided by the present invention, the key parts of the livestock to be scored include at least one of the ischium, hip joint, clavicle, short ribs, sacral ligament and coccygeal ligament;
[0042] Correspondingly, the key part point cloud image template sets corresponding to the respective standard body condition scores include key part point cloud image templates corresponding to the respective key parts of the sample livestock corresponding to the respective standard body condition scores.
[0043] The present invention also provides a livestock body condition scoring device, comprising:
[0044] An image acquisition module, used to acquire a target point cloud image of the livestock to be scored;
[0045] A registration module, used to register the target point cloud image with each key part point cloud image template set to obtain a first registration result;
[0046] The body condition score determination module is used to determine the body condition score of the livestock to be scored based on the first registration result and the standard body condition score corresponding to the point cloud image template set of each key part.
[0047] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the livestock body condition scoring methods described above are implemented.
[0048] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the livestock body condition scoring methods described above are implemented.
[0049] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the livestock body condition scoring methods described above are implemented.
[0050] The livestock body condition scoring method, device, electronic device and storage medium provided by the present invention first obtain a target point cloud image of the livestock to be scored; then align the target point cloud image with each key part point cloud image template set to obtain a first alignment result; finally, based on the first alignment result and the standard body condition score corresponding to each key part point cloud image template set, the body condition score of the livestock to be scored is determined. The body condition score of the livestock to be scored is determined by image alignment, without the need to introduce human operation, which can improve the automation of scoring and eliminate interference caused by human factors. Moreover, the method does not need to use SSD or KCF, avoids the occurrence of missed detection problems or missed frame tracking problems, has strong stability, and improves the accuracy of body condition scoring. Moreover, the method can be applied not only to dairy cows, but also to other livestock, and has strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings in the following description without paying any creative work.
[0052] Figure 1 It is a schematic flow chart of the livestock body condition scoring method provided by the present invention;
[0053] Figure 2 It is a structural schematic diagram of the livestock body condition scoring device provided by the present invention;
[0054] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0056] At present, the research on the body condition of livestock is limited to the body condition of dairy cows, and the body condition of other livestock has not been studied. Moreover, in the prior art, the cow body condition score (BCS) is usually used to quantify the cow body condition. There are mainly three methods for determining the cow body condition score:
[0057] 1) Experienced staff members score the body condition of dairy cows based on certain standards to obtain the body condition score of dairy cows.
[0058] 2) The 3D features of the single-view 3D images are extracted and dimensionally reduced, and 2D target detection algorithms such as faster rcnn are used to obtain the body condition score of the cow.
[0059] 3) Extract key frames from the RGB-D video, and use the improved SSD in the key frames to predict the cow's hip area. Then use KCF to track the cow's hip area to obtain key frames that meet the requirements. Finally, extract features from the RGB-D key frames of the cow's hip area at various viewing angles, and input the extracted features into the classification model to obtain the cow's body condition score.
[0060] In the above method, 1) due to the reliance on manual scoring, not only the labor cost is increased, but also a large amount of cow scoring work will increase the pressure of the staff. In addition, after a long period of scoring, the scoring results are prone to deviations, resulting in inaccurate cow body condition scores. 2) It is aimed at a single-view 3D image, and it is impossible to obtain comprehensive information about the cow, and the use of a 2D target detection algorithm for 3D images to score the cow's body condition will greatly increase the scoring error. 3) The use of RGB-D video only adds depth information, but only considers the cow's hip area, which is too one-sided. Moreover, since there is a problem of missed detection when using SSD for detection, there is also a problem of missed frames when using KCF for tracking, which will lead to a reduction in the stability of the method, thereby affecting the accuracy of the cow's body condition score. Moreover, since there are significant differences between cows and other livestock, the body condition scoring method applicable to cows in the prior art is not well applicable to other livestock.
[0061] To this end, an embodiment of the present invention provides a livestock body condition scoring method to solve the above technical problems.
[0062] Figure 1 FIG. 1 is a flow chart of a livestock body condition scoring method provided in an embodiment of the present invention, such as Figure 1 As shown, the method includes:
[0063] S1, obtaining a target point cloud image of the livestock to be rated;
[0064] S2, registering the target point cloud image with each key part point cloud image template set to obtain a first registration result;
[0065] S3, determining the body condition score of the livestock to be scored based on the first registration result and the standard body condition score corresponding to the point cloud image template set of each key part.
[0066] Specifically, the livestock body condition scoring method provided in the embodiment of the present invention is implemented by a livestock body condition scoring device, which can be configured in a server, which can be a local server or a cloud server. The local server can be a computer, a tablet computer, a smart phone, etc., which is not specifically limited in the embodiment of the present invention.
[0067] First, step S1 is performed to obtain a target point cloud image of the livestock to be scored, where the livestock to be scored refers to the livestock whose body condition score needs to be determined. The livestock involved in the embodiment of the present invention may be mammalian livestock such as dairy cows, beef cattle, and goats. The target point cloud image refers to the overall three-dimensional point cloud image of the livestock to be scored, which can be obtained by performing an all-round scan of the livestock to be scored, or by other methods, which is not specifically limited in the embodiment of the present invention.
[0068] The target point cloud image may include target key parts of the livestock to be scored, which refers to the body parts that affect the body condition score of the livestock to be scored, and may be one or more, for example, at least one of the ischium, hip joint, clavicle, short ribs, sacral ligament and coccygeal ligament, which is not specifically limited in the embodiment of the present invention. It should be noted that the target key parts may not include the head of the livestock that has no effect on the body condition score of the livestock to be scored.
[0069] Then, step S2 is performed to register the target point cloud image with each key part point cloud image template set to obtain a first registration result. Each key part point cloud image template set can be stored in a key part point cloud image template library, and each key part point cloud image template set contains key part point cloud image templates corresponding to each target key part. The key part point cloud image templates contained in each key part point cloud image template set correspond to the target key part one by one.
[0070] The key part point cloud image template refers to the point cloud image of the key part of the livestock sample with a known body condition score. Each key part point cloud image template set corresponds to a standard body condition score, that is, each key part point cloud image template set is a set of key part point cloud images corresponding to the key parts of the sample livestock with a certain standard body condition score. The known body condition score of the livestock sample is the standard body condition score corresponding to the key part point cloud image template set, and the standard body condition score can be obtained through expert annotation, which is not specifically limited in the embodiments of the present invention.
[0071] Since livestock with different basic information have different body condition scores, each standard body condition score corresponds to one basic information, that is, the standard body condition score, basic information and key part point cloud image template set are one-to-one corresponding. Here, the basic information may include at least one of the livestock age, livestock breed and other information.
[0072] The process of registering the target point cloud image with each key part point cloud image template set is to determine the number of key part point cloud image templates in each key part point cloud image template set that match the target key part. The first registration result may include successful registration and unsuccessful registration, and unsuccessful registration may include partial successful registration and failed registration.
[0073] First, determine a target key part point cloud image template set, which refers to a key part point cloud image template set containing the largest number of key part point cloud image templates whose matching results with the corresponding target key parts are successfully matched.
[0074] If the number of key part point cloud image templates in the target key part point cloud image template set whose matching results are successfully matched with the corresponding target key parts is greater than a first preset number, the first registration result is considered to be a successful registration, and the registration object is the target key part point cloud image template set; if the number of key part point cloud image templates in the target key part point cloud image template set whose matching results are successfully matched with the corresponding target key parts is greater than a second preset number and is less than or equal to the first preset number, the first registration result is considered to be a partially successful registration; if the number of key part point cloud image templates in the target key part point cloud image template set whose matching results are successfully matched with the corresponding target key parts is less than or equal to the second preset number, the first registration result is considered to be a failed registration.
[0075] Among them, the first preset number may be 2 / 3 of the total number of all key parts, and the second preset number may be 1 / 3 of the total number of all key parts. For example, if all key parts include ischium, hip joint, clavicle, short ribs, sacral ligament and coccygeal ligament, and the total number is 6, then the first preset number may be 4, and the second preset number may be 2. When the first registration result is successful registration, the number of key part point cloud image templates in the target key part point cloud image template set whose matching result with the corresponding target key part is a successful match may be 5 or 6. When the first registration result is a partial successful registration, the number of key part point cloud image templates in the target key part point cloud image template set whose matching result with the corresponding target key part is a successful match may be 3 or 4. When the first registration result is a failed registration, the number of key part point cloud image templates in the target key part point cloud image template set whose matching result with the corresponding target key part is a successful match may be any one of 0-2.
[0076] In particular, when there is only one key part, if the key part point cloud image template in the target key part point cloud image template set successfully matches the target key part, the first registration result can be considered as a successful registration; if the key part point cloud image template in the target key part point cloud image template set fails to match the target key part, the first registration result can be considered as a failed registration. That is, when there is only one key part, the first registration result only includes two categories: successful registration and failed registration.
[0077] Finally, step S3 is performed to determine the body condition score of the livestock to be scored based on the first registration result and the standard body condition score corresponding to each key part point cloud image template set. Here, the body condition score of the livestock to be scored can be determined based on the specific category of the first registration result and in combination with the standard body condition score corresponding to each key part point cloud image template set. For example, if the first registration result is a successful registration, the standard body condition score corresponding to the key part point cloud image template set that is successfully registered with the target point cloud image can be directly used as the body condition score of the livestock to be scored, or the body condition score of the livestock to be scored can be comprehensively determined in combination with other features in the target point cloud image. If the first registration result is an unsuccessful registration, the body condition score of the livestock to be scored can only be determined by combining other features in the target point cloud image.
[0078] The livestock body condition scoring method provided in an embodiment of the present invention first obtains a target point cloud image of the livestock to be scored; then aligns the target point cloud image with a set of point cloud image templates of each key part to obtain a first alignment result; finally, based on the first alignment result and the standard body condition score corresponding to the set of point cloud image templates of each key part, the body condition score of the livestock to be scored is determined. The body condition score of the livestock to be scored is determined by image alignment, without the need for manual operation, which can improve the automation of scoring and eliminate interference caused by human factors. Moreover, the method does not need to use SSD or KCF, avoids the occurrence of missed detection problems or missed frame tracking problems, has strong stability, and improves the accuracy of body condition scoring. Moreover, the method is not only applicable to dairy cows, but also to other livestock, and has strong adaptability.
[0079] On the basis of the above-mentioned embodiment, in the livestock body condition scoring method provided in the embodiment of the present invention, the point cloud image template set of each key part can be determined based on the following method:
[0080] Obtain sample point cloud images of sample livestock with different basic information;
[0081] Extracting key part point cloud images from the sample point cloud images and annotating them to obtain a key part point cloud image template set;
[0082] A standard body condition score of the sample livestock is determined, and the standard body condition score is associated with the key part point cloud image template set.
[0083] Specifically, in the embodiment of the present invention, the sample livestock with different basic information refers to sample livestock of different breeds and ages. The sample point cloud image refers to the overall three-dimensional point cloud image of the sample livestock, which can be obtained by omnidirectional scanning of the sample livestock or by other methods, which is not specifically limited in the embodiment of the present invention.
[0084] Then, the key parts in the sample point cloud image are extracted and segmented to obtain the key part point cloud image, which can then be manually annotated by experts, that is, the key part point cloud image is associated with the part name to obtain the key part point cloud image template carrying the part name.
[0085] Finally, the standard body condition score of the sample livestock can also be manually determined by experts, and the standard body condition score can be associated with the key part point cloud image template, that is, the corresponding relationship between the standard body condition score and the key part point cloud image template can be determined.
[0086] It can be understood that the above process of determining the set of point cloud image templates of each key part is the process of constructing a key part point cloud image template library.
[0087] In the embodiment of the present invention, the point cloud image template set of each key part can be determined independently from the scoring process, thereby improving the scoring efficiency.
[0088] On the basis of the above embodiment, the livestock body condition scoring method provided in the embodiment of the present invention, the body condition score of the livestock to be scored is determined based on the first registration result and the standard body condition score corresponding to the point cloud image template set of each key part, including:
[0089] If the first registration result is successful registration, the body condition score of the livestock to be scored is determined based on the first standard body condition score corresponding to the first key part point cloud image template set; the first key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image;
[0090] If the first registration result is that the registration is unsuccessful, the target body size features of the livestock to be scored are determined based on the target point cloud image, the target body size features are registered with each body size feature template set to obtain a second registration result, and the body condition score of the livestock to be scored is determined based on the first registration result, the second registration result and the standard body condition scores corresponding to each body size feature template set.
[0091] Specifically, in an embodiment of the present invention, when determining the body condition score of the livestock to be scored by using the first registration result and each standard body condition score, if the first registration result is a successful registration, the first key part point cloud image template set can be determined first, that is, the key part point cloud image template set that is successfully registered with the target point cloud image. Then, according to the first standard body condition score corresponding to the first key part point cloud image template, the body condition score of the livestock to be scored is determined. Here, the first standard body condition score is the standard body condition score corresponding to the first key part point cloud image template, and "first" only serves to distinguish. In an embodiment of the present invention, the first standard body condition score can be directly used as the body condition score of the livestock to be scored, or the first standard body condition score and other features in the target point cloud image can be combined to comprehensively determine the body condition score of the livestock to be scored, which is not specifically limited in the embodiment of the present invention.
[0092] If the first registration result is that the registration is unsuccessful, in order to ensure the accuracy of the body condition score, it is necessary to determine the target body size features of the livestock to be scored based on the target point cloud image. The target body size features refer to the body size features of the livestock to be scored, which can be two-dimensional features and can be obtained by extracting features from the target point cloud image. Here, the body size features refer to the body size features of the livestock that affect the body condition score. The body size features can be one or more, for example, at least one of the abdominal contour area and length, the hip area and length, the back contour area and length, the hip width, the back width, and the body height.
[0093] After determining the target body size feature, the target body size feature can be registered with each body size feature template set to obtain a second registration result. Each body size feature template set can be stored in a body size feature template library, and each body size feature template set contains body size feature templates corresponding to each body size feature. The number of body size feature templates contained in each body size feature template set can be one or more.
[0094] The body size feature template refers to the body size feature of a livestock sample with a known body condition score. Each body size feature template set corresponds to a standard body condition score, that is, each body size feature template set is a set of body size features of sample livestock with a certain standard body condition score. The known body condition score of the livestock sample is the standard body condition score corresponding to the body size feature template set, and the standard body condition score can also be obtained through expert annotation, which is not specifically limited in the embodiments of the present invention.
[0095] Since livestock with different basic information have different body condition scores, the standard body condition score, basic information and body size feature template set are in one-to-one correspondence.
[0096] The process of registering the target body size feature with each body size feature template set is to determine the number of body size feature templates in each body size feature template set that match the target body size feature. The second registration result may include two types: successful registration and failed registration.
[0097] To determine the second registration result, a target body size feature template set may be determined first, wherein the target body size feature template set refers to a body size feature template set containing the largest number of body size feature templates having successfully matched results with the corresponding target body size features.
[0098] If the number of body size feature templates in the target body size feature template set whose matching results with the corresponding target body size features are successfully matched is greater than or equal to a third preset number, the second registration result is considered to be a successful registration, and the registration object is the target body size feature template set; if the number of body size feature templates in the target body size feature template set whose matching results with the corresponding target body size features are successfully matched is less than the third preset number, the second registration result is considered to be a failed registration.
[0099] The third preset number may be 2 / 3 of the total number of all target body size features. For example, if all target body size features include the contour area and length of the abdomen, the contour area and length of the buttocks, the contour area and length of the back, the hip width, the back width, and the body height, and the total number is 9, then the third preset number may be 6. When the second registration result is a successful registration, the number of body size feature templates in the target body size feature template set whose matching result with the corresponding target body size feature is a successful match may be 6, 7, 8, or 9. When the second registration result is a failed registration, the number of body size feature templates in the target body size feature template set whose matching result with the corresponding target body size feature is a successful match may be any one of 0-5.
[0100] Finally, the body condition score of the livestock to be scored is determined according to the first registration result, the second registration result, and the standard body condition score corresponding to each body size feature template set. Here, the body condition score of the livestock to be scored can be determined according to the specific categories of the first registration result and the second registration result, and in combination with the standard body condition score corresponding to each body size feature template set. For example, if the first registration result is a partial registration success and the second registration result is a successful registration, the standard body condition score corresponding to the key part point cloud image template set that is partially successfully registered with the target point cloud image (i.e., the target key part point cloud image template set at this time) can be determined, and then the body condition score of the livestock to be scored is determined based on the standard body condition score and the body size feature template set that is successfully registered with the target body size feature (i.e., the target body size feature template set at this time).
[0101] If the first registration result is a registration failure and the second registration result is a registration success, the standard body condition score corresponding to the target body size feature template that is successfully registered with the target body size feature (that is, the target body size feature template set at this time) can be directly used as the body condition score of the livestock to be scored.
[0102] If the first registration result is a partial registration success and the second registration result is a registration failure, or if the first registration result is a registration failure and the second registration result is a registration failure, the body condition score of the livestock to be scored cannot be determined.
[0103] In the embodiment of the present invention, different methods are used to determine the body condition score of the livestock to be scored according to the specific types of the first registration result and the second registration result, which can be more targeted and ensure the successful acquisition of the body condition score of the livestock to be scored.
[0104] On the basis of the above-mentioned embodiment, in the livestock body condition scoring method provided in the embodiment of the present invention, each body size feature template set is constructed based on the following method:
[0105] 1) Obtain sample point cloud images of sample livestock with different basic information.
[0106] 2) Construct livestock coordinate system.
[0107] The right-handed livestock coordinate system can be constructed with the head of the sample livestock as the coordinate origin, the direction of the cattle body as the X-axis, the direction in the plane where the cattle body is located and perpendicular to the direction of the cattle body as the Y-axis, and the direction perpendicular to the plane where the cattle body is located as the Z-axis. In the embodiment of the present invention, the positive direction of the X-axis can be the direction from the cattle head to the cattle tail, and the positive direction of the Z-axis can be perpendicular to the plane where the cattle body is located and upward.
[0108] 3) Project the sample point cloud image along the X-axis to determine the point cloud contour of the hip area of the sample livestock.
[0109] After projection, the projection contour of the maximum area and the corresponding contour point cloud can be obtained. The contour point cloud is used as the abdominal contour point cloud of the sample livestock. According to the abdominal contour point cloud of the sample livestock, the coordinate position of the abdominal point cloud can be obtained. When the orientation of the sample livestock is set, the hip area of the sample livestock can be determined after the coordinate position of the abdominal point cloud is obtained. Then, the point cloud contour of the hip area of the sample livestock is calculated. Among them, the hip area of the sample livestock is the edge point of the livestock hip as the starting point, and the point with a preset distance value in the opposite direction of the X-axis is the end point. The cross section corresponding to the end point is the area surrounded by the edge of the hip of the sample livestock. Among them, the preset distance value can be a hip length value, which can be an empirical value.
[0110] 4) Project the sample point cloud image along the Z axis to determine the point cloud contour of the back area of the sample livestock.
[0111] After projection, the point cloud contour of the back area of the sample livestock can be obtained.
[0112] The corresponding area and length, hip width, back width, and body height can be calculated through the three body size features of the abdomen contour point cloud, the hip area contour point cloud, and the back contour point cloud.
[0113] 5) According to the basic information of the sample livestock, determine the set of body size feature templates corresponding to each basic information.
[0114] It can be understood that the above process of determining each body size feature template set is the process of establishing a body size feature template library.
[0115] In the embodiment of the present invention, each body size feature template set can be determined independently from the scoring process, thereby improving the scoring efficiency.
[0116] On the basis of the above-mentioned embodiment, in the livestock body condition scoring method provided in the embodiment of the present invention, the unsuccessful registration includes partial successful registration and failed registration;
[0117] Accordingly, determining the body condition score of the livestock to be scored based on the first registration result, the second registration result, and the standard body condition score corresponding to each body size feature template set includes:
[0118] If the first registration result is a partial registration success and the second registration result is a registration success, then the body condition score of the livestock to be scored is determined based on the average value of the second standard body condition score corresponding to the second key part point cloud image template set and the third standard body condition score corresponding to the target body size feature template set;
[0119] The second key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image part, and the target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
[0120] Specifically, in an embodiment of the present invention, when the body condition score of the livestock to be scored is determined according to the first registration result, the second registration result and each standard body condition score, if the first registration result is a partial registration success and the second registration result is a registration success, the standard body condition score corresponding to the key part point cloud image template set (i.e., the target key part point cloud image template set at this time) with which the target point cloud image is partially successfully registered can be first determined, and the standard body condition score is recorded as the second standard body condition score, and the standard body condition score corresponding to the body size feature template set (i.e., the target body size feature template set at this time) with which the target two-dimensional body size feature is successfully registered is determined, and the standard body condition score is recorded as the third standard body condition score, and then the average value between the second standard body condition score and the third standard body condition score is calculated, and the body condition score of the livestock to be scored is determined according to the average value.
[0121] It should be noted that, since the two objects of registration, the target point cloud image and the body size feature, are different, the second standard body condition score and the third standard body condition score obtained are not necessarily the same. In the embodiment of the present invention, the average value of the two can be calculated, and then the body condition score of the livestock to be scored can be determined by the average value. For example, the average value can be directly used as the body condition score of the livestock to be scored, and this is not specifically limited in the embodiment of the present invention.
[0122] In the embodiment of the present invention, when the first registration result is a partial registration success and the second registration result is a registration success, the body condition score of the livestock to be scored is determined by the average value between the second standard body condition score and the third standard body condition score, and the registration of the target point cloud image and the registration of the target body size features are combined to make the body condition score of the livestock to be scored more accurate.
[0123] On the basis of the above-mentioned embodiment, in the livestock body condition scoring method provided in the embodiment of the present invention, the target two-dimensional body size features include multiple body size feature templates, each of the body size feature template sets includes multiple body size feature templates, and the multiple body size feature templates correspond to the target body size features one by one;
[0124] Accordingly, registering the target body size feature with each body size feature template set to obtain a second registration result includes:
[0125] For any body size feature template set in the body size feature template sets, calculate a relative error between each body size feature template in the body size feature template set and a corresponding target body size feature, and determine a registration result corresponding to the body size feature template set based on the relative error;
[0126] The second registration result is determined based on the registration results corresponding to each body size feature template set.
[0127] Specifically, in the embodiment of the present invention, the target body size features may include multiple features, such as the contour area and length of the abdomen, the contour area and length of the buttocks, the contour area and length of the back, the hip width, the back width, and the body height. Correspondingly, the body size feature template may also include multiple features, and considering that the basic information of the livestock will affect the body size features, there is a one-to-one correspondence between the body size feature template set, the basic information, and the standard body condition score.
[0128] When the second registration result is obtained through registration, for any body size feature template set in each body size feature template set, a relative error between each body size feature template in the any body size feature template set and the corresponding target body size feature is calculated.
[0129] Here, the relative error between each body size feature template in any body size feature template set and the corresponding target body size feature can be calculated by the following formula:
[0130]
[0131] Where err is the relative error between any body size feature template in the body size feature template set and the corresponding target body size feature, P T is any body size feature template in the set of body size feature templates, P E is the target body size feature corresponding to any body size feature template.
[0132] There is a relative error between each body size feature template in any body size feature template set and the corresponding target body size feature. According to the relative error between each body size feature template in any body size feature template set and the corresponding target body size feature, the registration result corresponding to any body size feature template set can be determined.
[0133] For example, an error threshold may be set. If the relative error between any body size feature template and the corresponding target body size feature is less than the relative error of the error threshold, the registration result corresponding to the body size feature template may be considered to be a successful registration. If there is no relative error less than the error threshold, the registration result corresponding to the body size feature template may be considered to be a failed registration. The error threshold may be set as required, for example, may be set to be less than or equal to 10%.
[0134] Finally, based on the registration results corresponding to each body size feature template set, the second registration result is determined. Since there are 9 target body size features in total, and each body size feature template set contains 9 body size feature templates, the target body size feature template set can be determined first, which refers to the body size feature template set containing the largest number of body size feature templates that have successfully matched the corresponding target body size feature.
[0135] If the number of body size feature templates in the target body size feature template set whose matching results with the corresponding target body size features are successfully matched is greater than or equal to a third preset number, the second registration result is considered to be a successful registration, and the registration object is the target body size feature template set; if the number of body size feature templates in the target body size feature template set whose matching results with the corresponding target body size features are successfully matched is less than the third preset number, the second registration result is considered to be a failed registration.
[0136] In the embodiment of the present invention, the second registration result is determined by calculating the relative error between each body size feature template in each body size feature template set and the corresponding target body size feature, so that the accuracy of the second registration result can be improved.
[0137] On the basis of the above-mentioned embodiment, the livestock body condition scoring method provided in the embodiment of the present invention, the body condition score of the livestock to be scored is determined based on the first standard body condition score corresponding to the first key part point cloud image template set, including:
[0138] Determining the target body size feature of the livestock to be scored based on the target point cloud image, and registering the target body size feature with each body size feature template set to obtain a second registration result;
[0139] The body condition score of the livestock to be scored is determined based on the first standard body condition score, the second registration result, and the standard body condition scores corresponding to each body size feature template set.
[0140] Specifically, in the embodiment of the present invention, when the first registration result is a successful registration, the body condition score of the livestock to be scored can be directly determined according to the first standard body condition score. At this time, the target body size feature of the livestock to be scored can be determined according to the target point cloud image. Then, the target body size feature is registered with each body size feature template set to obtain a second registration result. Here, the method for obtaining the second registration result by registration is consistent with the method provided in the above embodiment, and will not be repeated here.
[0141] Finally, the body condition score of the livestock to be scored is determined according to the first standard body condition score, the second registration result, and the standard body condition score corresponding to each body size feature template set. That is, the target body size feature template set can be determined first in combination with the specific type of the second registration result, that is, the body size feature template set that is successfully registered with the target body size feature is determined. Then, the body condition score of the livestock to be scored is determined according to the standard body condition score corresponding to the target body size feature template set (recorded as the third standard body condition score) and the first standard body condition score.
[0142] In the embodiment of the present invention, when the first registration result is successful registration, the body condition score of the livestock to be scored can also be comprehensively determined in combination with the target body size features obtained in the target point cloud image, which can greatly improve the accuracy of the scoring.
[0143] On the basis of the above-mentioned embodiment, the livestock body condition scoring method provided in the embodiment of the present invention, wherein the target point cloud image is registered with the point cloud image template set of each key part to obtain a first registration result, comprises:
[0144] Based on the point cloud registration model, determine the error value corresponding to the target pose information;
[0145] Determining the first registration result based on the error value corresponding to the target posture information;
[0146] Among them, the target pose information is the pose information between the target point cloud image and the key part point cloud image template in the key part point cloud image template set; the point cloud registration model is trained by using the bulldozer distance loss function between the template features of the key part point cloud image template in the key part point cloud image template set and the corresponding template estimation features, the template estimation features are estimated based on the sample pose information, and the sample pose information is the pose information between the sample point cloud image of the sample livestock and the key part point cloud image template in the key part point cloud image template set.
[0147] Specifically, in an embodiment of the present invention, when the target point cloud image is registered with a set of point cloud image templates of key parts to obtain a first registration result, the error value corresponding to the target posture information can be first determined based on the point cloud registration model.
[0148] The point cloud registration model can be used to determine the pose information between the target point cloud image and the key part point cloud image templates in each key part point cloud image template set, that is, the target pose information and the error value corresponding to the target pose information. Here, the pose information refers to the rotation matrix, which can be understood as the rotation matrix between the target point cloud image and the features in the key part point cloud image templates in each key part point cloud image template set. For example, the target pose information is the rotation matrix between the target point cloud image and the key part point cloud image templates in each key part point cloud image template set.
[0149] In an embodiment of the present invention, the target point cloud image and each key part point cloud image template set can be respectively input into the point cloud registration model, and the point cloud registration model can output the pose information between the target point cloud image and each key part point cloud image template set, that is, the target pose information and its corresponding error value. In addition, each key part point cloud image template set can be pre-stored in the point cloud registration model, and then the target point cloud image can be input into the point cloud registration model to obtain the target pose information and its corresponding error value.
[0150] It should be noted that the point cloud registration model can be obtained through training, so it can directly determine the error value corresponding to the target pose information, and it is not necessary to determine the key part features in the target point cloud image during the determination process.
[0151] In the process of training the point cloud registration model, the Earth Mover's Distance (EMD) loss function (loss) between the template features of the key part point cloud image template in the key part point cloud image template set and the corresponding template estimation features can be used. The EMD loss calculation formula is:
[0152]
[0153] in, Estimated features for the template, P T is the template feature, for With P T The mapping function between .
[0154] Each key part point cloud image template in each key part point cloud image template set corresponds to a template estimation feature, and the template estimation feature can be estimated using sample pose information, and the sample pose information is the pose information between the sample point cloud image of the sample livestock and the key part point cloud image template in each key part point cloud image template set. The sample pose information is obtained based on the point cloud registration model. For example, the point cloud registration model can pre-store each key part point cloud image template set, and then the sample point cloud image of the sample livestock is input into the point cloud registration model and output by the point cloud registration model. Alternatively, the sample point cloud image of the sample livestock and each key part point cloud image template set can be input into the point cloud registration model and output by the point cloud registration model. While the point cloud registration model outputs the sample pose information, it can also output the error value corresponding to the sample pose information.
[0155] When using sample pose information to estimate template estimation features, since the sample pose information is the pose information between the sample point cloud image and the key part point cloud image template in each key part point cloud image template set, it can be used to characterize the association between the key part features in the sample point cloud image and the template features in the corresponding key part point cloud image template in each key part point cloud image template set. Therefore, the template estimation features can be determined by combining the key part features in the sample point cloud image.
[0156] After determining the error value corresponding to the target posture information, the first registration result can be determined by the error value. Since there is a target posture information between the target point cloud image and each key part point cloud image template in each key part point cloud image template set, and also has a corresponding error value, for any key part point cloud image template set, the first registration result can be determined according to the error value corresponding to each key part point cloud image template in the any key part point cloud image template set.
[0157] For example, a target threshold can be set. When the error value corresponding to any key part point cloud image template in any key part point cloud image template set is greater than the target threshold, the registration result corresponding to any key part point cloud image template is determined to be a successful registration. Furthermore, when the number of templates with successful registration results corresponding to each key part point cloud image template in any key part point cloud image template set is greater than a first preset number, it can be determined that the first registration result is a successful registration, and the object of the successful registration is any key part point cloud image template set.
[0158] In an embodiment of the present invention, a point cloud registration model is introduced, and a first registration result is obtained by determining an error value corresponding to target posture information, so that the accuracy of the first registration result can be improved.
[0159] On the basis of the above-mentioned embodiment, the livestock body condition scoring method provided in the embodiment of the present invention further includes: obtaining target basic information of the livestock to be scored;
[0160] Accordingly, registering the target point cloud image with each key part point cloud image template set to obtain a first registration result includes:
[0161] Determine the error value corresponding to the target posture information based on the target point cloud registration model corresponding to the target basic information;
[0162] Determining the first registration result based on the error value corresponding to the target posture information;
[0163] Among them, the target pose information is the pose information between the target point cloud image and the target key part point cloud image template, and the target key part point cloud image template is the key part point cloud image template in the key part point cloud image template set corresponding to the target basic information; the target point cloud registration model is trained by the bulldozer distance loss function between the template features of the target key part point cloud image template and the corresponding template estimation features, and the template estimation features are estimated based on the sample pose information, and the sample pose information is the pose information between the sample point cloud image of the sample livestock corresponding to the target basic information and the target key part point cloud image template.
[0164] Specifically, in the embodiment of the present invention, target basic information of the livestock to be scored can be obtained, such as the breed, age and other information of the livestock to be scored.
[0165] Furthermore, when registering the target point cloud image with the point cloud image template set of each key part to obtain the first registration result, the target point cloud registration model corresponding to the target basic information can be determined first according to the target basic information.
[0166] Then, the key part point cloud image template in the key part point cloud image template set corresponding to the target basic information is determined, that is, the target key part point cloud image template, and the target point cloud registration model can be used to determine the pose information between the target point cloud image and the target key part point cloud image template, that is, the target pose information and the error value corresponding to the target pose information. Here, the target pose information is the rotation matrix between the target point cloud image and the target key part point cloud image template.
[0167] In an embodiment of the present invention, the target point cloud image and the target key part point cloud image template can be respectively input into the point cloud registration model, and the point cloud registration model can output the pose information between the target point cloud image and the target key part point cloud image template, that is, the target pose information and its corresponding error value. In addition, the target key part point cloud image template can be pre-stored in the point cloud registration model, and then the target point cloud image can be input into the point cloud registration model to obtain the target pose information and its corresponding error value.
[0168] In the process of training the point cloud registration model, the EMD loss between the template features of the point cloud image template of the key part of the target and the corresponding template estimation features can be used, and the formula is as above.
[0169] The template estimation feature corresponding to the target key part point cloud image template can be estimated using the sample pose information, which is the pose information between the sample point cloud image of the sample livestock and the target key part point cloud image template. The sample pose information is obtained based on the target point cloud registration model. For example, the target point cloud registration model can pre-store the target key part point cloud image template, and then the sample point cloud image of the sample livestock is input into the target point cloud registration model and output by the target point cloud registration model. Alternatively, the sample point cloud image of the sample livestock and the target key part point cloud image template can be input into the target point cloud registration model and output by the target point cloud registration model. While the target point cloud registration model outputs the sample pose information, it can also output the error value corresponding to the sample pose information.
[0170] When using sample pose information to estimate template estimation features, since the sample pose information is the pose information between the sample point cloud image and the target key part point cloud image template, it can be used to characterize the association between the key part features in the sample point cloud image and the template features of the target key part point cloud image template. Therefore, the template estimation features can be determined by combining the key part features in the sample point cloud image.
[0171] After determining the error value corresponding to the target posture information, the first registration result can be determined by the error value. Since the target point cloud image and each target key part point cloud image template each have a target posture information and a corresponding error value, the first registration result can be determined according to the error value corresponding to each target key part point cloud image template.
[0172] For example, a target threshold can be set. When the error value corresponding to any target key part point cloud image template is greater than the target threshold, the alignment result corresponding to any target key part point cloud image template is determined to be a successful alignment. Furthermore, when the number of templates whose alignment results corresponding to all target key part point cloud image templates are successful alignments is greater than a first preset number, the first alignment result can be determined to be a successful alignment, and the object of successful alignment is the set of key part point cloud image templates corresponding to the target basic information, that is, the set consisting of all target key part point cloud image templates.
[0173] In an embodiment of the present invention, basic target information is introduced, and through its corresponding target point cloud registration model, the first registration result is obtained by determining the error value corresponding to the target posture information. On the basis of improving the accuracy of the first registration result, the amount of data processing during registration can be reduced and the registration efficiency can be improved.
[0174] On the basis of the above-mentioned embodiment, the livestock body condition scoring method provided in the embodiment of the present invention further includes: obtaining target basic information of the livestock to be scored;
[0175] Accordingly, registering the target body size feature with each body size feature template set to obtain a second registration result includes:
[0176] For the body size feature template set corresponding to the target basic information, calculate the relative error between each body size feature template in the body size feature template set corresponding to the target basic information and the corresponding target body size feature, and determine the registration result corresponding to the body size feature template set corresponding to the target basic information based on the relative error;
[0177] The registration result corresponding to the body size feature template set corresponding to the target basic information is used as the second registration result.
[0178] Specifically, in the embodiment of the present invention, the target basic information is introduced and the second registration result is determined by the target basic information. This can reduce the amount of data processing during registration and improve the registration efficiency while improving the accuracy of the second registration result.
[0179] On the basis of the above-mentioned embodiment, in the livestock body condition scoring method provided in the embodiment of the present invention, the template estimation feature is estimated based on the following method:
[0180] Performing feature extraction on each region in the sample point cloud image and the key part point cloud image template in each key part point cloud image template set to obtain the regional features of each region and the template features of the key part point cloud image template in each key part point cloud image template set;
[0181] Calculating the distance between the template feature and the regional features of each region, and selecting a preset number of regional features with smaller distances as target regional features;
[0182] Splicing the template feature with the target area feature to obtain a splicing feature, and determining the sample pose information based on the splicing feature;
[0183] The template estimation feature is determined based on the target area feature and the sample pose information.
[0184] Specifically, in the embodiment of the present invention, when estimating the template estimation feature, each region in the sample point cloud image can be determined first, and each region can be obtained by dividing the sample livestock in the sample point cloud image. Here, the external bounding box of the sample livestock can be determined first, and then a multi-scale stereo grid is constructed according to the external bounding box, and finally the sample livestock is divided by the multi-scale stereo grid to obtain each region in the sample point cloud image.
[0185] For example, the volume of the sample livestock is W*H*C, W is the width of the sample livestock, H is the length of the sample livestock, and C is the height of the sample livestock, then a three-dimensional sample livestock outer bounding box can be formed. Then a multi-scale three-dimensional grid is constructed. For the length and width of the sample livestock, the partition parameter is n, and the value range of n can be [3,4,5,6,7]. For the height of the sample livestock, the partition parameter is m, and the value range of m can be [2,3,4,5,5]. According to the value range of n and m, a multi-scale three-dimensional grid can be constructed.
[0186] According to the multi-scale stereo grid, the sample livestock is divided into regions according to the values of n and m, and regions of different scales in the sample point cloud image are obtained. For example, when n = 3 and m = 2, the sample livestock is divided into W*H*C / (n*n*m) W / 3*H / 3*C / 2 regions.
[0187] In the embodiment of the present invention, some three-dimensional grids without point cloud data can be used as background and do not participate in training.
[0188] Then, the feature extraction network of pointnet++ can be used to extract features from each region in the sample point cloud image and the key part point cloud image template in each key part point cloud image template set to obtain regional features. (i=1,2,3,4……, i represents the number of different regions) and the template features of the key part point cloud image template in any key part point cloud image template set. If there are 6 key parts, there are 6 key part point cloud image templates in any key part point cloud image template set, and the corresponding template features can be expressed as (j=1, 2, 3, 4, 5, 6; j represents the number of six parts of the sample livestock). This is used as an example below.
[0189] For any key part point cloud image template set, each part is first roughly aligned, that is, the L2 distance between the template features of each key part point cloud image template in any key part point cloud image template set of the sample livestock and each regional feature is calculated, and then the L2 distance can be sorted from small to large, and then the regional features corresponding to the front preset number of distances can be selected from the sorting results and used as the target regional features. Among them, the regional features corresponding to the front preset number of distances in the sorting results are the preset number of regional features with smaller distances. The preset number can be set as needed, for example, it can be k.
[0190] The template features of each key part point cloud image template in any key part point cloud image template set correspond to k target area features, and then the template features of each key part point cloud image template are concatenated with the corresponding k target area features to obtain k concatenated features, and then the k concatenated features are regressed to obtain k sample pose information. Here, the k concatenated features can be input into the fully connected (FC) layer to obtain the k sample pose information output by the FC layer. Finally, the k template estimation features can be obtained by calculating the k sample pose information and the k target area features.
[0191] In the embodiment of the present invention, when estimating template estimation features, key part features in the sample point cloud image are represented by a preset number of regional features with smaller distances, and actual extraction is not required, which can improve the estimation efficiency of the template estimation features.
[0192] On the basis of the above-mentioned embodiment, in the livestock body condition scoring method provided in the embodiment of the present invention, the template estimation feature is estimated based on the following method:
[0193] Extracting features from each region in the sample point cloud image and the target key part point cloud image template to obtain regional features of each region and template features of the target key part point cloud image template;
[0194] Calculating the distance between the template feature and the regional features of each region, and selecting a preset number of regional features with smaller distances as target regional features;
[0195] Splicing the template feature with the target area feature to obtain a splicing feature, and determining the sample pose information based on the splicing feature;
[0196] The template estimation feature is determined based on the target area feature and the sample pose information.
[0197] Specifically, the implementation of the embodiment of the present invention is based on the need to obtain the target basic information of the livestock to be scored. During the estimation process, the target key part point cloud image template in the key part point cloud image template set corresponding to the target basic information is used, which can reduce the amount of calculation.
[0198] On the basis of the above-mentioned embodiment, the livestock body condition scoring method provided in the embodiment of the present invention, wherein the step of obtaining a target point cloud image of the livestock to be scored comprises:
[0199] Acquire local point cloud images of the livestock to be scored at multiple viewing angles;
[0200] Inputting the local point cloud images under the multiple viewing angles into a three-dimensional reconstruction model to obtain the target point cloud image output by the three-dimensional reconstruction model;
[0201] The three-dimensional reconstruction model is trained based on sample local point cloud images from multiple perspectives.
[0202] Specifically, in an embodiment of the present invention, when obtaining a target point cloud image of the livestock to be scored, a local point cloud image of the livestock to be scored under multiple viewing angles may be obtained first. A livestock walking passage may be configured for the livestock to be scored, and a plurality of image acquisition devices may be distributedly arranged on the livestock walking passage, and each image acquisition device may acquire a local point cloud image of the livestock to be scored under one viewing angle. For example, the plurality of viewing angles may include a rear viewing angle, an upper viewing angle, a left viewing angle, and a right viewing angle, that is, four image acquisition devices may be arranged on the livestock walking passage, respectively arranged at the target end, the upper side, the left side, and the right side of the livestock walking passage.
[0203] When obtaining the local point cloud images of the livestock to be scored at multiple viewing angles, the livestock to be scored can be driven into the livestock walking passage with its tail facing the target end of the livestock walking passage.
[0204] In the embodiment of the present invention, the image acquisition device used may be a TOF-based depth camera, or other cameras that can acquire point cloud images, which is not specifically limited here.
[0205] Then, the local point cloud images under multiple viewing angles are input into the 3D reconstruction model to obtain the target point cloud image output by the 3D reconstruction model. The 3D reconstruction model can use one of the local point cloud images under multiple viewing angles as a reference image and the others as auxiliary images to finally obtain the target point cloud image of the livestock to be scored.
[0206] The adopted 3D reconstruction model can be constructed based on PointMVSNet and obtained by training sample local point cloud images under multiple viewing angles. The sample local point cloud images under multiple viewing angles can be obtained by driving the sample livestock into the livestock walking passage with its tail facing the target end of the livestock walking passage through an image acquisition device set on the livestock walking passage. In the process of training to obtain the 3D reconstruction model, one of the local point cloud images under multiple viewing angles of each sample livestock can be used as a reference image, and the others can be used as auxiliary images, and combined with the parameters of the image acquisition device, a supervised training method or an unsupervised training method can be used to obtain the 3D reconstruction model.
[0207] In the embodiment of the present invention, a complete target point cloud image of the livestock to be scored is obtained through local point cloud images under multiple viewing angles, which can make the target point cloud image more accurate, thereby improving the accuracy of the scoring result.
[0208] On the basis of the above embodiment, in the livestock body condition scoring method provided in the embodiment of the present invention, the key parts of the livestock to be scored include at least one of the ischium, hip joint, clavicle, short ribs, sacral ligament and coccygeal ligament;
[0209] Correspondingly, the part point cloud image template sets corresponding to the standard body condition scores all include key part point cloud image templates corresponding to the key parts of the sample livestock corresponding to the standard body condition scores. Each part point cloud image template set corresponding to the standard body condition score may correspond to one basic information.
[0210] In the embodiment of the present invention, various key parts that affect the body condition score are taken into consideration, and the accuracy can be greatly improved compared to considering only one key part.
[0211] like Figure 2 As shown, based on the above embodiment, an embodiment of the present invention provides a livestock body condition scoring device, comprising:
[0212] An image acquisition module 21 is used to acquire a target point cloud image of the livestock to be scored;
[0213] A registration module 22, used for registering the target point cloud image with each key part point cloud image template set to obtain a first registration result;
[0214] The body condition score determination module 23 is used to determine the body condition score of the livestock to be scored based on the first registration result and the standard body condition score corresponding to the point cloud image template set of each key part.
[0215] On the basis of the above-mentioned embodiment, the livestock body condition scoring device provided in the embodiment of the present invention, the body condition score determination module comprises:
[0216] A first determination submodule is configured to determine the body condition score of the livestock to be scored based on a first standard body condition score corresponding to a first key part point cloud image template set if the first registration result is successful registration; the first key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image;
[0217] The second determination submodule is used for determining the target body size feature of the livestock to be scored based on the target point cloud image if the first registration result is that the registration is unsuccessful, registering the target body size feature with each body size feature template set to obtain a second registration result, and determining the body condition score of the livestock to be scored based on the first registration result, the second registration result and the standard body condition score corresponding to each body size feature template set.
[0218] On the basis of the above-mentioned embodiment, in the livestock body condition scoring device provided in the embodiment of the present invention, the unsuccessful registration includes partial successful registration and failed registration;
[0219] Accordingly, the second determining submodule is used to:
[0220] If the first registration result is a partial registration success and the second registration result is a registration success, then the body condition score of the livestock to be scored is determined based on the average value of the second standard body condition score corresponding to the second key part point cloud image template set and the third standard body condition score corresponding to the target body size feature template set;
[0221] The second key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image part, and the target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
[0222] On the basis of the above-mentioned embodiment, in the livestock body condition scoring device provided in the embodiment of the present invention, the target body size feature includes multiple body size feature templates, each body size feature template set includes multiple body size feature templates, and the multiple body size feature templates correspond to the target body size feature one by one;
[0223] Accordingly, the second determining submodule is used to:
[0224] For any body size feature template set in the body size feature template sets, calculate a relative error between each body size feature template in the body size feature template set and a corresponding target body size feature, and determine a registration result corresponding to the body size feature template set based on the relative error;
[0225] The second registration result is determined based on the registration results corresponding to each body size feature template set.
[0226] On the basis of the above embodiment, in the livestock body condition scoring device provided in the embodiment of the present invention, the first determining submodule is used for:
[0227] Determining the target body size feature of the livestock to be scored based on the target point cloud image, and registering the target body size feature with each body size feature template set to obtain a second registration result;
[0228] The body condition score of the livestock to be scored is determined based on the first standard body condition score, the second registration result, and the standard body condition scores corresponding to each body size feature template set.
[0229] On the basis of the above embodiment, in the livestock body condition scoring device provided in the embodiment of the present invention, the first determining submodule is further used for:
[0230] If the second registration result is successful registration, and the difference between the first standard body condition score and the third standard body condition score corresponding to the target body size feature template set is within a preset range, then the body condition score of the livestock to be scored is determined based on the average value of the first standard body condition score and the third standard body condition score;
[0231] The target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
[0232] On the basis of the above embodiment, the livestock body condition scoring device provided in the embodiment of the present invention further includes a basic information acquisition module for:
[0233] Obtaining target basic information of the livestock to be scored;
[0234] Accordingly, the registration module is used to:
[0235] Determine the error value corresponding to the target posture information based on the target point cloud registration model corresponding to the target basic information;
[0236] Determining the first registration result based on the error value corresponding to the target posture information;
[0237] Among them, the target pose information is the pose information between the target point cloud image and the target key part point cloud image template, and the target key part point cloud image template is the key part point cloud image template in the key part point cloud image template set corresponding to the target basic information; the target point cloud registration model is trained by the bulldozer distance loss function between the template features of the target key part point cloud image template and the corresponding template estimation features, and the template estimation features are estimated based on the sample pose information, and the sample pose information is the pose information between the sample point cloud image of the sample livestock corresponding to the target basic information and the target key part point cloud image template.
[0238] On the basis of the above embodiment, the livestock body condition scoring device provided in the embodiment of the present invention further includes an estimation module for:
[0239] Extracting features from each region in the sample point cloud image and the target key part point cloud image template to obtain regional features of each region and template features of the target key part point cloud image template;
[0240] Calculating the distance between the template feature and the regional features of each region, and selecting a preset number of regional features with smaller distances as target regional features;
[0241] Splicing the template feature with the target area feature to obtain a splicing feature, and determining the sample pose information based on the splicing feature;
[0242] The template estimation feature is determined based on the target area feature and the sample pose information.
[0243] On the basis of the above-mentioned embodiment, in the livestock body condition scoring device provided in the embodiment of the present invention, the image acquisition module is used for:
[0244] Acquire local point cloud images of the livestock to be scored at multiple viewing angles;
[0245] Inputting the local point cloud images under the multiple viewing angles into a three-dimensional reconstruction model to obtain the target point cloud image output by the three-dimensional reconstruction model;
[0246] The three-dimensional reconstruction model is trained based on sample local point cloud images from multiple perspectives.
[0247] On the basis of the above-mentioned embodiment, in the livestock body condition scoring device provided in the embodiment of the present invention, the key parts of the livestock to be scored include at least one of the ischium, hip joint, clavicle, short ribs, sacral ligament and coccygeal ligament;
[0248] Correspondingly, the key part point cloud image template sets corresponding to the respective standard body condition scores include key part point cloud image templates corresponding to the respective key parts of the sample livestock corresponding to the respective standard body condition scores.
[0249] Specifically, the functions of each module in the livestock body condition scoring device provided in the embodiment of the present invention correspond one-to-one to the operating procedures of each step in the above method embodiment, and the effects achieved are also consistent. Please refer to the above embodiment for details, which will not be repeated in the embodiment of the present invention.
[0250] On the basis of the above embodiments, an embodiment of the present invention provides a livestock body condition scoring system, comprising: the livestock body condition scoring device provided in the above embodiments and a plurality of image acquisition devices;
[0251] The multiple image acquisition devices are distributedly arranged on the livestock walking path, and the multiple image acquisition devices are used to obtain local point cloud images of the livestock to be scored under multiple viewing angles.
[0252] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the livestock body condition scoring method provided in the above embodiments, the method comprising: obtaining a target point cloud image of the livestock to be scored; registering the target point cloud image with each key part point cloud image template set to obtain a first registration result; determining the body condition score of the livestock to be scored based on the first registration result and the standard body condition score corresponding to each key part point cloud image template set.
[0253] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0254] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the livestock body condition scoring method provided in the above-mentioned embodiments, the method including: obtaining a target point cloud image of the livestock to be scored; aligning the target point cloud image with a set of point cloud image templates of each key part to obtain a first alignment result; and determining the body condition score of the livestock to be scored based on the first alignment result and the standard body condition score corresponding to the set of point cloud image templates of each key part.
[0255] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the livestock body condition scoring method provided in the above-mentioned embodiments, the method comprising: obtaining a target point cloud image of the livestock to be scored; aligning the target point cloud image with a set of point cloud image templates of each key part to obtain a first alignment result; and determining the body condition score of the livestock to be scored based on the first alignment result and the standard body condition score corresponding to the set of point cloud image templates of each key part.
[0256] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0257] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0258] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for scoring the body condition of livestock, It is characterized in that include: Acquire a target point cloud image of the livestock to be scored; Registering the target point cloud image with each key part point cloud image template set to obtain a first registration result; If the first registration result is successful registration, the body condition score of the livestock to be scored is determined based on the first standard body condition score corresponding to the first key part point cloud image template set; the first key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image; If the first registration result is that the registration is unsuccessful, and the unsuccessful registration includes partial successful registration and failed registration, then based on the target point cloud image, the target body size feature of the livestock to be scored is determined, and the target body size feature is registered with each body size feature template set to obtain a second registration result; If the first registration result is a partial registration success and the second registration result is a registration success, then the body condition score of the livestock to be scored is determined based on the average value of the second standard body condition score corresponding to the second key part point cloud image template set and the third standard body condition score corresponding to the target body size feature template set; The second key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image part, and the target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
2. The livestock body condition scoring method according to claim 1, It is characterized in that The target body size features include multiple body size feature sets, each of the body size feature template sets includes multiple body size feature templates, and the multiple body size feature templates correspond to the target body size features one by one; Accordingly, registering the target body size feature with each body size feature template set to obtain a second registration result includes: For any body size feature template set in the body size feature template sets, calculate a relative error between each body size feature template in the body size feature template set and a corresponding target body size feature, and determine a registration result corresponding to the body size feature template set based on the relative error; The second registration result is determined based on the registration results corresponding to each body size feature template set.
3. The livestock body condition scoring method according to claim 1, It is characterized in that The step of determining the body condition score of the livestock to be scored based on the first standard body condition score corresponding to the first key part point cloud image template set includes: Determining the target body size feature of the livestock to be scored based on the target point cloud image, and registering the target body size feature with each body size feature template set to obtain a second registration result; The body condition score of the livestock to be scored is determined based on the first standard body condition score, the second registration result, and the standard body condition scores corresponding to each body size feature template set.
4. The livestock body condition scoring method according to claim 3, It is characterized in that The step of determining the body condition score of the livestock to be scored based on the first standard body condition score, the second registration result, and the standard body condition score corresponding to each body size feature template set comprises: If the second registration result is successful registration, and the difference between the first standard body condition score and the third standard body condition score corresponding to the target body size feature template set is within a preset range, then the body condition score of the livestock to be scored is determined based on the average value of the first standard body condition score and the third standard body condition score; The target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
5. The livestock body condition scoring method according to any one of claims 1 to 4, It is characterized in that Also includes: Obtaining target basic information of the livestock to be scored; Accordingly, registering the target point cloud image with each key part point cloud image template set to obtain a first registration result includes: Determine the error value corresponding to the target posture information based on the target point cloud registration model corresponding to the target basic information; Determining the first registration result based on the error value corresponding to the target posture information; Among them, the target pose information is the pose information between the target point cloud image and the target key part point cloud image template, and the target key part point cloud image template is the key part point cloud image template in the key part point cloud image template set corresponding to the target basic information; the target point cloud registration model is trained by the bulldozer distance loss function between the template features of the target key part point cloud image template and the corresponding template estimation features, and the template estimation features are estimated based on the sample pose information, and the sample pose information is the pose information between the sample point cloud image of the sample livestock corresponding to the target basic information and the target key part point cloud image template.
6. The livestock body condition scoring method according to claim 5, It is characterized in that The template estimation feature is estimated based on the following method: Extracting features from each region in the sample point cloud image and the target key part point cloud image template to obtain regional features of each region and template features of the target key part point cloud image template; Calculating the distance between the template feature and the regional features of each region, and selecting a preset number of regional features with smaller distances as target regional features; Splicing the template feature with the target area feature to obtain a splicing feature, and determining the sample pose information based on the splicing feature; The template estimation feature is determined based on the target area feature and the sample pose information.
7. The livestock body condition scoring method according to any one of claims 1 to 4, It is characterized in that The step of obtaining a target point cloud image of the livestock to be scored comprises: Acquire local point cloud images of the livestock to be scored at multiple viewing angles; Inputting the local point cloud images under the multiple viewing angles into a three-dimensional reconstruction model to obtain the target point cloud image output by the three-dimensional reconstruction model; The three-dimensional reconstruction model is trained based on sample local point cloud images from multiple perspectives.
8. The livestock body condition scoring method according to any one of claims 1 to 4, It is characterized in that The key parts of the livestock to be scored include at least one of the ischium, hip joint, clavicle, short ribs, sacral ligament and coccygeal ligament; Correspondingly, the key part point cloud image template sets corresponding to the respective standard body condition scores include key part point cloud image templates corresponding to the respective key parts of the sample livestock corresponding to the respective standard body condition scores.
9. A livestock body condition scoring device, It is characterized in that include: An image acquisition module, used to acquire a target point cloud image of the livestock to be scored; A registration module, used to register the target point cloud image with each key part point cloud image template set to obtain a first registration result; Body condition score determination module, used to: If the first registration result is successful registration, the body condition score of the livestock to be scored is determined based on the first standard body condition score corresponding to the first key part point cloud image template set; the first key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image; If the first registration result is that the registration is unsuccessful, and the unsuccessful registration includes partial successful registration and failed registration, then based on the target point cloud image, the target body size feature of the livestock to be scored is determined, and the target body size feature is registered with each body size feature template set to obtain a second registration result; If the first registration result is a partial registration success and the second registration result is a registration success, then the body condition score of the livestock to be scored is determined based on the average value of the second standard body condition score corresponding to the second key part point cloud image template set and the third standard body condition score corresponding to the target body size feature template set; The second key part point cloud image template set is a key part point cloud image template set that is successfully registered with the target point cloud image part, and the target body size feature template set is a body size feature template set that is successfully registered with the target body size feature.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of the livestock body condition scoring method according to any one of claims 1 to 8 are implemented.
11. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the livestock body condition scoring method according to any one of claims 1 to 8 are implemented.
Citation Information
Patent Citations
Automatic livestock phenotypic body size data measuring method based on three-dimensional point cloud
CN110986788A
Livestock weight estimation method, device and equipment and computer readable storage medium
CN111243005A
A livestock weighing system using the LiDAR and a livestock weighing method using the same
KR102131558B1
Method and apparatus for evaluating an animal
WO2017030448A1
