Cow body condition automatic scoring method and device

Abstract

The invention provides a dairy cow body condition automatic scoring method and device, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting a multi-view image of a dairy cow through a plurality of RGB cameras; space coordinates, observation angles and camera internal and external parameters of the RGB camera corresponding to each view angle image are determined, and the camera internal and external parameters comprise a camera optical center position, a camera light direction and a camera sampling step length; constructing an initial neural radiation field network model based on the multi-view image, the space coordinates and the observation angle; based on the initial neural radiation field network model and camera internal and external parameters, generating an initial dairy cow image at any view angle; and based on the initial dairy cow image and the quantized body condition parameters of the dairy cow, automatically scoring the body condition of the dairy cow to obtain a scoring result of the body condition of the dairy cow. According to the dairy cow body condition automatic scoring method provided by the invention, the dairy cow body condition scoring precision and efficiency are improved, the cost of a dairy cow farm is reduced, and the application efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an automated method and apparatus for scoring the physical condition of dairy cows. Background Technology

[0002] The health of dairy cows is closely related to their Body Condition Score (BCS). Reasonable body condition management not only helps maintain the good health of dairy cows, but also plays an important role in improving lactation performance and reproductive efficiency.

[0003] Existing methods for assessing the body condition of dairy cows mainly include manual assessment, semi-automatic assessment, and deep learning assessment. Manual and semi-automatic assessment methods rely on human intervention, resulting in issues such as strong human subjectivity, low efficiency, high cost, and large assessment errors. In deep learning assessment methods, the complex environment of farms, with severe environmental obstruction and pollution, leads to poor depth image acquisition of dairy cows. Furthermore, two-dimensional images are difficult to capture deep curvature information, resulting in low accuracy of deep learning models in assessing the body condition of dairy cows. Summary of the Invention

[0004] This invention provides an automated method and apparatus for scoring the body condition of dairy cows, which solves the technical problem of low accuracy and efficiency in scoring the body condition of dairy cows in the prior art, and realizes accurate and efficient scoring of the body condition of dairy cows.

[0005] This invention provides an automated method for assessing the body condition of dairy cows, comprising the following steps: Multiple RGB cameras were used to capture multi-view images of dairy cows. Determine the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB camera corresponding to each viewpoint image; the intrinsic and extrinsic parameters include the camera optical center position, the direction of the incident light rays, and the camera sampling step size. Based on the multi-view images, the spatial coordinates, and the observation angle, an initial neural radiation field network model is constructed. Based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters, an initial cow image from any viewpoint is generated; Based on the initial cow image and the cow's quantitative body condition parameters, the cow's body condition is automatically scored to obtain the score result.

[0006] According to the present invention, an automated body condition scoring method for dairy cows is provided, wherein constructing an initial neural radiation field network model based on the multi-view images, the spatial coordinates, and the observation angle includes: Map the spatial coordinates and the observation angle to a three-dimensional feature space; Based on the multi-view images, determine the RGB color value and volume density of each spatial point in the three-dimensional feature space; An initial neural radiation field network model is constructed based on the RGB color value and volume density of each spatial point.

[0007] According to the present invention, an automated method for assessing the body condition of dairy cows, wherein generating an initial image of the dairy cow from an arbitrary viewpoint based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters includes: Based on the optical center position of the camera and the sampling step size of the camera, layered integral sampling is performed along the direction of the incident light from the camera to obtain multiple three-dimensional spatial points; Determine the RGB color value and volume density of each three-dimensional spatial point in the initial neural radiation field network model; Based on the RGB color value and volume density of each three-dimensional spatial point in the initial neural radiation field network model, an initial cow image from any viewpoint is generated.

[0008] According to the present invention, an automated body condition scoring method for dairy cows is provided, wherein the automated scoring of the cow's body condition based on the initial cow image and the cow's quantitative body condition parameters, to obtain the scoring result, includes: Based on the initial cow image, the initial neural radiation field network model is optimized to obtain the target radiation field network model; Based on the target radiation field network model and camera intrinsic and extrinsic parameters, a 3D model of the target cow under a standard viewpoint is generated. Based on the target cow's 3D model and the cow's quantitative body condition parameters, the cow's body condition is automatically scored to obtain the score result.

[0009] According to the present invention, an automated body condition scoring method for dairy cows includes optimizing an initial neural radiation field network model based on an initial dairy cow image to obtain a target radiation field network model, comprising: The loss value of the initial cow image is calculated based on a preset loss function; the loss function includes smoothness loss, geometric loss, color loss, and grayscale loss. Based on the loss value, backpropagation gradient optimization is performed on the initial neural radiation field network model to obtain an intermediate radiation field network model. Repeat the preset iterative steps to obtain the target radiation field network model.

[0010] According to the present invention, an automated body condition scoring method for dairy cows includes an iterative step comprising: Based on the intermediate radiation field network model obtained in the previous iteration and the camera intrinsic and extrinsic parameters, the intermediate three-dimensional model of the cow obtained in this iteration is obtained. The projected image of the intermediate 3D cow model is evaluated using preset evaluation metrics to obtain evaluation results; the evaluation metrics include mean absolute error, maximum F-value, weighted F-value, structural similarity measure, and enhanced alignment measure. If the evaluation result meets the preset threshold requirement, the iteration step is stopped, and the intermediate radiation field network model corresponding to the intermediate cow three-dimensional model is used as the target radiation field network model. If the evaluation result does not meet the preset threshold requirement, an intermediate cow image from any perspective is generated based on the intermediate radiation field network model corresponding to the intermediate cow 3D model and the camera intrinsic and extrinsic parameters. Based on the intermediate cow image, the intermediate neural radiation field network model is optimized to obtain the intermediate radiation field network model obtained in this iteration.

[0011] The present invention also provides an automated body condition scoring device for dairy cows, comprising the following modules: The acquisition module is used to acquire multi-view images of cows using multiple RGB cameras; The 3D reconstruction module is used to determine the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB camera corresponding to each viewpoint image; the intrinsic and extrinsic parameters of the camera include the position of the camera optical center, the direction of the incident light rays, and the camera sampling step size. A three-dimensional mapping module is used to construct an initial neural radiation field network model based on the multi-view images, the spatial coordinates, and the observation angle. The generation module is used to generate initial cow images from any viewpoint based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters. The scoring module is used to automatically score the body condition of the cows based on the initial cow image and the cow's quantitative body condition parameters, and obtain the scoring result of the cow's body condition.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement any of the above-described automated cow condition scoring methods.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the automated cow condition scoring method as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described automated dairy cow condition scoring methods.

[0015] The automated bovine body condition scoring method and apparatus provided by this invention acquires multi-view images of bovines using multiple RGB cameras, overcoming the shortcomings of traditional two-dimensional image methods in capturing the three-dimensional features of bovine body condition and improving image acquisition quality. It determines the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB cameras corresponding to each viewpoint image. These intrinsic and extrinsic parameters include the camera's optical center position, the direction of incident light rays, and the camera's sampling step size, thereby recovering the bovine's three-dimensional structure and the motion trajectory of the RGB cameras. Based on the multi-view images, spatial coordinates, and observation angles, an initial neural radiation field network model is constructed, providing a model foundation for rendering bovine images from any viewpoint. Based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters, initial bovine images from any viewpoint are generated, thereby quickly acquiring comprehensive image information about the bovine body condition. Based on the initial bovine images and the quantified bovine body condition parameters, the bovine body condition is automatically scored, obtaining the scoring results and improving the accuracy and efficiency of bovine body condition scoring. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating an automated body condition scoring method for dairy cows provided by the present invention.

[0018] Figure 2 This is a schematic diagram of the visualization results of RGB camera pose calculation under a shooting angle interval of 30° between adjacent images provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the three-dimensional structure of the restored cow movement scene and the motion trajectory of the RGB camera provided by the present invention.

[0020] Figure 4 This is a schematic diagram of a three-dimensional reconstruction model of a dairy cow provided by the present invention.

[0021] Figure 5 This is a schematic diagram of the structure of an automated dairy cow condition scoring device provided by the present invention.

[0022] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Existing dairy cow body condition score assessment technologies can be divided into three main categories: manual scoring methods, semi-automatic scoring methods, and deep learning scoring methods.

[0025] Manual assessment methods involve assessors using their hands, eyes, or tools such as folding protractors to measure a cow's condition. While professional assessors can score a cow in a short time, assessing body condition in large-scale dairy farms is extremely time-consuming and labor-intensive. Furthermore, the assessment results from different assessors can vary significantly, and the method heavily relies on the assessor's level of responsibility. This results in problems such as strong human subjectivity, low efficiency, high cost, and large scoring errors.

[0026] Semi-automatic scoring methods combine human intervention with automated equipment to assess the body condition of dairy cows. For example, ultrasonic instruments are used to measure the body condition and body fat content of dairy cows, and a unified body condition score is formed by combining their body appearance data. This method still relies mainly on human intervention, and the automated equipment used generally has the problems of high price and short lifespan. Therefore, the body condition scoring method is characterized by high cost and low efficiency.

[0027] Deep learning-based scoring methods utilize 2D or 3D image acquisition devices, employing image analysis, pixel processing, and deep learning techniques to calculate cow body condition scores, which are then combined with mathematical statistical models to achieve the final score. While 3D image acquisition devices (such as depth cameras and depth scanners) provide a high degree of accuracy in reproducing cow body features through active measurement, the high cost and poor acquisition quality of these devices, coupled with the complex environment of farms, significant environmental obstructions, and pollution, result in low accuracy in cow body condition scoring. Conversely, 2D image acquisition devices capture two-dimensional images of cows, but this process loses details about the contours of the cow's body (such as rump strength), making it impossible to directly measure the muscle richness between two bony prominences and capture deep curvature information.

[0028] To address the aforementioned problems in the existing dairy cow body condition scoring process, this invention proposes an automated dairy cow body condition scoring method. This method involves deploying high-definition RGB acquisition devices and depth sensing devices in a rotary milking parlor at the dairy farm. Combined with the built-in computing devices of these devices and a model-based joint calculation method, the method achieves automated scoring of individual dairy cows, standardizes the evaluation criteria for dairy cow appearance on farms, avoids the subjective influence of traditional scoring methods, effectively reduces manual time and labor on farms, and ultimately reduces costs and improves application efficiency in dairy farms.

[0029] The following is combined Figures 1 to 6 The present invention describes an automated method and apparatus for assessing the physical condition of dairy cows.

[0030] Figure 1 This is a flowchart illustrating an automated body condition scoring method for dairy cows provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps: Step 101: Acquire multi-view images of cows using multiple RGB cameras; Specifically, this invention provides a dairy cow image acquisition device. The device primarily utilizes a high-definition RGB camera to collect individual dairy cow body condition information. This device can be placed in the milking parlor's de-cupping area to collect information from cows in a static state. Combined with wearable exchange devices, neck collars, and RFID readers (the RFID readers are used to read individual dairy cow information; the tag protocol is EPC C1 GEN2, ISO18000-6C / 6; the operating frequency is 860MHz–960MHz; RFID performance: >700 images / second), the device enables the identification and management of individual dairy cow information, helping farm managers efficiently monitor the growth information of each individual cow.

[0031] For example, the device used to acquire body condition image data of dairy cows could be an RGB camera with the following characteristics: 1 / 1.2-inch Progressive Scan CMOS imaging, 8 million effective pixels, a focal length of 2.8mm, a horizontal field of view of 101.7°, a vertical field of view of 50.9°, a diagonal field of view of 125.2°, an aperture of F1.0, a lens size interface of M16, and a compression bitrate of 32Kbps-16Mbps.

[0032] Before acquiring multi-view images of cows using multiple RGB cameras, an experimental design can be used to determine the minimum number of cameras required for acquiring multi-view images of cows through statistical analysis, thereby enabling effective data acquisition while saving costs.

[0033] In this embodiment of the invention, shooting and reconstruction experiments were first conducted at different sampling points. Table 1 shows the results of the shooting and reconstruction experiments at different sampling points provided by this invention. As shown in Table 1, since the overlap rate of the head and tail of the cow differs significantly from the overlap rate of the sides, the overlap rates of the head and tail and the sides were calculated separately, and the average overlap rate was calculated.

[0034] Table 1. Results of the imaging and reconstruction experiments at different acquisition points. The experimental data in Table 1 show that the success rate of completing all pose calculations is significantly correlated with the horizontal angle of adjacent cameras: when the shooting angle interval between adjacent images is less than 30°, the system can successfully estimate the camera pose of all input images. Figure 2 This is a schematic diagram illustrating the visualization results of RGB camera pose calculation under a 30° shooting angle interval between adjacent images provided by the present invention, as shown below. Figure 2 As shown, as the image spacing exceeds 30°, the number of cameras capable of performing pose calculations begins to fall below the total number of input images. During the reconstruction process, it was found that the images of the head and tail regions were initially unable to be used for pose calculation; that is, the calculated head-tail overlap rate was generally lower than the side overlap rate.

[0035] Because the head and tail of a cow occupy a relatively small area within the frame, the overlapping area is even smaller at the same angular interval. Furthermore, the head and tail regions have complex structures, irregular shapes, and contain a large amount of detail, requiring a relatively dense camera distribution to complete the camera pose calculation for the head and tail. For example, when the spacing angle is further increased to 45°, the pose calculation failure becomes more pronounced, and only three images in the vertical direction can be matched.

[0036] As the angle continues to increase, the image overlap rate decreases, making it impossible to ensure a sufficient overlap area for feature matching. Therefore, by calculating the overlap rate at different angles, it can be concluded that when the average overlap rate of adjacent images of the cow's head, tail, and both sides is above 0.6, the required viewpoint spacing density for completing all image pose calculations is met. At this point, the number of RGB cameras and the pose of each RGB camera can be used to acquire comprehensive multi-view images of the cow, thereby meeting the requirements of 3D reconstruction for image spacing and quality at the lowest cost.

[0037] This invention, through the deployment of acquisition devices in rotary milking parlors in dairy farms, can accurately and quickly acquire multi-view images of dairy cows, while saving equipment and labor costs.

[0038] Step 102: Determine the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB camera corresponding to each viewpoint image; the intrinsic and extrinsic parameters of the camera include the position of the camera optical center, the direction of the incident light rays, and the camera sampling step size; Specifically, this involves generating sparse point clouds from multi-view images to reconstruct the three-dimensional motion structure of the cow. To ensure both the speed and quality of motion structure reconstruction, different labels can be assigned to the multi-view views according to the RGB camera numbers used for acquisition. Since the feature points acquired by images with the same label are roughly the same, images with adjacent labels can be grouped together, with the absolute value of adjacent labels being 2.

[0039] By analyzing the correspondence of feature points in multiple images using Structure from Motion (SFM) technology, sparse point clouds are generated, thereby restoring the three-dimensional structure of the cow movement scene and the motion trajectory of the RGB camera, and determining the spatial coordinates, observation angle, and camera parameters of the RGB camera corresponding to each viewpoint image. Figure 3 This is a schematic diagram of the three-dimensional structure of the restored cow movement scene and the motion trajectory of the RGB camera provided by the present invention, as shown below. Figure 3 As shown.

[0040] In some embodiments, in order to further improve the accuracy of 3D reconstruction of cow movement scenes, a dense point cloud or depth map can be generated based on the sparse point cloud generated by the above-mentioned SFM technology, using multi-view stereo matching technology to enhance the detail representation of the 3D model. Figure 4 This is a schematic diagram of the three-dimensional reconstruction model of a dairy cow provided by the present invention, as shown below. Figure 4 As shown.

[0041] This invention reconstructs the three-dimensional structure of the cow's movement scene and the motion trajectory of the RGB camera based on multi-view images of dairy cows. It determines the spatial coordinates, observation angle, and internal and external parameters of the RGB camera corresponding to each view image, thereby obtaining accurate structural features. This overcomes the shortcomings of traditional two-dimensional image methods in capturing deep curvature information, obtains three-dimensional features of the cow's body condition, and improves the acquisition effect of dairy cow image data.

[0042] Step 103: Based on the multi-view images, the spatial coordinates, and the observation angle, construct an initial neural radiation field network model; Furthermore, the construction of the initial neural radiation field network model based on the multi-view images, the spatial coordinates, and the observation angle includes: Map the spatial coordinates and the observation angle to a three-dimensional feature space; Based on the multi-view images, determine the RGB color value and volume density of each spatial point in the three-dimensional feature space; An initial neural radiation field network model is constructed based on the RGB color value and volume density of each spatial point.

[0043] Specifically, after determining the pose (i.e., spatial coordinates and observation angle) of the RGB camera corresponding to each viewpoint image based on the image sequence corresponding to the multi-view images of the cow, the spatial coordinates are decomposed in the frequency domain using position coding technology. Low-frequency geometric information and high-frequency detail features are fused into a multi-scale embedding vector. An end-to-end mapping relationship is constructed through a multilayer perceptron, thereby mapping the spatial coordinates and observation angle to a three-dimensional feature space. The RGB color value and volume density of each spatial point are determined, and a continuously differentiable neural radiation field is constructed to obtain the initial neural radiation field network model.

[0044] For example, for each input viewpoint image, the spatial coordinates of the RGB camera And perspective (i.e., observation angle) Through multilayer perceptron Learning to map color values ​​to high-dimensional scenes and volume density The expression is as follows: Multilayer perceptrons generate implicit fields by fusing features layer by layer, constructing a continuously differentiable neural radiation field that can simultaneously encode scene geometry and viewpoint-related appearance attributes.

[0045] The embodiments of the present invention significantly improve the ability to capture detailed features of key dairy cow body condition scoring parts such as the tail root and hip bone through mapping in three-dimensional feature space, and can accurately represent the complex geometric shape and material properties of the dairy cow's body surface.

[0046] Step 104: Based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters, generate initial cow images from any viewpoint; Furthermore, the step of generating initial cow images from arbitrary viewpoints based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters includes: Based on the optical center position of the camera and the sampling step size of the camera, layered integral sampling is performed along the direction of the incident light from the camera to obtain multiple three-dimensional spatial points; Determine the RGB color value and volume density of each three-dimensional spatial point in the initial neural radiation field network model; Based on the RGB color value and volume density of each three-dimensional spatial point in the initial neural radiation field network model, an initial cow image from any viewpoint is generated.

[0047] Specifically, firstly, based on the camera's optical center position and camera sampling step size, a hierarchical integral sampling strategy is adopted to select multiple three-dimensional spatial points along the camera's incident light rays; then, based on the volume density and color distribution in the initial neural radiation field network model, the volume density corresponding to each spatial point is directly queried, and the pixel color corresponding to each spatial point is synthesized through the volume rendering integral equation.

[0048] For example, for a given camera viewpoint A pixel's viewpoint direction In the scene, it can be represented as a line from position Arrive at the location rays The expression is as follows: To represent the degree to which light is absorbed or scattered when it passes through a point, the transmission function is used. Calculate its relationship with light in Bulk density at The relevant expression is as follows: In order to determine the final pixel color Perform numerical calculations to transfer the ray from arrive The interval discretization is as follows a short segment For each segment, density, color, and transparency are calculated separately, and then weighted summation is performed to obtain the final pixel color value, as shown in the following expression: This invention calculates the attenuation probability of light in a medium, then performs a transmittance-weighted integral on the color values ​​of spatial points, and finally renders an initial image of a cow from any viewing angle. This allows for convenient observation of key parts representing the cow's physical condition from any angle, realizing multi-angle visualization analysis of the cow's physical condition and improving the accuracy and efficiency of cow physical condition scoring.

[0049] Step 105: Based on the initial cow image and the cow's quantitative body condition parameters, automatically score the cow's body condition to obtain the score result.

[0050] Specifically, for the automated scoring of dairy cow body condition, unlike traditional visual estimation methods, this embodiment of the invention employs a hierarchical decision tree structure. First, it divides the cow's body condition into two main scoring paths based on the angles connecting the hip joint, hip tubercle, and ischial tuberosity. For cows with lower scores, a fine-grained score with an accuracy of 0.25 points is applied in four progressive levels: the roundness of the hip tubercle, the fullness of the ischial tubercle, the fat coverage of the ischial tubercle, and the positional deviation of the transverse process of the lumbar vertebrae. For cows with high body condition scores, a fine-grained scoring process with five levels of assessment is constructed using body condition parameters such as ligament visibility, fat accumulation, and visibility of the transverse processes of the lumbar vertebrae. Taking the angle between the hip joint, hip tubercle, and ischial tuberosity as an example, cows with a BCS below 3.25 have a relatively smaller angle, while those above 3.25 have a relatively larger angle. Therefore, this parameter can be used to initially divide the body condition score interval between 3.25 and 3. After the initial division, further division can be made using the hip joint; if its shape is rounded, the body condition score can be determined to be 3, and vice versa. If a cow is initially determined to have a high score, the visibility of the ischial tuberosity can be combined to further determine whether its BCS reaches 3.5 or higher.

[0051] In this embodiment of the invention, firstly, based on expert advice regarding the reasonable range and grading of quantitative body condition parameters for various parts of the cow, and combined with the specific values ​​calculated from the three-dimensional model of the cow, the range of quantitative body condition parameters for the cow is determined. Then, based on the preset scoring calculation method and the quantitative body condition parameters of the cow, the cow's body condition is automatically scored, realizing the correlation calculation between the initial cow image of the cow's spine, both sides of the rump, both sides of the tail root, hip bone, and ischial tuberosity and the actual cow's body condition score, thus obtaining the scoring result of the cow's body condition. Table 2 is the cow body condition scoring step table provided by the present invention, as shown in Table 2.

[0052] Table 2. Steps for scoring the body condition of dairy cows The quantitative body condition parameters for dairy cows include: the position of the distal transverse process of the lumbar vertebrae (LTP), the visibility of the transverse process of the lumbar vertebrae (LTV), the visibility of the sacral ligament (SBLV), the angle between the hip joint, hip tubercle, and ischial tuberosity (AAJS), the fat accumulation between the hip tubercle and ischial tuberosity (FADBS), the fat accumulation at the tail root (FATR), the visibility of the tail root ligament (TRLV), the degree of fat accumulation at the ischial tuberosity (FADSI), the visibility of the hip tubercle (TBJV), and the visibility of the ischial tuberosity (ISJV).

[0053] This invention combines a three-dimensional reconstruction model with a hierarchical decision tree structure to achieve automated scoring of individual dairy cows, unify the evaluation criteria for dairy cow body condition in farms, avoid the subjective influence of traditional manual scoring, effectively reduce farm time and labor, and improve the accuracy of dairy cow body condition scoring, thereby reducing farm costs and improving application efficiency.

[0054] The automated bovine body condition scoring method provided by this invention overcomes the shortcomings of traditional two-dimensional image methods in capturing the three-dimensional features of bovine body condition by acquiring multi-view images of bovines using multiple RGB cameras, thus improving image acquisition quality. It determines the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB cameras corresponding to each viewpoint image. These parameters include the camera's optical center position, the direction of incident light rays, and the camera's sampling step size, thereby reconstructing the three-dimensional structure of the bovine and the motion trajectory of the RGB cameras. Based on the multi-view images, spatial coordinates, and observation angles, an initial neural radiation field network model is constructed, providing a model foundation for rendering bovine images from any viewpoint. Based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters, initial bovine images from any viewpoint are generated, thereby quickly acquiring comprehensive image information of the bovine body condition. Based on the initial bovine images and the quantified bovine body condition parameters, the bovine body condition is automatically scored, obtaining the scoring results and improving the accuracy and efficiency of bovine body condition scoring.

[0055] Further, the step of automatically scoring the cow's body condition based on the initial cow image and the cow's quantified body condition parameters to obtain the cow's body condition score includes: Based on the initial cow image, the initial neural radiation field network model is optimized to obtain the target radiation field network model; Based on the target radiation field network model and the camera intrinsic and extrinsic parameters of the RGB camera corresponding to each viewpoint image, a 3D model of the target cow under the standard viewpoint is generated. Based on the target cow's 3D model and the cow's quantitative body condition parameters, the cow's body condition is automatically scored to obtain the score result.

[0056] Further, the optimization of the initial neural radiation field network model based on the initial cow image to obtain the target radiation field network model includes: The loss value of the initial cow image is calculated based on a preset loss function; the loss function includes smoothness loss, geometric loss, color loss, and grayscale loss. Based on the loss value, backpropagation gradient optimization is performed on the initial neural radiation field network model to obtain an intermediate radiation field network model. Repeat the preset iterative steps to obtain the target radiation field network model.

[0057] Further, the iterative steps include: Based on the intermediate radiation field network model obtained in the previous iteration and the camera intrinsic and extrinsic parameters, the intermediate three-dimensional model of the cow obtained in this iteration is obtained. The projected image of the intermediate 3D cow model is evaluated using preset evaluation metrics to obtain evaluation results; the evaluation metrics include mean absolute error, maximum F-value, weighted F-value, structural similarity measure, and enhanced alignment measure. If the evaluation result meets the preset threshold requirement, the iteration step is stopped, and the intermediate radiation field network model corresponding to the intermediate cow three-dimensional model is used as the target radiation field network model. If the evaluation result does not meet the preset threshold requirement, an intermediate cow image from any perspective is generated based on the intermediate radiation field network model corresponding to the intermediate cow 3D model and the camera intrinsic and extrinsic parameters. Based on the intermediate cow image, the intermediate neural radiation field network model is optimized to obtain the intermediate radiation field network model obtained in this iteration.

[0058] Specifically, after generating initial cow images from any viewpoint, the loss value of the initial cow images is calculated based on a preset loss function.

[0059] The loss functions include smoothness loss, geometric loss, color loss, and grayscale loss.

[0060] The gradient is calculated by backpropagation, and the network parameters are iteratively optimized to gradually converge the initial neural radiation field network model to obtain the intermediate radiation field network model. Then, the preset iterative steps are repeated to gradually converge the intermediate neural radiation field network model to the optimal representation of the target scene, thus obtaining the target radiation field network model.

[0061] The loss function formula is as follows: In the formula, These represent color loss, geometric loss, smoothness loss, and grayscale loss, respectively. It is a hyperparameter that represents the weight of the geometric loss. It is a hyperparameter that represents the weight of the smoothness loss. These are hyperparameters that represent the weight of the grayscale loss. These hyperparameters are used to balance the various losses.

[0062] Color loss The expression is: In the formula, It is the total number of training rays. It is a specific ray of light. This represents the color value of the initial cow image from any viewpoint obtained after rendering. Represents the color value of the actual image.

[0063] In the formula, K is the number of sampling points along the direction of the light ray. It is the transmittance from the origin of the light ray to the k-th sampling point where it is not blocked. The weight of the k-th sampling point represents the density contribution of the sampling point. It is the true color of the kth sampling point.

[0064] In the formula, This represents the density of the j-th sampling point. Indicates the distance between adjacent sampling points. Indicates opacity (Alpha value). This represents the color of the k-th sampling point.

[0065] Geometric loss The expression is: In the middle of the time, yes The gradient of the normal vector direction.

[0066] Smoothness loss The expression is: In the formula, The second derivative at that point is expressed in terms of curvature.

[0067] Grayscale loss The expression is: In the formula, This represents the grayscale value obtained from the integration. This represents the grayscale value of the actual value.

[0068] This invention improves model performance through joint optimization of multiple loss functions to obtain a target radiation field network model. It comprehensively considers factors such as geometric consistency, color accuracy, and image smoothness, thereby significantly improving the geometric accuracy and visual quality of the first target cow image generated by the subsequent target radiation field network model.

[0069] To measure whether the intermediate neural radiation field network model meets the preset requirements, this embodiment of the invention uses the intermediate radiation field network model and camera intrinsic and extrinsic parameters to generate an intermediate 3D model of a cow. Then, the projected image of the intermediate 3D model of the cow is evaluated using preset evaluation indicators to obtain the evaluation results.

[0070] The embodiments of the present invention set a mean absolute error (MAE). ), maximum F-measure, ), weighted F-measure, ), structural similarity measure (Structure measure, Enhanced alignment measure Five evaluation metrics are used to assess the projected image (predictive mask) of the intermediate dairy cow 3D model, as follows: It is used to measure the pixels of the predicted mask. With the corresponding real mask pixels The absolute error between the predicted and actual results is expressed as follows: A smaller value indicates that the predicted result is closer to the actual result. In the formula, This represents the width (number of columns) of the mask image, i.e., the number of pixels per row. This represents the height (number of rows) of the mask image, which is the number of pixels per column. The goal is to achieve normalization, which involves comparing the target pixels separately in rows and columns.

[0071] The formula used to measure the overall segmentation performance of a high-precision binary image segmentation model at the optimal threshold is as follows: It indicates the model's comprehensive performance in accurately identifying and fully scoring targets. In the formula, This represents the proportion of foreground pixels that belong to the true foreground in the predicted mask; This represents the proportion of foreground pixels that are accurately predicted as foreground pixels in the actual mask. These are parameters used to control the weights between them. It is the maximum value calculated under different thresholds.

[0072] Is Based on this, a weighting mechanism is introduced, assigning higher weights to pixels closer to the object boundary, resulting in a more refined evaluation of segmentation quality. The formula is as follows: In the formula, and These represent weighted precision and weighted recall, respectively, used to weight pixels based on their distance from the edge, making pixels in edge regions more sensitive to the metric.

[0073] To assess structural similarity, the focus is primarily on shape, contour, and spatial relationships. In embodiments of the present invention... Structural similarity is assessed using the Intersection over Union (IoU) ratio, as shown in the formula below: In the formula, and These correspond to the mask ranges of the predicted results and the true labels, respectively. When the value approaches 1, it indicates that the two structures are more similar.

[0074] It is used to evaluate pixel-level and image-level matching, taking into account the similarity between the predicted result and the real mask at both the pixel level and the overall image level. The higher the value, the greater the consistency between the model's predictions and the actual situation at the pixel and image levels, and the better the model's performance. The formula is as follows: In the formula, It is the predicted prospect probability; It is a true foreground annotation; It is a minimum value used to avoid the denominator being 0.

[0075] The projected image of the intermediate cow 3D model is evaluated using five preset evaluation metrics. If the evaluation results meet preset threshold requirements, the iteration process stops, and the intermediate radiation field network model corresponding to the intermediate cow 3D model is taken as the target radiation field network model. If the evaluation results do not meet the preset threshold requirements, an intermediate cow image from any viewpoint is generated based on the intermediate radiation field network model corresponding to the intermediate cow 3D model and the camera's intrinsic and extrinsic parameters. Then, based on the smoothness loss, geometric loss, color loss, and grayscale loss of the intermediate cow image, backpropagation is performed to calculate the gradient, and the network parameters are iteratively optimized to further converge the intermediate neural radiation field network model. The intermediate neural radiation field network model is optimized until the target radiation field network model is obtained.

[0076] After obtaining the target radiation field network model, a 3D model of the target cow under a standard viewing angle is generated using the target radiation field network model and the camera's intrinsic and extrinsic parameters. Then, using the 3D model of the target cow and its corresponding RFID or ID information as input, the cow's body condition is automatically scored according to the preset scoring calculation method in Table 2 and the cow's quantitative body condition parameters, obtaining the score result. The standard viewing angle of the 3D model is a 45° view directly above the cow's tail. From this angle, the concave and convex structural features of the cow's tail and the curvature features of the cow's back can be clearly presented.

[0077] This invention provides a target radiation field network model through multi-index comprehensive evaluation, which in turn generates a target three-dimensional model of the dairy cow from a standard perspective. Based on the target dairy cow three-dimensional model, the curvature data and various quantitative body condition parameters of the dairy cow are calculated, and the body condition of the dairy cow is scored with fine granular precision. This significantly improves the accuracy of the scoring of the dairy cow's body condition and ensures the accuracy and reliability of the scoring results.

[0078] The following describes an automated dairy cow body condition scoring device provided by the present invention. The automated dairy cow body condition scoring device described below can be referred to in correspondence with the automated dairy cow body condition scoring method described above.

[0079] Based on any of the above embodiments Figure 5 This is a schematic diagram of the structure of an automated body condition scoring device for dairy cows provided by the present invention, as shown below. Figure 5 As shown. This embodiment of the invention provides an automated dairy cow body condition scoring device, including a data acquisition module 501, a three-dimensional reconstruction module 502, a three-dimensional mapping module 503, a generation module 504, and a scoring module 505, wherein: The acquisition module 501 is used to acquire multi-view images of cows using multiple RGB cameras; the 3D reconstruction module 502 is used to determine the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB camera corresponding to each view image; the intrinsic and extrinsic parameters include the camera optical center position, the direction of the incident light ray, and the camera sampling step size; the 3D mapping module 503 is used to construct an initial neural radiation field network model based on the multi-view images, the spatial coordinates, and the observation angle; the generation module 504 is used to generate initial cow images from arbitrary viewpoints based on the initial neural radiation field network model and the intrinsic and extrinsic parameters of the cameras; the scoring module 505 is used to automatically score the cow's condition based on the initial cow images and the cow's quantified body condition parameters, and obtain the scoring result of the cow's condition.

[0080] The automated dairy cow body condition scoring device provided by this invention acquires multi-view images of dairy cows using multiple RGB cameras, overcoming the shortcomings of traditional two-dimensional image methods in capturing the three-dimensional features of dairy cow body condition and improving image acquisition quality. It determines the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB cameras corresponding to each viewpoint image. These intrinsic and extrinsic parameters include the camera's optical center position, the direction of incident light rays, and the camera's sampling step size, thereby recovering the three-dimensional structure of the dairy cow and the motion trajectory of the RGB cameras. Based on the multi-view images, spatial coordinates, and observation angles, an initial neural radiation field network model is constructed, providing a model foundation for rendering dairy cow images from any viewpoint. Based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters, an initial dairy cow image from any viewpoint is generated, thereby quickly acquiring comprehensive image information of the dairy cow's body condition. Based on the initial dairy cow image and the quantified body condition parameters of the dairy cow, the dairy cow's body condition is automatically scored, obtaining the scoring results and improving the accuracy and efficiency of dairy cow body condition scoring.

[0081] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute an automated cow body condition scoring method, which includes: Multiple RGB cameras were used to capture multi-view images of dairy cows. Determine the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB camera corresponding to each viewpoint image; the intrinsic and extrinsic parameters include the camera optical center position, the direction of the incident light rays, and the camera sampling step size. Based on the multi-view images, the spatial coordinates, and the observation angle, an initial neural radiation field network model is constructed. Based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters, an initial cow image from any viewpoint is generated; Based on the initial cow image and the cow's quantitative body condition parameters, the cow's body condition is automatically scored to obtain the score result.

[0082] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the automated cow body condition scoring method provided by the above methods, the method comprising: Multiple RGB cameras were used to capture multi-view images of dairy cows. Determine the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB camera corresponding to each viewpoint image; the intrinsic and extrinsic parameters include the camera optical center position, the direction of the incident light rays, and the camera sampling step size. Based on the multi-view images, the spatial coordinates, and the observation angle, an initial neural radiation field network model is constructed. Based on the initial neural radiation field network model and the phase internal and external machine parameters, an initial cow image from any viewpoint is generated; Based on the initial cow image and the cow's quantitative body condition parameters, the cow's body condition is automatically scored to obtain the score result.

[0084] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the automated cow body condition scoring method provided by the methods described above, the method comprising: Multiple RGB cameras were used to capture multi-view images of dairy cows. Determine the spatial coordinates, observation angle, and intrinsic and extrinsic parameters of the RGB camera corresponding to each viewpoint image; the intrinsic and extrinsic parameters include the camera optical center position, the direction of the incident light rays, and the camera sampling step size. Based on the multi-view images, the spatial coordinates, and the observation angle, an initial neural radiation field network model is constructed. Based on the initial neural radiation field network model and the camera intrinsic and extrinsic parameters, an initial cow image from any viewpoint is generated; Based on the initial cow image and the cow's quantitative body condition parameters, the cow's body condition is automatically scored to obtain the score result.

[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0087] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0088] It should also be noted that the terms "target," "first," and "second" in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more.

[0089] In this application's embodiments, "determine B based on A" means that factor A must be considered when determining B. It is not limited to "B can be determined based solely on A," but should also include: "determine B based on A and C," "determine B based on A, C, and E," "determine C based on A, and further determine B based on C," etc. Additionally, it can include using A as a condition for determining B, for example, "when A meets the first condition, determine B using the first method"; another example, "when A meets the second condition, determine B," etc.; another example, "when A meets the third condition, determine B based on the first parameter," etc. Of course, it can also be a condition where A is a factor in determining B, for example, "when A meets the first condition, determine C using the first method, and further determine B based on C," etc.

[0090] In this invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automated scoring of dairy cow body condition, characterized in that, The method comprises the following steps: acquiring multi-view images of a cow through multiple RGB cameras; determining the spatial coordinates, observation angles and camera internal and external parameters of the RGB cameras corresponding to each view image; the camera internal and external parameters comprise camera optical center positions, camera incident light directions and camera sampling steps; constructing an initial neural radiance field network model based on the multi-view images, the spatial coordinates and the observation angles; generating an initial cow image of an arbitrary view based on the initial neural radiance field network model and the camera internal and external parameters; automatically scoring the body condition of the cow based on the initial cow image and the quantitative body condition parameters of the cow, and obtaining a scoring result of the body condition of the cow.

2. The method of automatic body condition scoring of dairy cows according to claim 1, characterized in that, The step of constructing an initial neural radiance field network model based on the multi-view images, the spatial coordinates and the observation angles comprises the following steps: mapping the spatial coordinates and the observation angles to a three-dimensional feature space; determining the RGB color values and the body density of each spatial point in the three-dimensional feature space based on the multi-view images; constructing an initial neural radiance field network model based on the RGB color values and the body density of each spatial point.

3. The method of automatic body condition scoring of dairy cows according to claim 2, characterized in that, The step of generating an initial cow image of an arbitrary view based on the initial neural radiance field network model and the camera internal and external parameters comprises the following steps: performing layered integral sampling along the camera incident light directions based on the camera optical center positions and the camera sampling steps, and obtaining multiple three-dimensional spatial points; determining the RGB color values and the body density of each three-dimensional spatial point in the initial neural radiance field network model; generating an initial cow image of an arbitrary view based on the RGB color values and the body density of each three-dimensional spatial point in the initial neural radiance field network model.

4. The method of automatic body condition scoring of dairy cows according to claim 3, characterized in that, The step of automatically scoring the body condition of the cow based on the initial cow image and the quantitative body condition parameters of the cow, and obtaining a scoring result of the body condition of the cow comprises the following steps: optimizing the initial neural radiance field network model based on the initial cow image, and obtaining a target radiance field network model; generating a target cow three-dimensional model under a standard view based on the target radiance field network model and the camera internal and external parameters; automatically scoring the body condition of the cow based on the target cow three-dimensional model and the quantitative body condition parameters of the cow, and obtaining a scoring result of the body condition of the cow.

5. The method of claim 4, wherein, The step of optimizing the initial neural radiance field network model based on the initial cow image, and obtaining a target radiance field network model comprises the following steps: calculating a loss value of the initial cow image based on a preset loss function; the loss function comprises a smoothness loss, a geometry loss, a color loss and a grayscale loss; optimizing the initial neural radiance field network model through back propagation of a gradient based on the loss value, and obtaining an intermediate radiance field network model; repeating a preset iteration step to obtain a target radiance field network model.

6. The method of automatic body condition scoring of a dairy cow according to claim 5, characterized in that, The iteration step comprises the following steps: obtaining an intermediate cow three-dimensional model obtained in the current iteration based on the intermediate radiance field network model obtained in the last iteration and the camera internal and external parameters; The projection image of the intermediate cow three-dimensional model is evaluated by a preset evaluation index to obtain an evaluation result; the evaluation index includes mean absolute error, maximum F value, weighted F value, structural similarity measurement, and enhanced alignment measurement; In a case where the evaluation result meets a preset threshold requirement, the iteration step is stopped, and the intermediate radiance field network model corresponding to the intermediate cow three-dimensional model is taken as the target radiance field network model; In a case where the evaluation result does not meet the preset threshold requirement, an intermediate cow image of an arbitrary view is generated based on the intermediate radiance field network model corresponding to the intermediate cow three-dimensional model and the camera internal and external parameters; The intermediate neural radiance field network model is optimized based on the intermediate cow image to obtain an intermediate radiance field network model obtained in the current iteration.

7. An automated dairy cow body condition scoring device, characterized by, Comprise: The acquisition module is used for acquiring multi-view images of a cow through a plurality of RGB cameras; The three-dimensional reconstruction module is used for determining the spatial coordinates, observation angles and camera internal and external parameters of the RGB camera corresponding to each view image; the camera internal and external parameters include camera optical center position, camera incident light direction and camera sampling step; The three-dimensional mapping module is used for constructing an initial neural radiance field network model based on the multi-view images, the spatial coordinates and the observation angles; The generation module is used for generating an initial cow image of an arbitrary view based on the initial neural radiance field network model and the camera internal and external parameters; The scoring module is used for automatically scoring the body condition of the cow based on the initial cow image and the quantitative body condition parameters of the cow to obtain a score result of the body condition of the cow.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the cow body condition automatic scoring method in any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the cow body condition automatic scoring method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the cow body condition automatic scoring method in any one of claims 1 to 6.

Images