A method for reconstructing a three-dimensional model of a cow face and generating point cloud data based on multi-angle two-dimensional images

Abstract

A method for reconstructing a three-dimensional model of a cow face and generating point cloud data based on multi-angle two-dimensional images is provided. The method includes: obtaining a front face image, a left 45-degree face image, and a right 45-degree face image of a face of a to-be-identified cow, and generating a 3D mesh model of the face of the to-be-identified cow; identifying the availability of the 3D mesh model according to an index of the 3D mesh model for predicting the availability of the 3D mesh model; for the 3D mesh model having an availability greater than a specified threshold, detecting and removing black edges of the 3D mesh model and converting the 3D mesh model into a 3D point cloud; cropping the aligned 3D point cloud to remove a 3D point cloud region corresponding to a cheek of the cow face and retain a 3D point cloud region above the cheek; generating a 3D feature vector according to the cropped 3D point cloud; and obtaining a candidate result most similar to the 3D feature vector from a cow feature database according to the 3D feature vector and sorting the candidate result according to similarity.

Description

Technical Field

[0001] This invention relates to livestock identification technology, and more specifically, to a method for reconstructing a three-dimensional model of a cow's face and generating point cloud data based on multi-angle two-dimensional images. Background Technology

[0002] Identification applications aim to determine the uniqueness of an individual target. This differs from category identification, which focuses on the type of target, by identifying which of multiple objects of the same type the target belongs to. Currently, physical identification methods such as ear tags and radio frequency identification (RFID) are commonly used in livestock farms. However, these methods suffer from problems such as fragility, loss, the need for close contact, and the risk of animal stress. Therefore, the industry is dedicated to researching non-contact, stress-free biometric identification technologies to achieve more intelligent and traceable livestock management. These technologies have significant application value in areas such as dairy cow performance testing, precision feeding of beef cattle, disease prevention and control, breed registration, and agricultural insurance.

[0003] In the field of biometric-based visual recognition, existing research is mainly divided into two categories: identity recognition based on two-dimensional images and identity recognition based on three-dimensional point clouds.

[0004] 1. A two-dimensional image-based cow face recognition scheme for identifying cattle. This scheme draws on mature facial recognition technology, directly extracting and comparing features from acquired 2D images of cow faces. The implementation process typically involves the following: First, preprocessing the input cow face image, including grayscale conversion, normalization, keypoint detection, and alignment; then, using a deep convolutional neural network (such as FaceNet or VGG) as a feature extractor to map the input image into a high-dimensional feature vector; finally, by calculating the cosine or Euclidean distance between the feature vector of the image to be identified and the pre-stored feature vectors in the database, identity recognition is achieved based on similarity ranking.

[0005] Such methods are widely used in academic research. For example, in patent literature (such as Chinese patent application CN120148068A, "A method for recognizing cow faces based on multi-scale feature interaction and its recognition system"), convolutional neural networks are used to identify the identity of cows.

[0006] 2. A cattle identification scheme based on 3D point cloud for identifying cattle heads. This scheme aims to overcome the shortcomings of two-dimensional images, which are easily affected by environmental interference, by utilizing the three-dimensional geometric structure information of the cow's head for identification. Its typical implementation process is as follows: a three-dimensional point cloud model of the cow's head is reconstructed using a 3D scanning device or from multiple 2D images; subsequently, a deep learning network specifically designed for processing point cloud data is used to learn features and classify the point cloud; the network output is either the corresponding cow's identification ID or a point cloud feature vector used for comparison.

[0007] This type of method falls under the application of 3D target recognition technology in computer vision to the agricultural field. Patent literature (such as Chinese patent application number CN119992588A, "A method for recognizing cow faces based on dense point cloud 3D reconstruction") discloses a technical solution that achieves individual recognition by acquiring dense 3D point clouds of cow faces and using deep learning models for feature matching.

[0008] Although the aforementioned technical solutions have made some progress, they all rely on information from a single mode and have inherent, insurmountable performance bottlenecks. Specific shortcomings are analyzed below: 1. Deficiencies of recognition schemes based on two-dimensional images The core flaw of this scheme lies in its feature stability, which is heavily dependent on the imaging environment. A two-dimensional image is essentially a projection of a three-dimensional scene under specific lighting conditions; therefore: Susceptible to changes in lighting: The facial texture and color of the same cow vary greatly under different times, weather conditions, or indoor and outdoor light sources, causing significant shifts in the extracted feature vectors and drastic fluctuations in recognition accuracy.

[0009] Sensitive to pose and occlusion: When a cow's head turns, tilts, or its face is partially obscured by feed or dirt, the texture information captured in the image is incomplete or deformed, leading to feature matching failure. This deficiency stems from the fact that 2D methods lack an understanding of the target's 3D structure and cannot compensate at the geometric level. Moreover, cows do not actively cooperate with the 2D image acquisition process, and the environment in which they are located is usually not conducive to image acquisition, which further exacerbates the problems caused by pose and occlusion.

[0010] Limited ability to distinguish similar individuals: For cattle of the same breed, with similar coat color and pattern, it is difficult to make a high-confidence distinction based solely on the appearance and texture features, which can easily lead to misidentification.

[0011] 2. Deficiencies of 3D point cloud-based recognition schemes 3D point cloud-based cow face recognition technology has advantages such as resistance to light interference and capture of geometric features, avoiding some environmental sensitivity of two-dimensional images, and is therefore gradually replacing traditional 2D recognition solutions.

[0012] However, high-quality datasets are a core prerequisite for 3D cow face recognition. While specialized equipment (such as depth cameras, binocular cameras, 3D scanners, and LiDAR) can obtain relatively high-quality 3D point clouds, it significantly increases costs and reduces the applicability of the technical solution. For example, such specialized equipment is typically unavailable in cattle ranches and farms. 3D point clouds can be generated using multiple 2D images. However, the quality of the generated 3D point clouds fluctuates greatly; some may contain structural defects or noise masking core features. This leads to uncontrolled 3D point cloud quality, ultimately making it difficult to meet the recognition accuracy requirements of large-scale farm applications. Furthermore, low-quality 3D point clouds not only affect the cattle identification process but also severely impact the training performance of neural networks when using neural network-based 3D feature extraction schemes for cattle.

[0013] Existing technologies for processing 3D model datasets of cattle identification have several shortcomings.

[0014] 1. Dataset source quality out of control: The 3D reconstruction process based on 2D images directly converts 3D models into 3D point clouds without quantitatively evaluating the integrity, geometric consistency, and noise content of the 3D models, resulting in low-quality 3D models polluting the dataset; 2. Low noise filtering accuracy, interference with effective signals in the dataset: The use of a global single threshold filter without combining prior features of cow face color makes it impossible to accurately distinguish between black edge noise and effective features, resulting in a large amount of invalid noise or erroneous deletion of effective features in the dataset, thus reducing the quality of the dataset. 3. Poor point cloud consistency and insufficient dataset feature consistency: No systematic scale standardization was performed, and only a single initial pose was used for ICP registration. Point clouds from different acquisition scenarios and different cattle showed significant differences in scale and pose. The 3D point cloud features of the same cattle collected at different times had low consistency, and the dataset did not have standardization characteristics. 4. Severe data redundancy and low proportion of effective features in the dataset: The complete point cloud data of cow faces is retained, but non-critical areas are not removed. These areas have large individual differences and a lot of dynamic noise, which mask the core features, resulting in an insufficient proportion of effective features in the dataset and increasing the redundant cost of subsequent model training and recognition. 5. Insufficient algorithm adaptability and weak dataset generalization ability: The differences in point cloud density between different breeds of cattle and different collection scenarios were not considered. The preprocessing parameters were fixed, which made it difficult for the dataset to be compatible with diverse scenarios and resulted in insufficient generalization ability. 6. Disconnected technology links, resulting in a lack of coordination between dataset quality and reconstruction effect: The 3D reconstruction process and subsequent data processing are independent of each other, and the built-in indicators of the reconstruction algorithm are not utilized, resulting in a disconnect between reconstruction quality and dataset optimization effect, and an inability to form a quality closed loop.

[0015] To address the aforementioned shortcomings in the data processing of existing cattle identification 3D model datasets, this invention takes "constructing a high-quality, highly consistent, and highly pure cattle point cloud dataset" as its core objective. Based on the Dust3R 3D reconstruction model, it constructs an integrated processing solution encompassing reconstruction, verification, correction, preprocessing, and optimization, providing underlying data support for cattle identification.

[0016] This application also provides a cattle identification method based on 3D features, using a processed 3D model dataset. Furthermore, it integrates the advantages of two-dimensional and three-dimensional modalities to achieve highly accurate and stable cattle identification in complex real-world scenarios. Summary of the Invention

[0017] According to a first aspect of this application, a method for reconstructing a 3D model of a cow's face and generating point cloud data based on multi-angle 2D images for cattle identification is provided. The method includes: acquiring a frontal cow face image, a left 45-degree cow face image, and a right 45-degree cow face image; inputting the frontal cow face image, the left 45-degree cow face image, and the right 45-degree cow face image into a Dust3R model to generate a 3D mesh model of the cow's face; and predicting the usability of the 3D mesh model. The prediction of the usability of the 3D mesh model includes: acquiring indicators for predicting the usability of the 3D mesh model, including the best edge confidence score and the second-best edge confidence score output by the Dust3R model, and the Dust... The 3R model generates the 3D mesh model, including the low-confidence point cleanup ratio, average confidence level, and high-confidence point ratio. The low-confidence point cleanup ratio refers to the ratio of the number of vertices with a confidence level below a first threshold to the total number of vertices in the 3D mesh model output. The average confidence level refers to the average confidence level of the vertices in the 3D mesh model output. The high-confidence point ratio refers to the ratio of the number of vertices with a confidence level above a second threshold to the total number of vertices in the 3D mesh model output, where the second threshold is higher than the first threshold. A usability verification model based on the random forest algorithm is used to predict the... The method describes an index for the usability of a 3D mesh model, identifies the usability of the 3D mesh model; for 3D mesh models with usability greater than a specified threshold, detects and removes black edges from the 3D mesh model; converts the 3D mesh model with black edges removed into a 3D point cloud; scales the 3D point cloud to a uniform unit scale, and transforms the pose of the 3D point cloud to be consistent with a reference point cloud, obtaining an aligned 3D point cloud, wherein the reference point cloud represents a 3D point cloud of a cow face with a standard pose; and crops the aligned 3D point cloud, removing the 3D point cloud region corresponding to the cow face cheeks, while retaining the 3D point cloud region above the cheeks. A 3D feature vector is generated based on the cropped 3D point cloud; the candidate results most similar to the 3D feature vector are obtained from the cattle feature database by sorting them by similarity; wherein, the cattle feature database records the cattle's identity and its 3D feature vector in association.

[0018] According to the data processing method of the first aspect of this application, the 3D model training samples for training the usability verification model based on the random forest algorithm include the best edge confidence score, the second-best edge confidence score, the proportion of low confidence points cleaned, the average confidence score, and the proportion of high confidence points obtained when the 3D mesh model is generated by the Dust3R model based on cow face images covering multiple cow breeds, multiple cow face image acquisition distances, and cow face image acquisition lighting conditions, as well as label data that distinguishes whether the 3D mesh model is qualified; the usability verification model based on the random forest algorithm is trained using the 3D model training samples.

[0019] According to the data processing method of the first aspect of this application, the method for detecting and removing black edges of the 3D mesh model includes: detecting and removing black edges one by one from each viewpoint of the 3D model output by the Dust3R model; wherein, for each viewpoint of the 3D model, all vertices are traversed, and the sum of the color component values ​​of the R, G, and B color channels of each vertex is calculated; when the sum of the color component values ​​of the R, G, and B channels of a vertex is lower than the determination threshold of 0.03, the vertex is determined to be a black edge, and the vertex is removed from the 3D model of the current viewpoint; the 3D mesh model is obtained by fusing the 3D models of each viewpoint that have had their black edges removed.

[0020] According to the data processing method of the first aspect of this application, scaling the 3D point cloud to a uniform unit scale includes: calculating the centroid of the 3D point cloud; subtracting the centroid coordinates from the coordinates of each point in the 3D point cloud, such that the centroid of the 3D point cloud coincides with the origin of the coordinate system; calculating the dimensions of the bounding box of the 3D point cloud in the three coordinate axis directions; and scaling the coordinates of each point in the 3D point cloud using the maximum value of the dimensions of the bounding box of the 3D point cloud in the three coordinate axis directions.

[0021] According to the data processing method of the first aspect of this application, the method involves: transforming the pose of the 3D point cloud to be consistent with that of the reference point cloud, comprising: acquiring multiple pose rotation combinations, the multiple pose rotation combinations representing possible poses of the cow face during the acquisition of the cow face image; wherein, the pose rotation combination includes rotation angles around each of the three coordinate axes; for each pose rotation combination, rotating the 3D point cloud according to its rotation angle to obtain a rotated 3D point cloud; performing ICP iterative registration between the rotated 3D point cloud and the reference point cloud to obtain the optimal transformation matrix T and the best registration quality score of the rotated 3D point cloud relative to the reference point cloud, wherein the best registration quality score represents the difference between the rotated 3D point cloud and the reference point cloud after transformation with the optimal transformation matrix T; performing ICP iterative registration between the rotated 3D point cloud corresponding to each pose rotation combination and the reference point cloud to obtain the historically optimal optimal transformation matrix and the historically optimal registration quality score; and transforming the 3D point cloud using the historically optimal optimal transformation matrix.

[0022] According to the data processing method of the first aspect of this application, cropping the aligned 3D point cloud to remove the 3D point cloud region corresponding to the cow face cheek includes: fitting a cow face overall geometric reference plane from the aligned 3D point cloud, wherein the cow face overall geometric reference plane represents the plane with the flattest direction of the aligned 3D point cloud distribution, and its plane equation is represented by n·x + d = 0; where n represents the unit normal vector of the plane equation, and d is a plane constant parameter; wherein, based on the aligned 3D point cloud, a 3×3 covariance matrix is ​​constructed; the eigenvalues ​​of the covariance matrix are decomposed; the eigenvector corresponding to the smallest eigenvalue is taken, and the vector is normalized to obtain the unit normal vector n of the plane; the parameter d is solved by the constraint condition of the cow face overall geometric reference plane passing through the centroid of the aligned 3D point cloud; finally, the plane equation n·x + d = 0 is obtained; points located below the cow face overall geometric reference plane are removed from the aligned 3D point cloud, and points located above the cow face overall geometric reference plane are retained.

[0023] According to the data processing method of the first aspect of this application, if the normal vector of the plane equation represented by n points in a specified direction, the plane equation is flipped by n = -n and d = -d; wherein the specified direction is the direction in which its Z-axis component is negative, and the Z-axis is a coordinate axis perpendicular to the ground and pointing upwards.

[0024] According to the data processing method of the first aspect of this application, the signed distance from a point x in the 3D point cloud to the face plane is calculated according to s = n·x + d, where x is the coordinate vector of the point in the 3D point cloud; the step of deleting points located below the face plane relative to the face plane from the 3D point cloud includes: deleting the corresponding point x from the 3D point cloud when s < 0; and retaining the corresponding point x in the 3D point cloud when s > 0.

[0025] According to the data processing method of the first aspect of this application, for the 3D mesh model whose availability is not greater than a specified threshold, it is not used to identify the identity of the cattle to be identified, nor is it used to generate 3D feature vectors.

[0026] According to the data processing method of the first aspect of this application, the acquisition of a frontal cow face image, a left 45-degree cow face image, and a right 45-degree cow face image includes: when acquiring the cow face image, identifying the cow face posture in real time using a cow face recognition model based on YOLOv8; triggering a capture to acquire the cow face image when the acquired cow face posture is in a frontal view, a left 45-degree view, or a right view; extracting the cow face region from the acquired cow face image and deleting the background outside the cow face region to obtain a segmented cow face image; and comparing the segmented cow face image with the cow face image. The optimal viewing angle template is matched to output a qualified cow face image with the same posture as the optimal viewing angle template. The optimal viewing angle template includes a preset frontal cow face image, a left 45° cow face image, and a right 45° cow face image. If the obtained qualified cow face images include at least one qualified cow face image from each of the three views (frontal cow face image, left 45° cow face image, and right 45° cow face image), then 3D feature vectors are generated using these qualified cow face images, and 2D feature vectors are generated using the qualified cow face images from the frontal view, left 45° view, and / or right 45° view.

[0027] According to the data processing method of the first aspect of this application, the step of generating a 3D feature vector from the cropped 3D point cloud includes: using a PointNet++ model to receive the cropped 3D point cloud and generate a 3D feature vector, wherein the output head of the PointNet++ model is replaced with a feature vector output head, wherein the feature vector output head receives the output of the last SA layer of the PointNet++ model and generates a 3D feature vector representing the 3D point cloud; wherein the feature vector output head includes multiple neural network layer groups, each neural network layer group including a feature compression layer, a batch normalization (BN) layer, a ReLU activation function layer, and a Dropout layer; the feature vector output head also includes an L2 normalization layer to output a normalized 3D feature vector.

[0028] According to the data processing method of the first aspect of this application, the training samples for training the PointNet++ model are triplets including anchor samples, positive samples, and negative samples; wherein, the anchor samples serve as a reference benchmark; the positive samples and anchor samples are 3D point clouds obtained from cow face images corresponding to the same cow; the negative samples and anchor samples are 3D point clouds from different cows; and the loss function L = max(d(a,p) - d(a,n) + ... The PointNet++ model is trained using the margin, 0) function, where a, p, and n represent the feature vectors of anchor samples, positive samples, and negative samples, respectively; d() represents the Euclidean distance calculation function; and margin represents the preset boundary value. The optimization objective of the training process is: for each anchor sample, the feature distance between the anchor sample and the positive sample is adjusted through network parameters to make the feature distance between the anchor sample and the negative sample smaller than the feature distance between the anchor sample and the negative sample, and the difference between the two must meet the preset boundary value (margin) requirement. During training, semi-hard samples and hard samples are selected from the training samples. The selection criteria for semi-hard samples are: the similarity between the feature vector of the negative sample and the feature vector of the anchor is greater than the distance between the feature vector of the positive sample and the feature vector of the anchor sample, and this distance difference is less than the margin. The selection criteria for hard samples are: the distance between the feature vector of the negative sample and the feature vector of the anchor sample is less than the distance between the feature vector of the positive sample and the feature vector of the anchor sample. During training, semi-hard samples are used for training first, then semi-hard samples and hard samples are used together for training, and finally hard samples are used for training.

[0029] The data processing method according to the first aspect of this application further includes: generating a 2D feature vector based on a frontal cow face image, a left 45-degree cow face image, and / or a right 45-degree cow face image; obtaining a first plurality of candidate results most similar to the 2D feature vector from a cow feature database according to similarity sorting based on the 2D feature vector; obtaining a second plurality of candidate results most similar to the 3D feature vector from a cow feature database according to similarity sorting based on the 3D feature vector; if the first plurality of candidate results and the second plurality of candidate results have an intersection, and the confidence level of the candidate results in the intersection is greater than a threshold, the candidate results in the intersection with a confidence level greater than the threshold are used as the identification result of the cow to be identified. According to the second aspect of this application, an information processing apparatus according to the second aspect of this application is provided, including a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that the processor implements the method according to the first aspect of this application when executing the program.

[0030] The technical effects achieved by this application include: 1. Establish a closed-loop mechanism for 3D reconstruction, verification, and correction. Select high-quality 3D models from the source of the dataset, trigger re-collection and reconstruction for unqualified models, and ensure that every piece of data entering the dataset meets the quality requirements. 2. Reuse the built-in metrics of the 3D reconstruction model to achieve synergy between reconstruction quality and dataset optimization effect, and improve the controllability of dataset quality; 3. Accurately remove black edge noise, purify the dataset, and avoid invalid information interfering with core features; 4. Eliminate scale differences and pose shifts in point clouds to improve feature consistency and standardization of the dataset; 5. Focus on high-discrimination core feature regions to increase the proportion of effective features in the dataset and reduce redundant costs; Finally, a high-quality 3D cow face dataset was constructed, providing excellent data support for subsequent feature extraction and recognition model training, and significantly improving the accuracy, robustness and engineering practicality of the cow face recognition system. Attached Figure Description

[0031] When read in conjunction with the accompanying drawings, and by referring to the following detailed description of illustrative embodiments, this application, its preferred modes of use, and further objects and advantages will be best understood, wherein the drawings include: Figure 1A The working principle of a cattle identification method according to one embodiment of this application is demonstrated.

[0032] Figure 1B The working principle and processing flow of a cattle identification method according to another embodiment of this application are demonstrated.

[0033] Figure 2 A schematic diagram illustrating the process of acquiring and preprocessing a cow's face image according to an embodiment of this application is shown.

[0034] Figure 3 A schematic diagram illustrating the process of generating and processing a cow face 3D point cloud according to an embodiment of this application is shown.

[0035] Figure 4A This shows a 3D model view of the cow's face before the black edges were removed.

[0036] Figure 4B A 3D model view of a cow's face after removing black edges using an embodiment of this application is shown.

[0037] Figure 5 The flowchart illustrates the size standardization stage in the 3D point cloud alignment process of the data processing flow of a 3D model according to an embodiment of this application.

[0038] Figure 6The flowchart illustrates the multi-angle ICP pose alignment stage in the 3D point cloud alignment process of the 3D model data processing flow according to an embodiment of this application.

[0039] Figure 7 The before-and-after effects of 3D point cloud alignment according to an embodiment of this application are shown.

[0040] Figure 8 A flowchart illustrating the key region trimming stage of the data processing flow for a 3D model according to an embodiment of this application is shown.

[0041] Figure 9 The images show a comparison view before and after cropping the 3D point cloud of a cow's face according to an embodiment of this application.

[0042] Figure 10 A schematic diagram of the network structure of the feature vector output head according to an embodiment of this application is shown.

[0043] Figure 11 This is a block diagram of an information processing device that implements the embodiments of this application. Detailed Implementation

[0044] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0045] This application embodiment processes the 3D model in the process of generating 3D features of cattle using 2D image data for cattle identification. By reconstructing a 3D mesh model of a cow's face from multi-view 2D images, the 3D model data is optimized layer by layer in the order of quality screening, precise noise reduction, point cloud transformation, standardized alignment, and key region extraction, resulting in a standardized, high-purity target point cloud.

[0046] The core processing flow of the 3D model data in this application embodiment is as follows: 3D mesh model of cow face → 3D model usability verification (quality screening) → black edge noise filtering (precise noise reduction) → generation of 3D point cloud → point cloud standardization (pose + scale alignment) → cropping of key areas of cow face (focusing on core features) → target point cloud.

[0047] Figure 1A The working principle of a cattle identification method according to one embodiment of this application is demonstrated.

[0048] The cattle identification system of this application includes a data acquisition device, a computer, and a cattle feature database.

[0049] See Figure 1AImages of cattle heads are captured using image acquisition devices, such as cameras or mobile phones. To generate 3D point clouds, specific pose images of the cattle faces are captured: a frontal view, a 45-degree left-side view, and a 45-degree right-side view. For each cow, at least one image of each of these specific poses is captured.

[0050] Optionally, a YOLOv8-based cow face recognition model is run in the image acquisition device to identify the pose of cow faces in captured images in real time. When the cow face image acquired by the data acquisition device is a frontal, left 45-degree, or right 45-degree angle, automatic capture is triggered to obtain a cow face image with a specific pose required for subsequent processing. It is understood that running the YOLOv8-based cow face recognition model to achieve automatic capture is not mandatory, and users can capture cow face images with specific poses. Alternatively, video data of cow faces including various poses can be acquired through the image acquisition device, and the cow face images with specific poses required for 3D point cloud generation for 3D reconstruction can be extracted from the video data in real time, or not.

[0051] Images of cow faces captured by image acquisition equipment are provided to a computer.

[0052] For each cow, three cow face images are received: a frontal view, a 45-degree left-side view, and a 45-degree right-side view, and 3D feature vectors are generated. Specifically, a 3D model of the cow face is generated using the Dust 3R deep learning model (Dense and Unconstrained Stereo 3D Reconstruction). A 3D point cloud is generated from the 3D model. Finally, a 3D feature vector of the cow face is generated from the 3D point cloud using a 3D recognition model for classification or object detection. 3D recognition models include, for example, PointMLP, PointNet, PointNet++, PointTransformer, Uni3D, etc. The generated 3D feature vector of the cow face is, for example, a 256-dimensional stereo geometric feature vector.

[0053] The identification of cattle is based on the extracted 3D feature vectors of their faces. For example, it identifies the identity ID or identifier of cattle photographed by an image acquisition device within a cattle feature database. The similarity between the 3D feature vectors and the 3D feature vectors in the cattle database is calculated. For example, the identification result can be the top N cattle identified based on their 3D face feature vectors, or the cattle in the database with the highest similarity ranking and a similarity exceeding a confidence threshold.

[0054] The cattle feature database pre-stores 3D facial feature vectors of cattle. Before cattle identification, a cattle feature entry process is performed. This entry only needs to be done once. The cattle feature entry process is the same as... Figure 1A The demonstrated process is similar. Ideal images of cow faces are acquired using data acquisition equipment, including a frontal view, a 45-degree left-side view, and a 45-degree right-side view. A 3D feature vector is jointly generated from the frontal, left-side, and / or right-side cow face images. This 3D feature vector, along with the cow's identification information, is recorded in a cow feature database for subsequent feature vector similarity calculations.

[0055] In another embodiment of this application, addressing the inherent bottlenecks in accuracy and robustness of existing single-modal 2D image recognition or 3D point cloud recognition methods, a dual-branch recognition architecture based on a "multimodal collaborative authentication strategy" is proposed. This architecture fully leverages the insensitivity of 3D point cloud features to lighting, pose changes, and dirt interference to accurately extract the three-dimensional geometric features of a cow's face. Simultaneously, relying on the high discriminative power of 2D images in details such as patterns and hair texture, planar texture features are efficiently extracted. Through a collaborative decision-making mechanism combining 3D and 2D features, accurate identification of the cow's identity is achieved.

[0056] Figure 1B The working principle and processing flow of a cattle identification method according to another embodiment of this application are demonstrated.

[0057] same Figure 1A compared to, Figure 1B The illustrated embodiment also generates two-dimensional features of cow face images for cattle identification. One or more cow face images from the front, left 45-degree angle, and right 45-degree angle can be used to generate the two-dimensional features.

[0058] The image of the cow's face captured by the image acquisition device is provided to the computer. The computer-implemented processing flow includes both 2D processing branches and 3D processing branches.

[0059] The 2D processing branch receives a cow face image and generates a 2D feature vector from it. Generating 2D feature vectors from images is a pre-existing technique. For example, a pre-trained FaceNet model can be used to process a frontal cow face image to generate a 2D feature vector reflecting the cow's identity. Another example is using a deep convolutional neural network to extract a 256-dimensional feature vector reflecting the cow's texture and patterns.

[0060] The 3D processing branch receives three cow face images for each cow: a frontal cow face image, a cow face image at a 45-degree angle to the left, and a cow face image at a 45-degree angle to the right, and generates a 3D feature vector.

[0061] Multimodal fusion decision-making is performed on the extracted 2D and 3D feature vectors of the cow's face to identify the cow's identity. For example, it identifies the identity ID or identifier of the cow captured by the image acquisition device in the cow feature database. The similarity between the 2D and 3D feature vectors of the cow's face and the 2D and 3D feature vectors of the cow's face in the cow database is calculated to obtain the top N cow identification results based on the 2D feature vectors and the top N cow identification results based on the 3D feature vectors of the cow's face. Intersection analysis and confidence judgment are performed on the top-N results of the two modalities, and the final identification result is output according to the preset rules (successfully identified cow, identified as a known cow not belonging to the cow feature database, or prompts to re-acquire cow face images).

[0062] The cattle feature database pre-stores 2D and 3D facial feature vectors of cattle. Before cattle identification, a cattle feature input process is performed. The cattle feature input process is the same as... Figure 1B The demonstrated process is similar. Ideal images of cow faces are acquired using data acquisition equipment, including a frontal view, a 45-degree left-side view, and a 45-degree right-side view. 2D feature vectors are generated from the frontal, left-side, and / or right-side cow face images. A 3D feature vector is then generated by jointly analyzing these three images. The 2D and 3D feature vectors, along with the cow's identification information, are recorded in a cow feature database for subsequent feature vector similarity calculations.

[0063] Figure 2 A schematic diagram illustrating the process of acquiring and preprocessing a cow's face image according to an embodiment of this application is shown.

[0064] The image acquisition and preprocessing workflow is responsible for transforming the raw images acquired by the image acquisition device into high-quality, standardized input images and determining their subsequent processing path. The workflow is as follows: Figure 2 As shown, the specific steps are as follows: Step 1: Cow Face Data Acquisition. The image acquisition device is supported by a cow face recognition model based on YOLOv8, possessing automated data acquisition and classification capabilities. When acquiring cow face images, the cow face recognition model identifies the cow face posture in real time. When the acquired cow face posture is in a frontal view, a 45-degree left view, or a right view, automatic image capture is triggered, and the cow face image is simultaneously classified by viewpoint to label the posture information corresponding to the cow face image (frontal view, 45-degree left view, or right view).

[0065] Step 2: Cow face region segmentation. A cow face recognition model based on YOLOv8 is used to accurately segment the clean cow face region from the acquired original image, eliminating interference from complex backgrounds.

[0066] Optionally, steps 1 and 2 are performed by video capture, and multiple images of cow faces can be continuously accumulated before performing subsequent steps.

[0067] Step 3: Image quality screening and viewing angle determination.

[0068] After obtaining the segmented cow face image output from step 2, the images are first filtered, and those meeting quality requirements are used for subsequent processing. For example, the image is identified to determine if it contains a complete cow face; images containing a complete cow face are selected as meeting quality requirements. Next, the pose of the cow face in the image is identified. The poses of the cow face from a frontal view, a 45-degree left-side view, or a right-side view are used as the target poses. If the cow face image does not have a target pose, the current image is discarded, and the process returns to the step of obtaining the processed cow face image to obtain the next cow face image, repeating the above processing steps in step 3.

[0069] To identify whether a cow face image represents the target pose, optionally, the cow face image is aligned with preset optimal viewpoint templates: a frontal cow face image, a cow face image at a 45° left angle, and a cow face image at a 45° right angle. The confidence score of the cow face image to be processed matching each optimal viewpoint template is then output. When the confidence score of a certain viewpoint is the highest and exceeds a threshold (e.g., 0.8), the image to be processed is determined to have the target pose. By filtering the quality and viewpoint of the cow face images, high-quality cow face images are selected for subsequent 2D feature extraction and / or 3D point cloud modeling.

[0070] Step 4: Intelligent scheduling of image processing branches.

[0071] When extracting 3D features from a cow face image solely through 3D branches (e.g., Figure 1A In one embodiment, the cow face image of the target pose is sent to the 3D processing branch.

[0072] When generating 2D and 3D features of a cow's face using 2D and 3D branches respectively (e.g., Figure 1B (In the embodiment), intelligent scheduling of 2D processing branches and 3D processing branches is used to send the selected cow face images to the appropriate processing branches.

[0073] For example, for selected cow face images, if there are at least one qualified image each from three perspectives—frontal view, left 45° view, and right 45° view—then these images are sent to the 3D processing branch, and simultaneously sent to the 2D recognition branch in parallel. If the above multi-view condition is not met, but there are qualified frontal or near-frontal view images, then all images are sent to the 2D processing branch.

[0074] Specifically, see Figure 2Each time a preprocessed cow face image is acquired, if the current cow face image has a target pose (frontal view, left 45° view, or right 45° view), in addition to sending the cow face image to the 2D processing branch, it is further determined whether cow face images of all three views exist. If there is at least one cow face image of each of the three views, then the cow face images of all three views are sent to the 3D processing branch; otherwise, the process returns to the step of acquiring the preprocessed cow face image.

[0075] Optionally, when determining whether cow face images from all three perspectives exist, the number of cow face images already acquired for the current cow is also counted. If the number of cow face images from any perspective exceeds a threshold (e.g., 1), it is further determined whether the total number of cow face images already acquired for the current cow exceeds a threshold (e.g., 5). If the total number of cow face images exceeds the threshold (e.g., 5), all these cow face images are sent to the 2D processing branch; otherwise, the process returns to the step of acquiring preprocessed cow face images. Optionally, if the total number of acquired cow face images exceeds the threshold (e.g., 5), the cow face image from the frontal view or the closest to the frontal view is selected from these images and sent to the 2D processing branch.

[0076] When determining whether the current cow face image has the target pose (frontal view, left 45° view, or right 45° view), if the current cow face image does not have the target pose, the process proceeds to the step of determining whether the total number of cow face images acquired for the current cow exceeds a threshold (e.g., 5). If the total number of cow face images exceeds the threshold, all of these cow face images are sent to the 2D processing branch; otherwise, the process returns to the step of acquiring the preprocessed cow face image.

[0077] See also Figure 1B In the 3D processing branch, the Dust 3R model generates a 3D point cloud of the cow's face based on receiving three cow face images for each cow: a frontal cow face image, a cow face image at a 45-degree angle to the left, and a cow face image at a 45-degree angle to the right.

[0078] Figure 3 A schematic diagram illustrating the process of generating and processing a cow face 3D point cloud according to an embodiment of this application is shown.

[0079] In this embodiment, to generate the 3D feature vector of a cow's face, a 3D model of the cow's face is obtained from 2D images from multiple perspectives through 3D reconstruction. A high-quality, standardized 3D point cloud is then generated and fed into a 3D recognition model to extract the 3D feature vector. Generating the 3D point cloud of the cow's face is also a fundamental operation in the 3D recognition branch.

[0080] The data processing workflow generates a standardized, high-purity target 3D point cloud. This workflow includes usability verification, black edge filtering, 3D point cloud generation, point cloud alignment, and key area cropping.

[0081] Step 1: 3D mesh reconstruction.

[0082] Using the Dust3R model (https: / / arxiv.org / abs / 2312.14132, https: / / github.com / naver / dust3r), input three selected images of cow faces (each with a frontal view, a left 45° view, or a right 45° view) to obtain a 3D mesh model of the cow's face.

[0083] Step 2: 3D mesh model usability verification.

[0084] Due to factors such as occlusion at the 2D image acquisition angle, fluctuations in equipment accuracy, and uneven lighting, the 3D models obtained from 3D reconstruction may have problems such as incomplete edges, dense noise, and poor geometric consistency. If such low-quality 3D models are input into subsequent processing, it will lead to distortion of point cloud data, with effective features being masked by noise, ultimately resulting in a decrease in the accuracy of feature extraction, and the accuracy of cow face recognition will be difficult to meet the needs of practical applications.

[0085] In this embodiment, a quantitative quality assessment mechanism is established to screen out high-quality cow face 3D models, providing "pure and reliable" input data for subsequent point cloud noise filtering, standardization processing and feature extraction. This fundamentally avoids interference from low-quality models on feature extraction, ensures the authenticity and integrity of the cow face geometric features, and ultimately improves the robustness and accuracy of the overall recognition system.

[0086] See Figure 3 In this embodiment, low-quality bull-face 3D mesh models are eliminated by verifying the usability of the 3D models. Only 3D models that pass the usability verification are used for subsequent 3D point cloud generation. The confidence level of the 3D model output by, for example, the Dust3R model is used as the basis for usability verification. Optionally, the usability of the 3D model is comprehensively evaluated based on multiple indicators related to the 3D model. For example, the usability of the bull-face 3D mesh model is identified by fitting and training multiple indicators reflecting the usability of the 3D model based on binary classification methods (such as random forest, logistic regression, XGBoost, etc.).

[0087] Step 3: Black edge detection and filtering.

[0088] During 3D reconstruction, the 3D mesh model of a cow face is prone to black edge noise due to factors such as uneven lighting during image acquisition, gaps in multi-view image stitching, and vignetting in device imaging. This type of noise is essentially the mesh vertices corresponding to black pixels that have no effective geometric information. These vertices are spatially adjacent to the effective features of the cow face (such as horns, bridge of the nose, and periorbital contours), making them difficult to distinguish using traditional simple threshold filtering.

[0089] In this embodiment, for each mesh vertex on the edge of a 3D mesh model that has been screened for usability through availability verification, black edge vertices are detected and removed. Black edges are identified by color-based thresholding and distinguished from valid cow face data, achieving targeted noise removal, ensuring the purity of subsequent point cloud data, and providing a high-quality data source free from noise interference for the feature extraction stage.

[0090] Step 4: 3D mesh model to 3D point cloud conversion.

[0091] The 3D mesh model with black edges removed is converted into an initial point cloud. Optionally, a random downsampling method is used to unify the number of points in the cloud to a fixed size to improve the efficiency of subsequent processing.

[0092] Step 5: 3D Point Cloud Alignment When reconstructing a 3D model of a cow's face from multi-view 2D images, the point cloud of the cow's face in different scenes suffers from two main categories of problems due to the influence of the acquisition angle, acquisition distance (such as close-up shots of adult cows and long-distance shots of calves), and the cow's own posture (such as head turning): Attitude deviation: The spatial attitude of the cow face cloud is inconsistent (such as the head turning left and right, or tilting up and down), which leads to significant differences in the position of key features (such as horns and bridge of the nose) of the same cow in the coordinate system; Scale differences: There are significant differences in the facial size between adult cattle and calves. Even for the same animal, the overall scale of the point cloud will differ at different sampling distances.

[0093] These two types of problems directly lead to poor consistency in cow face point cloud features, causing point clouds of the same cow to be misclassified as different individuals due to differences in pose and scale, and making it difficult to distinguish point clouds of different cows due to scale inconsistencies. Ultimately, this reduces the accuracy of feature extraction and the robustness of cow face recognition. Therefore, in this embodiment, a 3D point cloud alignment process is used to eliminate pose shifts and scale differences in the 3D point clouds of cow faces, ensuring that point clouds of different scenes and different cows are consistent in spatial location and scale, providing a highly consistent data source for subsequent feature extraction.

[0094] 3D point cloud alignment processing calculates the centroid of the 3D point cloud and translates it to the origin of the coordinate system, completing initial position calibration and serving as the target point cloud for subsequent processing. Then, the Iterative Closest Point (ICP) algorithm is used to transform the pose of the target point cloud to match that of the reference point cloud. The reference point cloud represents a 3D point cloud of a cow's face with a standard pose. It is obtained, for example, by selecting typical samples from multiple cow face 3D point cloud samples.

[0095] By calculating the centroid of the 3D point cloud, the centroid is translated to the origin of the coordinate system to complete the initial position calibration, serving as the target point cloud for subsequent processing. Then, the Iterative Closest Point (ICP) algorithm is used to transform the pose of the target point cloud to match that of the reference point cloud. The reference point cloud represents a 3D point cloud of a cow's face with a standard pose. It is obtained, for example, by selecting typical samples from multiple cow face 3D point cloud samples.

[0096] Step 6, Crop key areas Because cattle are ruminants, they frequently engage in rumination, characterized by continuous chewing and resulting in cheek swaying. This introduces noise into the collected images of cattle faces and the resulting 3D point clouds of the cheeks. Regions such as the cheeks, jawline, and neck in the cattle face point clouds exhibit significant individual variations, dynamic noise due to muscle swaying, and low discriminative power. Point cloud data from these areas can obscure the geometric information of core feature regions such as horns, bridge of the nose, periorbital area, and nose, interfering with effective signals during subsequent feature extraction and reducing feature recognition accuracy. Furthermore, retaining these regions increases data redundancy and impacts processing efficiency. These regions are referred to as non-critical regions.

[0097] In the data processing of this application embodiment, non-critical areas are precisely cropped and eliminated, focusing on core features with high discriminative power, reducing interference from invalid information, and improving the purity of point cloud data and the targeting of feature extraction.

[0098] Specifically, the 3D point cloud of the cattle is cropped, removing the 3D point cloud area below the cheeks, while retaining the 3D point cloud area above the cheeks that remains relatively stable even during rumination.

[0099] Through the above data processing flow, a high-quality 3D point cloud model reflecting the 3D features of a cow's face is obtained, which is then fed into a 3D recognition model to generate a 3D feature vector.

[0100] In an optional implementation, the 3D mesh model availability verification step of the data processing flow of the 3D model in this application embodiment also combines the reliability index output by the Dust3R model itself with the newly added evaluation index to comprehensively verify the reliability of the 3D mesh model.

[0101] ① Metrics used to verify the reliability of 3D models Combining the built-in reliability metrics of the 3D reconstruction model (Dust3R) with the geometric quality metrics of the bullface model, this embodiment selects five metrics to verify the reliability of the 3D model. The selected metrics, their categories, meanings, detailed descriptions, and technical significance are listed in Table 1.

[0102] Table 1

[0103] Referring to Table 1, the selected metrics include two categories: custom metrics and Dust3R built-in metrics.

[0104] Dust3R's built-in metrics include edge1_score and edge2_score.

[0105] Dust3R is a mainstream multi-view 3D reconstruction algorithm. Its core advantage lies in achieving high-precision geometric reconstruction through image pair matching. The "best edge confidence score" (edge1_score) and the "second-best edge confidence score" (edge2_score) are the core outputs of Dust3R that quantify the quality of image pair matching. The matching accuracy of the core image pair (best edge, edge1) directly determines the accuracy of the model's geometric structure, while the second-best edge (edge2) reflects the redundancy and anti-interference capability of the reconstruction link. Choosing these two metrics allows for the direct reuse of Dust3R's reconstruction quality evaluation results, ensuring the basic quality of the model from the dimension of "reconstruction link reliability" and avoiding geometric distortion caused by image pair matching errors.

[0106] When reconstructing a 3D model, the Dust3R model outputs `edge1_score` and `edge2_score`, and simultaneously outputs `edge1_i` (best edge source image index), `edge1_j` (best edge target image index), `edge2_i` (second best edge source image index), and `edge2_j` (second best edge target image index). The core function of these parameters is to locate the image pairs corresponding to the positional confidence scores, ensuring the physical meaning of `edge1_score` is traceable. Index verification filters out samples with abnormal image pair matching (such as inconsistent `edge1_i` and `edge2_i`, or duplicate `edge1_j` and `edge2_j`). These samples are removed during the index preprocessing stage to prevent invalid indices from affecting the training accuracy of the verification model.

[0107] The built-in metrics of the Dust3R model only focus on "image pair matching quality" and do not cover the geometric quality characteristics after the model is generated. Therefore, problems such as "high noise in 3D models, poor overall reliability, and insufficient effective feature coverage" in existing technologies cannot be solved by the built-in metrics. Therefore, the embodiments of this application make up for this shortcoming by using three custom metrics to form a full-dimensional verification of "reconstruction link + geometric quality".

[0108] Custom metrics include the proportion of low-confidence points cleared, the average confidence level, and the proportion of high-confidence points.

[0109] Low-confidence point removal ratio: Addressing the issue of excessive noise in 3D models, this quantifies the removal ratio of low-confidence noise points, directly reflecting the initial purity of the model. The confidence level of each noise point is derived from the confidence level of each vertex in the Dust3D model output. During the data processing phase (before usability verification), low-confidence noise points are further removed from the 3D model based on the vertex's confidence level. Accordingly, the low-confidence point removal ratio is calculated based on the proportion of removed low-confidence noise points to the total number of vertices. For example, a criterion for selecting low-confidence points is a vertex with a confidence level < 0.3.

[0110] Average Confidence Score: Addressing the deficiency of "no evaluation standard for overall model reliability," this metric quantifies the confidence level of all vertices from a global perspective. It is obtained by calculating the average confidence score of all vertices in the 3D model reconstructed from the Dust3R model, or the average confidence score of all remaining vertices after removing low-confidence points.

[0111] High-confidence point ratio: Addressing the deficiency of "insufficient effective feature coverage," this metric focuses on the proportion of high-discriminative feature regions, ensuring sufficient high-value data support for subsequent feature extraction. The high-confidence point ratio is the percentage of high-confidence vertices out of the total number of vertices in the model. The selection criteria for high-confidence points are, for example, vertices with a confidence level ≥ 0.8.

[0112] ②Availability Fitting In this embodiment, the usability assessment objective is "whether the 3D model reconstructed by Dust3R meets the requirements for subsequent processing". Three mainstream binary classification methods (random forest, logistic regression, and XGBoost) are used for fitting and training to select the optimal method. The specific process is as follows: (2.1) Sample data preparation: Sample size: 476 sets of 3D cow face model samples (covering three breeds: dairy cows, beef cattle, and yellow cattle, including reconstruction results under different collection distances and lighting conditions); the above 5 indicators were obtained from each 3D model sample to form sample data for training.

[0113] Labeling: Manually label the sample data with usability labels (binary classification) to distinguish between qualified and unqualified 3D models. For example, "1" represents qualified models (meeting subsequent processing requirements: complete edges, low noise, and strong geometric consistency), totaling 380 groups; "0" represents unqualified models (having problems such as incomplete edges, dense noise, and geometric distortion), totaling 96 groups.

[0114] Feature preprocessing: The five indicators of the sample data are normalized (mapped to the [0,1] interval) to eliminate the difference in units; outliers (samples with |Z|>3) are removed by Z-score method to avoid outliers affecting the training accuracy of the model.

[0115] (2.2) Model training and validation: Random Forest, Logistic Regression, and XGBoost models were trained using sample data.

[0116] Three-fold cross-validation (80% training set, 20% validation set) is used to ensure the model's generalization ability; validation metrics include: accuracy, precision, recall, and F1 score.

[0117] ③ Model performance comparison and optimal selection Table 2 shows the cross-validation results of the models corresponding to the three binary classification methods.

[0118] Table 2

[0119] Of the three methods, logistic regression performed the worst: it assumes that the features are linearly related to the target variable, but there is a non-linear interaction between the five indicators in this module, which makes the model unable to capture complex feature relationships and has insufficient generalization ability.

[0120] XGBoost's performance is close to that of Random Forest, but its training time is 2.67 times that of Random Forest. It has a large number of parameters and poor real-time performance, which cannot meet the needs of large-scale real-time processing in the ranch.

[0121] Random forests offer the best overall performance: leading in accuracy and F1 score, with fast training speed, simple parameter tuning, and strong resistance to overfitting. They can achieve a balance between "high recognition accuracy" and "high real-time performance", making them perfectly suited for ranch application scenarios.

[0122] Therefore, in the preferred embodiment of this application, a random forest model is used to verify the usability of the cow face 3D model.

[0123] ④ Availability verification In this embodiment, five metrics are obtained from the 3D cow face model reconstructed from the Dust3R model (low-confidence points removed). These five metrics are then input into a trained random forest model, which outputs a usability score in the range of 0-1. A threshold of, for example, 0.75 (the optimal threshold based on ROC curve analysis, balancing precision and recall) is set. A usability score ≥ 0.75 indicates a qualified 3D cow face model, which proceeds to subsequent processing; a score < 0.75 indicates an unqualified model, which is directly discarded (not used for subsequent processing).

[0124] In the process of establishing a database of cattle features and identifying cattle, the reconstructed 3D models of cattle faces undergo the same usability verification and other data processing operations.

[0125] (3) Results and Analysis To verify the effectiveness of the five indicators selected in this application's embodiments, an effectiveness verification was also conducted. Through ablation experiments (removing one indicator at a time), it was found that the model accuracy reached 94.2% when all five indicators worked together. If any one indicator was removed, the accuracy decreased by 4%-13%. This indicates that the two-dimensional indicator system, which removed five specific indicators in this application's embodiments, can comprehensively cover the core evaluation dimensions of model quality, without redundant or invalid indicators.

[0126] The model's generalization ability was also verified. In an additional 107 sets of untrained 3D cow face model samples, the random forest model achieved a validation accuracy of 91.5% and an F1 score of 90.8%. This demonstrates that the random forest model possesses strong generalization ability in the usability verification of the cow face 3D model, and can adapt to different scenarios.

[0127] In an optional implementation, the black edge detection and filtering step in the data processing flow of the 3D model in this application embodiment also employs "divide-view processing". The processing method of "pixel color feature extraction → adaptive threshold determination → multi-view integrated filtering" achieves accurate removal of black edge noise.

[0128] Split-view processing: Since the cow face 3D data is obtained by joint reconstruction of multi-view images using DUSt3R, black edges are detected and removed from the 3D models of each viewpoint one by one to ensure that noise can be completely removed across the entire viewpoint range. The split-view 3D models with black edges removed are used to fuse the overall 3D mesh model of the cow face, rather than removing black edges after obtaining the overall 3D mesh model.

[0129] Pixel color feature extraction: For each cow face 3D model with different viewpoints, traverse all vertices within it, extract the RGB pixel color information of each vertex, and calculate the sum of the values ​​of the three color channel components (R+G+B).

[0130] Adaptive threshold determination: The screening threshold is set based on the color feature distribution pattern of the cow face model. Through color prior analysis of 100 sets of cow face model data containing black edges, the threshold normalized to the [0,1] interval is determined to be 0.03. This threshold can accurately distinguish between black edges and effective features.

[0131] Multi-view integration and denoising: Colors and 3D points below a set threshold in each view are identified as black edge noise and removed. The purified 3D models from all views are then globally integrated to finally output a clean cow face 3D mesh model with black edge interference removed.

[0132] Figure 4AThis shows a 3D model view of the cow's face before the black edges were removed. Figure 4B A 3D model view of a cow's face after removing black edges using the embodiments of this application is shown. A comparison of the two shows that the black edges of the cow's face 3D model have been effectively removed, while retaining the important features of the cow's face.

[0133] In an optional implementation, this application embodiment further improves the 3D point cloud alignment processing step in the data processing flow of 3D models. Specifically, it includes a size standardization stage and a multi-angle ICP pose alignment stage.

[0134] Figure 5 The flowchart illustrates the size standardization stage in the 3D point cloud alignment process of the data processing flow of a 3D model according to an embodiment of this application.

[0135] See Figure 5 Obtain the generated 3D point cloud of the cow's face. Calculate the 3D centroid coordinates (Cx, Cy, Cz) of the cow's face 3D point cloud using the formula: Cx = (x1 + x2 + ... + x...). n ) / n、Cy=(y1+y2+...+y n ) / n、Cz=(z1+z2+...+z n ) / n, where n is the total number of points in the point cloud.

[0136] Subtract the centroid coordinates from the coordinates of each point in the 3D point cloud (points = points - centroid) to translate the centroid of the cow face 3D point cloud to the origin (0,0,0), eliminating the translational offset of the point cloud and laying the foundation for subsequent pose alignment. Here, centroid is the centroid coordinate calculated in the previous step.

[0137] Calculate the bounding box dimensions of the 3D point cloud: Determine the maximum and minimum values ​​of the bull face 3D point cloud along the X, Y, and Z axes to obtain the dimensions in three directions: extent_x = Xmax - Xmin, extent_y = Ymax - Ymin, extent_z = Zmax - Zmin. Here, Xmax and Xmin represent the maximum and minimum values ​​of the 3D point cloud along the X-axis, Ymax and Ymin represent the maximum and minimum values ​​of the 3D point cloud along the Y-axis, and Zmax and Zmin represent the maximum and minimum values ​​of the 3D point cloud along the Z-axis.

[0138] Extract the maximum size: Take the maximum value of the size in the three directions (scale = max (extent_x, extent_y, extent_z)), which represents the overall scale of the bullface point cloud.

[0139] When size normalization is required, unit scale scaling is performed. The coordinates of each point in the cow face 3D point cloud are divided by the maximum size (points = points / scale) to unify the maximum scale of different cow face point clouds to 1 unit, eliminating scale differences caused by adult cows and calves, as well as different acquisition distances.

[0140] Normal estimation: Based on an adaptive radius search algorithm, the nearest points of each point in the bull face 3D point cloud are found, and the normal vector (nx, ny, nz) perpendicular to the surface of the bull face is calculated, providing the necessary geometric information for the subsequent point-to-plane ICP pose calibration. This yields a size-normalized bull face 3D point cloud.

[0141] Figure 6 The flowchart illustrates the multi-angle ICP pose alignment stage in the 3D point cloud alignment process of the 3D model data processing flow according to an embodiment of this application.

[0142] See Figure 6 , obtain Figure 5 The processing flow outputs a size-standardized 3D point cloud of a cow's face, which serves as the source point cloud to be aligned.

[0143] Obtain a reference point cloud with a standard pose. The reference point cloud represents a 3D point cloud of a cow's face with a standard pose. The standard pose is, for example, a frontal pose with no pose shift, obtained by, for example, selecting representative samples from multiple cow face 3D point cloud samples. The standard pose has, for example, a rotation angle of 0 relative to the X, Y, and Z axes. The reference point cloud is then... Figure 5 The displayed dimensions are standardized, with a scale consistent with the source point cloud. Its centroid is located at the origin of the coordinate axis, and the positions of key features (horns, bridge of the nose, and around the eyes) are fixed.

[0144] Generate a set of pose rotation combinations. For the possible pose deviations (left-right rotation, up-down pitch, roll, etc.) of the cow's face during 2D image acquisition, generate various pose rotation combinations. Examples of pose rotation combinations include {Rx, Ry, Rx}, where Rx represents the rotation angle around the X-axis, Ry represents the rotation angle around the Y-axis, and Rz represents the rotation angle around the Z-axis. The rotation angles around each coordinate axis have multiple values, thus all values ​​of the pose rotation combination {Rx, Ry, Rx} form the pose rotation combination set, whose elements are pose rotation combinations {Rx, Ry, Rx} with specific values.

[0145] For example, if the selected rotation angles around each coordinate axis are (0°, 90°, 180°, 270°) (4 possible values), then the set of pose rotation combinations will have 4x4x4=64 combinations of the elements around each of the three coordinate axes. The selected rotation angles cover the entire circumferential value space to adapt to the pose distribution of the cow face 3D point cloud. In practice, it was found that although the target pose (frontal, 45 degrees left and right) was filtered when acquiring the cow face image, the pose distribution in the obtained cow point 3D point cloud exists in a larger pose space. Therefore, selecting rotation angles of (0°, 90°, 180°, 270°) is beneficial for providing a suitable pose search space and better identifying the cow face pose in subsequent ICP registration. Optionally, more rotation angles can be selected for each coordinate axis to cover various pose change scenarios of the acquired cow face. Still, optionally, the rotation angle values ​​can have smaller granularity, such as 45°, 30°, 15°.

[0146] Next, the pose rotation combination set is traversed, and the pose rotation combination in the pose rotation combination set is used as a coarse-grained initialization enumeration for the cow face pose. The various poses of the source point cloud after rotation are used as the search starting point to search for the pose matching between the source point cloud and the target point cloud, thereby obtaining the pose of the source point cloud.

[0147] Specifically, see also Figure 6 Obtain a current attitude rotation combination from the attitude rotation combination set, representing a rotation angle of (Rx, Ry, Rz), generate a 3×3 rotation matrix R, and apply it to the normalized preprocessed source point cloud (after normalization). Figure 5 The processing flow outputs the source point cloud (denoted as pcd), and the rotated point cloud (denoted as pcd_rotated = R) is obtained. pcd).

[0148] Perform ICP (Iterative ClosestPoint) iterative registration on the rotated source point cloud (pcd_rotated) and the reference point cloud.

[0149] The ICP iterative registration process includes finding the nearest point pair between the rotated source point cloud (pcd_rotated) and the reference point cloud, using a point-to-plane registration mode, and utilizing the previously calculated normal vectors of each point ( Figure 5Calculate the perpendicular distance between each nearest point pair. Solve for the optimal transformation matrix T (including rotation and translation) to minimize the sum of distances between all point pairs. Update the source point cloud with the optimal transformation matrix T, denoted as p_new = R·p_old + t, where p_old is the source point cloud in the current iteration (initially pcd_rotated), p_new is the updated source point cloud for the next iteration, and R and t are the rotation and translation components of the optimal transformation matrix T, respectively. Calculate the best registration quality score (current_fitness) for the current iteration, which represents, for example, the perpendicular distance between corresponding points in the updated source point cloud and the reference point cloud. Perform the next iteration of registration with the updated source point cloud p_new and the reference point cloud. After multiple iterations of registration, obtain the optimal transformation matrix T and the best registration quality score.

[0150] Traverse the set of attitude rotation combinations. For each rotation attitude combination, perform the above ICP iterative registration process to obtain the optimal transformation matrix T and the best registration quality score. Select the historically best optimal transformation matrix T and best registration quality score from all optimal transformation matrices T and best registration quality scores in current_fitness, and also obtain the rotation matrix R of the attitude rotation combination corresponding to the historically best optimal transformation matrix T.

[0151] The historical best transformation matrix T is applied to the normalized preprocessed source point cloud (pcd) (unnormalized raw data, preserving accuracy) to obtain the pose-aligned bull's face point cloud (pcd_aligned = best_transform). pcd), where best_transform=T R, best_transform represents the bull face point cloud after pose alignment.

[0152] After the pose alignment, the bull face point cloud has a uniform pose (standard frontal pose) and scale (unit scale), and can then proceed to the subsequent key region clipping stage.

[0153] Figure 7 The before-and-after effects of 3D point cloud alignment according to an embodiment of this application are shown.

[0154] Figure 7 middle, Figure 7 (a) represents the unprocessed raw bull face point cloud, showing that its spatial orientation remains in its original state; Figure 7 (b) represents the bull face point cloud after normalization preprocessing and pose alignment; Figure 7 (c) represents the comparison results of the point cloud effect before and after processing, which is used to intuitively show the changes brought about by attitude correction.

[0155] Depend on Figure 7It can be seen that the orientation of the unprocessed raw bull face point cloud is uncertain, manifested as a certain deviation in the bull face orientation relative to the front. Moreover, the deviation is random for different bull face point clouds, resulting in uncertainty in the orientation of multiple bull face point clouds. After normalization and pose alignment processing, the overall orientation of the point cloud is more uniform (facing forward), and the correspondence between the main direction and the reference coordinate system is more stable. Figure 7 (c) Further explanation: the embodiments of this application can effectively reduce the spatial deviation caused by the difference in the original posture between different samples.

[0156] By standardizing the size and aligning the pose of 3D point clouds in stages, the position, scale and density of the point clouds are first unified to reduce the registration complexity. Then, the traversal of the pose rotation combination set is used to avoid the ICP iterative registration from getting stuck in the optimal solution. At the same time, the final historical best transformation matrix is ​​directly applied to the normalized preprocessed source point cloud (pcd) to preserve the geometric detail accuracy. The registration quality is quantified by the fitness score to achieve traceability of the effect. The problem of pose offset, scale difference and density unevenness of bullface point clouds is comprehensively solved from five dimensions: reducing complexity, improving robustness, maintaining accuracy, strong adaptability and quantifiability. Finally, a highly consistent and high-precision aligned point cloud is output.

[0157] Figure 8 A flowchart illustrating the key region trimming stage of the data processing flow for a 3D model according to an embodiment of this application is shown.

[0158] See Figure 8 , obtain Figure 6 The processing flow outputs a pose-aligned 3D point cloud of a bull's face. A global geometric reference plane for the bull's face is then fitted from this 3D point cloud. The equation of this global geometric reference plane is represented by n·x + d = 0, where n (n = (nx, ny, nz)) represents the unit normal vector of the plane equation, and d is a plane constant parameter. Fitting the global geometric reference plane for the bull's face is essentially the process of solving for the parameters n and d from the 3D point cloud.

[0159] To obtain the overall geometric reference plane for the cow's face, the centroid of the 3D point cloud was calculated and then decentered (see also...). Figure 5 , and according to Figure 5 In the embodiments, the calculation of the centroid coordinates and the translation of the 3D point cloud centroid to the origin are the same operations to obtain a standardized 3D point cloud of the bull's face. Then, principal component analysis (PCA) least squares plane fitting is used to fit the overall geometric reference plane of the bull's face.

[0160] Specifically, a 3×3 covariance matrix is ​​constructed based on the decentralized point cloud data. The covariance matrix represents the dispersion of the 3D point cloud along the X, Y, and Z coordinate axes. Its diagonal elements represent the variance of the points in the 3D point cloud in the directions represented by each of the three coordinate axes, and the off-diagonal elements represent the covariance of the points in the 3D point cloud between two coordinate axes.

[0161] The covariance matrix is ​​decomposed into eigenvalues, with its three eigenvalues ​​corresponding to the distribution variance of the point cloud in three orthogonal directions. The eigenvector corresponding to the smallest eigenvalue is taken as the normal vector in the "flattest" direction of the 3D point cloud distribution. This vector is normalized to obtain the unit normal vector n(nx, ny, nz) of the plane. The plane constant parameter d is then calculated, for example, by using the constraint condition of the overall geometric reference plane of the bull's face passing through the centroid of the 3D point cloud, and solving for the parameter d. Finally, the plane equation n·x + d = 0 is obtained, and this plane accurately represents the overall surface geometry of the bull's face.

[0162] The eigenvector corresponding to the smallest eigenvalue represents the normal vector in the "flattest" direction of the 3D point cloud distribution. Based on the inherent shape characteristics of a cow's face, the plane corresponding to the normal vector in the "flattest" direction of the cow's face 3D point cloud distribution is roughly defined by the tip of the nose and the base of the ears. Multiple facial features, such as the eyes, nose, and horns, are located on the same side of this plane, while parts prone to noise, such as the cheeks and mouth, are located on the opposite side. Therefore, this plane allows for the differentiation between noisy portions and features that represent the cow's facial characteristics within the 3D point cloud of the cow's face.

[0163] Optionally, since normalization and pose alignment have already been performed during the data processing for generating the 3D point cloud, the operation of calculating the centroid of the 3D point cloud and performing decentralization is no longer required when constructing the overall geometric reference plane of the cow's face.

[0164] Optionally, the normal vector *n* of the plane equation is also standardized so that for any bullface 3D point cloud, the normal vector *n* of the fitted plane equation uniformly points to the same side of the overall geometric reference plane of the bullface. Specifically, it is determined whether the normal vector *n* of the plane equation points in a specified direction. The specified direction is, for example, above the overall geometric reference plane of the bullface (e.g., the z-axis component is positive). When the normal vector *n* points above the plane, no adjustment to the plane equation is required. When the normal vector *n* points below the plane, the plane equation is flipped, for example, by flipping the plane equation by *n = -n* and *d = -d*. For example, whether it points in a specified direction is determined by the Z-axis component value of the normal vector *n*. A positive Z-axis component value of *n* indicates that *n* points above the plane, while a negative Z-axis component value of *n* indicates that *n* points below the plane.

[0165] Next, for each point x in the 3D point cloud of the cow face, calculate its distance to the overall geometric reference plane of the cow face represented by the plane equation n·x+d=0 (denoted as s=n·x+d). When s>0, it means that point x is above the plane, and the point is retained in the 3D point cloud of the cow face. When s<0, it means that point x is below the plane, and the point is deleted from the 3D point cloud of the cow face. This is to achieve cropping of the 3D point cloud of the cow face, remove the influence of cheek movement on 3D feature extraction, and enhance the stability of the extracted 3D feature vectors. It can be understood that for a point s=0, it is located on the face plane, and it can be deleted from the 3D point cloud or retained in the 3D point cloud.

[0166] Optionally, a mask representing the front and side spaces of the overall geometric reference plane of the cow's face is created (when s>0, mask is True(1); when s≤0, mask is False(0)), where points s>0 correspond to key feature regions on the front side of the cow's face (horns, bridge of the nose, periorbital area, and nose), and points s≤0 correspond to non-key noise regions on the negative side of the cow's face. This mask is applied to filter the 3D point cloud, retaining points that meet the mask conditions and removing non-key regions. For example, multiplying the points in the 3D point cloud by the corresponding mask yields the result, where the point values ​​in the front and side spaces of the overall geometric reference plane of the cow's face are retained, while the points in the negative space of the overall geometric reference plane of the cow's face are set to 0.

[0167] And output the cropped target 3D point cloud with focused core features.

[0168] Figure 9 The images show a comparison view before and after cropping the 3D point cloud of a cow's face according to an embodiment of this application.

[0169] Figure 9 In the image, (a) is a view of the 3D point cloud of the cow's face before cropping, and (b) is a view of the 3D point cloud of the cow's face after cropping. After cropping, the cow's cheek area in the middle right region of (a) has been removed in (b).

[0170] In this embodiment, the cropped 3D point cloud of the cow's face is fed into a 3D recognition model to extract 3D feature vectors. Optionally, for the 3D recognition model that outputs classification results, the output of the last hidden layer before its classification output head is used as the extracted 3D feature vector. Common 3D recognition models include, for example, PointNet, PointNet++, Point Transformer, Uni3D, etc.

[0171] In an alternative embodiment of this application, the PointNet++ network has also been specifically improved to better capture the 3D features of a cow's face.

[0172] PointNet++ (https: / / arxiv.org / pdf / 1706.02413) is a landmark achievement in the field of 3D vision. Through its innovative design of hierarchical sampling and local feature aggregation, it can accurately capture the local details and global structure of 3D point clouds. Existing PointNet++ models are mainly applied to point cloud classification and segmentation tasks, and are not well-suited for cattle face recognition scenarios—its training data consists mostly of point clouds with large class differences, uniform and regular distribution, and no noise. However, the point clouds in cattle identification tasks are characterized by disorder and high similarity between samples, showing a significant difference from the distribution of PointNet++ training data. The original design of the PointNet++ model is primarily for object classification or part segmentation. When directly applied to the identification of similar individuals of the same species requiring extremely high discriminative power, the learned features are not sensitive enough to fine-grained discrimination (such as nose prints, spots, subtle hair direction, and other texture information that has high discriminative power for individual cattle).

[0173] To extract highly discriminative features from a cow face 3D point cloud, an improvement upon the PointNet++ model is proposed. PointNet++ employs a hierarchical point cloud feature extraction framework, including multiple cascaded SA layers and a feature vector output head. While PointNet++ aims for class discrimination, this embodiment replaces the classification output head with a feature vector output head, making the entire network a three-dimensional identity feature mapping function used to map the input 3D point cloud into identity feature vectors of a uniform scale.

[0174] The output of the last layer SA in PointNet++ is provided to the feature vector output head after passing through the feature extraction network. Figure 10 A schematic diagram of the network structure of the feature vector output head according to an embodiment of this application is shown.

[0175] The core structure of the feature vector output head combines batch normalization (BN) and ReLU activation functions while compressing the feature dimension. First, a feature compression layer (fully connected layer) compresses the 1024-dimensional features output from the backbone network (the fourth SA) to 512 dimensions. The 512-dimensional output features are then fed into a subsequent combination of BN, ReLU, and Dropout (random deactivation layer) for a complete round of normalization and regularization. BN normalizes the feature distribution, effectively stabilizing the training process and reducing parameter sensitivity. ReLU provides non-linear transformation capabilities, enhancing the model's ability to represent complex features. After ReLU, Dropout regularization is introduced, randomly suppressing some neurons during training to prevent the network from over-relying on specific features, thereby suppressing overfitting and improving the generalization performance of features on unknown cow face data. Then, it goes through a combination of neural network layers including linear layers and BN, ReLU and Dropout. The linear layer compresses the 512-dimensional features output by the previous neural network layer group into 256 dimensions. The 256-dimensional features output then enter the subsequent combination of BN, ReLU and Dropout to complete another round of standardization and regularization.

[0176] Finally, the feature vectors are subjected to L2 normalization through a linear layer and a normalization layer, so that the output feature vectors meet the requirements of Euclidean distance measurement and ensure the accuracy of similarity calculation.

[0177] This feature vector output head structure can progressively distill the complex geometric information of the captured cow face into 256 ready-to-use, highly discriminative features. This design not only ensures the efficiency and stability of model training, but also makes the output features suitable for downstream tasks requiring high-precision similarity comparisons, such as cow identity verification.

[0178] Training strategy This application's embodiments use triplet loss instead of traditional cross-entropy loss to complete end-to-end training of the model. As the core loss function of the metric learning model, the training objective of triplet loss is not to directly learn the sample classification ability, but to construct a highly discriminative feature embedding space—in this space, the feature vectors of similar samples are close to each other, while the feature vectors of dissimilar samples are far apart. This characteristic makes it more suitable for cow face authentication tasks.

[0179] The training mechanism of triplet loss uses "sample triplet" as the core unit. Each triplet contains three types of samples: anchor sample (a), positive sample (p), and negative sample (n). All three are mapped to the feature space through a parameter-sharing neural network. Among them, anchor sample (a) serves as a reference benchmark; positive sample (p) and anchor sample (a) are point cloud data obtained from cow face images corresponding to the same cow; negative sample (n) comes from point cloud data of different cows, respectively.

[0180] The core optimization objective of the training process is: for each anchor sample (a), through network parameter adjustment, make the feature distance between the anchor sample (a) and the positive sample (p) smaller than the feature distance between the anchor sample (a) and the negative sample (n), and the difference between the two needs to meet the preset margin requirement. The expression of its loss function is as follows: L = max(d(a,p) - d(a,n) + margin, 0) In this formula, a, p, and n respectively represent the feature vectors of the anchor sample, the positive sample, and the negative sample in the feature space; d() represents the Euclidean distance calculation function, and margin represents the preset boundary value. By directly optimizing the distance distribution of samples in the feature space, this loss function enhances the category discrimination of feature vectors, and is more in line with the requirements of the identity verification task for the discriminability of sample features than traditional loss functions.

[0181] Triplet sample sampling strategy and training process To optimize the feature extraction capability of the model, the embodiments of the present application propose a progressive adaptive hard sample mining strategy. This strategy simulates the human cognitive law of "from easy to difficult", dynamically divides the model training process into three stages, and adaptively adjusts the learning difficulty of samples, so as to solve the problems of poor stability of traditional hard sample mining in the early stage of training and easy to fall into local optimum, and enable the model to perform feature learning under appropriately adapted challenge intensity at all times.

[0182] The training samples are divided into easy samples, semi-hard samples and hard samples. Each sample includes a triplet {anchor sample, positive sample, negative sample}, where a, p, and n respectively represent the feature vectors of the anchor sample, the positive sample, and the negative sample in the feature space. After the model outputs the feature vectors according to the samples, the distance of the feature vectors of all samples in this batch is calculated in units of one batch.

[0183] The screening basis for easy samples is that the distance between the positive sample and the anchor is smaller than the distance between the negative sample and the anchor, and this distance difference is greater than the threshold margin. The screening basis for semi-hard samples is: the distance between the negative sample and the anchor is greater than the distance between the positive sample and the anchor, but this distance difference is less than margin, which can be expressed as 0<d(a, n) - d(a, p)<margin. In other words, for semi-hard samples, their positive samples and negative samples can be distinguished, but the degree of distinction is not large. The screening basis for hard samples is that the distance between the negative sample and the anchor is smaller than the distance between the positive sample and the anchor, which can be expressed as d(a, n)<d(a, p). The ideal situation should be that the distance between the positive sample and the anchor is small, and the distance between the negative sample and the anchor is large.

[0184] Training does not use simple samples. In the early stages of training, a semi-hard sample mining mode is used to calculate the loss function and update the model parameters with semi-hard samples. This moderate learning intensity helps the PointNet++ model build the basic structure of the feature space and ensures the stability of the initial training. In the middle stage, a random mixed mining strategy is used, which mixes semi-hard and hard samples during training to balance the learning signals of samples of different difficulties and avoid learning bias caused by the model favoring samples of a single difficulty. In the later stage, full hard samples are used for training, focusing on optimizing the decision boundary of the feature space and strengthening the model's ability to distinguish fine-grained differences.

[0185] The trained model is used to receive 3D point clouds of cow faces obtained from acquired cow face images and output the corresponding 3D feature vectors.

[0186] According to an embodiment of this application, cattle identification is performed based on the calculated 3D feature vector representing a cow's face.

[0187] Optionally, also see Figure 1A The similarity (e.g., Euclidean distance) between the calculated 3D feature vector and the 3D feature vectors of known cattle in the cattle feature database is calculated. The highest-ranked or top N candidate results in the cattle feature database are returned as the cattle identification results.

[0188] Optionally, a confidence threshold is also used, based on the similarity between the candidate results and the 3D feature vector of the cattle to be identified. Only one or more candidate results with a similarity exceeding the confidence threshold are recognized as cattle identification results. If the similarity of any candidate result does not exceed the confidence threshold, an "identification failure" result is returned.

[0189] To evaluate the technical effectiveness of the data processing method provided in this application for obtaining a 3D point cloud for extracting 3D feature vectors, the data processing method of this application (see also) was compared. Figure 3 , Figure 5 , Figure 6 and Figure 8Table 3 shows the changes in the accuracy of cattle identification results before and after the extraction of 3D feature vectors based on the PointMLP model (https: / / arxiv.org / abs / 2202.07123) and the PointNet++ model. In Table 3, Top1 accuracy refers to the comparison of the input sample with the feature vectors of all cattle in the cattle feature database, selecting the most similar one as the identification result, i.e., Top1. Similarly, Top5 accuracy is the selection of the top 5 results in terms of similarity when comparing with the feature vectors of the cattle feature database. The data in Table 3 show that the data processing method of this application embodiment has achieved optimization of the accuracy of cattle identification, and a significant improvement in accuracy has been achieved regardless of whether the 3D feature vectors are extracted based on the PointMLP model or the PointNet++ model. Among them, the Top1 identification accuracy obtained based on the PointNet++ model increased from 69.6% to 73.6%, an improvement of about 4%.

[0190] Table 3

[0191] The core of this application's embodiments lies in improving the quality of the generated 3D point cloud through a series of data processing steps in the over-reconstruction of the cow face 3D model, thereby achieving better cow identification results based on 3D feature vectors. Its key technical points include: Dual-dimensional model availability verification mechanism: An innovative dual-dimensional evaluation system of "Dust3R built-in indicators + custom geometric quality indicators" is designed, and combined with the random forest algorithm to achieve high-quality model screening, ensuring data reliability from the source; A precise noise reduction algorithm based on color prior: For the color features of the cow face model, an adaptive threshold is set, and black edge noise is filtered through multi-view traversal to achieve precise distinction between noise and effective features. A phased, multi-angle ICP alignment scheme: First, standardize the size to unify the basic scale attributes of the point cloud, and then try various combinations of rotation angles to avoid ICP getting trapped in local optima and ensure the uniformity of point cloud pose and scale.

[0192] Geometrically driven key region clipping: Based on least squares plane fitting and signed distance calculation, combined with the anatomical characteristics of the cow face, high discriminative feature regions are extracted in a targeted manner to reduce interference from non-critical regions.

[0193] The beneficial effects of the data processing embodiments of this application Significantly improved data quality: Low-quality models were removed through availability verification (removal rate of approximately 20%), black edge noise filtering accuracy reached over 98%, and the proportion of effective features after key area cropping increased from 62.3% to 89.7%, providing a high-purity data source for feature extraction.

[0194] Processing efficiency is greatly optimized: After point cloud simplification, the amount of data is reduced by 70%-80%. Combined with efficient algorithms such as random forest and ICP, the overall preprocessing time is ≤0.5 seconds, which meets the needs of large-scale real-time processing in ranches.

[0195] Recognition performance has been continuously optimized: adapting to cow face data of different breeds and collection scenarios has improved the accuracy of subsequent feature extraction by more than 4% (the accuracy of PointNet++ network Top1 recognition has increased from 69.6% to 73.6%).

[0196] According to embodiments of this application, in order to identify the identity of cattle, a cattle feature database is also established to store the 3D feature vectors of the 3D point cloud of the cattle faces of cattle with known identities. For example, in an insurance application scenario, when insuring cattle, qualified images of the cattle's face from three perspectives—front, left 45°, and right 45°—are taken (at least one image for each perspective).

[0197] Based on the embodiments of this application (see also Figure 1), images from three perspectives are used to illustrate the concepts presented in this application. Figure 8 The processing flow is as follows: image preprocessing, Dust3R model to build 3D mesh model, black edge detection and filtering, point cloud standardization and key region cropping, and then fed into the trained improved Point Net++ model to obtain 3D feature vectors.

[0198] All qualified images of the insured cattle from three perspectives are fed into the FaceNet model to obtain 2D feature vectors.

[0199] The obtained 3D feature vectors and 2D feature vectors are combined with the identity ID of the insured cattle to obtain the cattle feature database.

[0200] When using the embodiments of this application to identify whether a target cattle is insured, qualified images of the target cattle's face are collected from three perspectives: frontal, left 45°, and right 45°, based on the embodiments of this application (see also the embodiments of this application). Figures 1A-8 The processing flow of the identification results is as follows: output the ID of the identified insured cattle, indicate insufficient confidence, or identify failure (non-insured cattle)

[0201] According to embodiments of this application, it is executed on an information processing device such as a computer or server. Figure 11 This is a block diagram of an information processing device that implements an embodiment of the present invention. Figure 11The information processing device shown is a computer. The computer is merely one example of a suitable computing environment and is not intended to imply any limitation on the use or scope of the invention. Figure 11 The information processing equipment shown should not be construed as having any dependency or requirement on any of the components or combinations of components shown.

[0202] Information processing devices include memory, one or more processors, one or more displays, I / O components, and a power supply, directly or indirectly coupled to a bus. A bus can represent one or more types of buses (such as an address bus, a data bus, or a combination thereof). However, in practice, the definition of each component is not necessarily as straightforward. Figure 11 In this way, for example, a display device can be considered as an I / O component. Furthermore, the processor may have memory. The inventors of this invention recognize that this is precisely the nature of the art and reiterate... Figure 11 The illustrations are merely for illustrating exemplary computer systems that can be used in conjunction with one or more embodiments of the present invention.

[0203] Information processing devices typically include various types of memory. By way of example and not limitation, memory may include: random access memory (RAM), read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), flash memory, CD-ROM, DVD or other optical or holographic media, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices. Computer storage media may be non-volatile.

[0204] Information processing devices include one or more processors that read data from various entities such as buses, memory, or I / O groups. One or more displays present data indications to a user or other device. Exemplary displays include display devices, speakers, printing components, vibrating components, flat panel displays, projectors, head-mounted displays, etc. A display can also be an I / O port for coupling display devices, speakers, printing components, vibrating components, flat panel displays, projectors, head-mounted displays, etc. Illustrative I / O components include cameras, microphones, joysticks, game controllers, dish satellite transceiver antennas, scanners, printers, wireless devices, etc.

[0205] Although the examples referred to in the present application are described for illustrative purposes only and not for limiting the scope of the application, changes, additions and / or deletions to the implementation may be made without departing from the scope of the application.

[0206] Those skilled in the art who benefit from the teachings presented in the above description and associated drawings will recognize many modifications and other embodiments of the present application described herein. Therefore, it should be understood that the present application is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terminology is used herein, it is used only in its general and descriptive sense and not for limiting purposes.

Claims

1. A method for reconstructing a 3D model of a cow's face and generating point cloud data based on multi-angle 2D images for cattle identification, the method comprising: Acquire frontal image, left 45-degree angle image, and right 45-degree angle image of the cow's face to be identified; The frontal, left 45-degree, and right 45-degree images of the cow's face to be identified are fed into the Dust3R model to generate a 3D mesh model of the cow's face to be identified. Predicting the availability of the 3D mesh model; wherein, predicting the availability of the 3D mesh model includes: The metrics used to predict the usability of the 3D mesh model are obtained, including the best and second-best edge confidence scores output by the Dust 3R model, and the proportion of low-confidence points cleaned, the average confidence score, and the proportion of high-confidence points when the Dust 3R model generates the 3D mesh model. The proportion of low-confidence points cleaned refers to the ratio of the number of vertices with a confidence score below a first threshold to the total number of vertices in the output of the 3D mesh model. The average confidence score refers to the average confidence score of the vertices in the output of the 3D mesh model. The proportion of high-confidence points refers to the ratio of the number of vertices with a confidence score above a second threshold to the total number of vertices in the output of the 3D mesh model, where the second threshold is higher than the first threshold. The availability of the 3D mesh model is identified by using a availability verification model based on the random forest algorithm and the indicators used to predict the availability of the 3D mesh model. For the 3D mesh model with availability greater than a specified threshold, detect and remove black edges from the 3D mesh model; and The 3D mesh model with black edges removed was converted into a 3D point cloud; The 3D point cloud is scaled to a uniform unit scale, and the pose of the 3D point cloud is transformed to be consistent with that of the reference point cloud to obtain an aligned 3D point cloud, wherein the reference point cloud represents a 3D point cloud of a cow's face with a standard pose. The aligned 3D point cloud is cropped to remove the 3D point cloud area corresponding to the cow's cheek, while retaining the 3D point cloud area above the cheek. Generate 3D feature vectors from the cropped 3D point cloud; Based on the 3D feature vector, candidate results most similar to the 3D feature vector are obtained from the cattle feature database by sorting them by similarity; wherein, the cattle feature database records the cattle identity and its 3D feature vector in association.

2. The data processing method according to claim 1, wherein: The training samples for the 3D model of the availability verification model based on the random forest algorithm include the best edge confidence score, the second best edge confidence score, the proportion of low confidence points cleaned, the average confidence score and the proportion of high confidence points obtained when the 3D mesh model is generated by the Dust3R model based on cow face images covering multiple breeds of cattle, multiple cow face image acquisition distances, and cow face image acquisition lighting conditions, as well as label data to distinguish whether the 3D mesh model is qualified; The availability verification model based on the random forest algorithm is trained using the training samples of the 3D model.

3. The data processing method according to claim 2, wherein: Detecting and removing black edges from the 3D mesh model includes: detecting and removing black edges from each viewpoint of the 3D model output by the Dust3R model; wherein, for each viewpoint of the 3D model, all its vertices are traversed, and the sum of the color component values ​​of the R, G, and B color channels of each vertex is calculated; when the sum of the color component values ​​of the R, G, and B channels of a vertex is lower than the judgment threshold of 0.03, the vertex is determined to be a black edge, and the vertex is removed from the 3D model of the current viewpoint; The 3D mesh model is obtained by fusing 3D models from various perspectives with black edges removed.

4. The data processing method according to claim 3, wherein: Scaling the 3D point cloud to a uniform unit scale includes: Calculate the centroid of the 3D point cloud; Subtract the centroid coordinates from the coordinates of each point in the 3D point cloud so that the centroid of the 3D point cloud coincides with the origin of the coordinate system; Calculate the dimensions of the bounding box of the 3D point cloud in the three coordinate axis directions; The coordinates of each point in the 3D point cloud are scaled using the maximum value of the size of the bounding box of the 3D point cloud in the three coordinate axis directions.

5. The data processing method according to claim 4, wherein: Transforming the pose of the 3D point cloud to match that of the reference point cloud includes: Multiple pose rotation combinations are obtained, which represent the possible poses of the cow's face during the acquisition of the cow face image; wherein, the pose rotation combination includes rotation angles around each of the three coordinate axes; For each posture rotation combination, the 3D point cloud is rotated according to its rotation angle to obtain the rotated 3D point cloud; The rotating 3D point cloud is then registered with the reference point cloud using ICP iterative registration to obtain the optimal transformation matrix T and the best registration quality score of the rotating 3D point cloud relative to the reference point cloud. The best registration quality score represents the difference between the rotating 3D point cloud and the reference point cloud after being transformed with the optimal transformation matrix T. For each posture rotation combination, the rotated 3D point cloud is registered with the reference point cloud by performing ICP iterative registration to obtain the historical best transformation matrix and the historical best registration quality score. The 3D point cloud is transformed using the historically best transformation matrix.

6. The data processing method according to claim 5, wherein, The aligned 3D point cloud is cropped to remove the 3D point cloud region corresponding to the cow's cheek, including: A bull's face global geometric reference plane is fitted from the aligned 3D point cloud, where the bull's face global geometric reference plane represents the plane with the flattest direction of the aligned 3D point cloud distribution, and its plane equation is represented by n·x + d = 0; where n represents the unit normal vector of the plane equation, and d is a plane constant parameter; where, Based on the aligned 3D point cloud, a 3×3 covariance matrix is ​​constructed; the eigenvalues ​​of the covariance matrix are decomposed; the eigenvector corresponding to the smallest eigenvalue is taken, and the vector is normalized to obtain the unit normal vector n of the plane; the parameter d is solved by the constraint condition of the centroid of the aligned 3D point cloud through the overall geometric reference plane of the bull's face; finally, the plane equation n·x + d = 0 is obtained. Points located below the overall geometric reference plane of the bull's face are removed from the aligned 3D point cloud, while points located above the overall geometric reference plane of the bull's face are retained.

7. The data processing method according to claim 6, wherein, If the normal vector of the plane equation represented by n points in a specified direction, the plane equation is flipped by n = -n and d = -d; where the specified direction is the direction in which its Z-axis component is negative, and the Z-axis is the coordinate axis perpendicular to the ground and pointing upwards.

8. The data processing method according to claim 7, wherein, The signed distance from point x in the 3D point cloud to the face plane is calculated according to s = n·x + d, where x is the coordinate vector of the point in the 3D point cloud; Deleting points from the 3D point cloud that are located below the face plane relative to the face plane includes: When s < 0, the corresponding point x is deleted from the 3D point cloud; When s>0, the corresponding point x is retained in the 3D point cloud.

9. The data processing method according to claims 1-8, wherein: For the 3D mesh model whose availability is not greater than a specified threshold, it is not used to identify the identity of the cattle to be identified, nor is it used to generate 3D feature vectors.

10. An information processing device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1-9.

Images