A feature extraction network model, a training method and device for livestock point cloud data recognition

Abstract

Provided are a feature extraction network model, a training method and an apparatus for livestock point cloud data recognition, comprising: obtaining a 3D point cloud of a face of a to-be-identified cow; using an H-PointNet++ model to receive the 3D point cloud to generate a 3D feature vector, wherein the H-PointNet++ model comprises a plurality of SA layers and a feature vector output head connected in sequence; wherein each SA layer comprises a sampling layer, a grouping layer and a PointNet layer; a first SA layer of the plurality of SA layers is used to uniformly sample 1024 center points from the 3D point cloud according to farthest point sampling, a local neighborhood sampling radius of each center point is less than a sampling radius of other SA layers except the first SA layer, and a local neighborhood sampling point number of each center point is greater than a local neighborhood sampling point number of other SA layers except the first SA layer; the feature vector output head receives an output of a last SA layer of the plurality of SA layers and generates a 3D feature vector representing the 3D point cloud; and a candidate result most similar to the 3D feature vector is obtained from a cow feature database according to a similarity order of the 3D feature vector.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud processing and identity feature recognition technology, specifically to a 3D point cloud feature extraction network and its training method for open set identity recognition tasks, which is particularly suitable for identity recognition scenarios of highly similar individuals such as livestock. Background Technology

[0002] Identification applications aim to determine the uniqueness of an individual target. This differs from category identification, which focuses on the type of target, by identifying which of multiple objects of the same type the target belongs to. Currently, physical identification methods such as ear tags and radio frequency identification (RFID) are commonly used in livestock farms. However, these methods suffer from problems such as fragility, loss, the need for close contact, and the risk of animal stress. Therefore, the industry is dedicated to researching non-contact, stress-free biometric identification technologies to achieve more intelligent and traceable livestock management. These technologies have significant application value in areas such as dairy cow performance testing, precision feeding of beef cattle, disease prevention and control, breed registration, and agricultural insurance.

[0003] In the field of biometric-based visual recognition, existing research is mainly divided into two categories: identity recognition based on two-dimensional images and identity recognition based on three-dimensional point clouds.

[0004] 1. A two-dimensional image-based cow face recognition scheme for identifying cattle. This scheme draws on mature facial recognition technology, directly extracting and comparing features from acquired 2D images of cow faces. The implementation process typically involves the following: First, preprocessing the input cow face image, including grayscale conversion, normalization, keypoint detection, and alignment; then, using a deep convolutional neural network (such as FaceNet or VGG) as a feature extractor to map the input image into a high-dimensional feature vector; finally, by calculating the cosine or Euclidean distance between the feature vector of the image to be identified and the pre-stored feature vectors in the database, identity recognition is achieved based on similarity ranking.

[0005] Such methods are widely used in academic research. For example, in patent literature (such as Chinese patent application CN120148068A, "A method for recognizing cow faces based on multi-scale feature interaction and its recognition system"), convolutional neural networks are used to identify the identity of cows.

[0006] 2. A cattle identification scheme based on 3D point cloud for identifying cattle heads. This scheme aims to overcome the vulnerability of 2D images to environmental interference by utilizing the 3D geometric structure information of a cow's head for identification. A typical implementation process involves: reconstructing a 3D point cloud model of the cow's head using a 3D scanning device or from multiple 2D images; subsequently, using a deep learning network specifically designed for processing point cloud data to learn features and classify the point cloud; the network output is either the corresponding cow's identification ID or a point cloud feature vector used for comparison. Commonly used 3D point cloud classification networks include PointNet and PointNet++, which employ hierarchical sampling and grouping strategies, extracting local and global geometric features step-by-step through a multilayer perceptron, and finally outputting class probabilities through fully connected layers. These networks were initially designed for closed-set classification tasks and have achieved excellent performance on standard datasets such as ModelNet and ShapeNet. In recent years, they have been increasingly applied to individual identification research in the agricultural field.

[0007] This type of method falls under the application of 3D target recognition technology in computer vision to the agricultural field. Patent literature (such as Chinese patent application CN119992588A, "A method for recognizing cow faces based on dense point cloud 3D reconstruction") discloses a technical solution that achieves individual recognition by acquiring dense 3D point clouds of cow faces and using deep learning models for feature matching. Furthermore, network structures such as PointNet++ have been described in detail in relevant academic papers as general point cloud feature extractors.

[0008] Compared to identity recognition based on two-dimensional images, research on three-dimensional point clouds in the field of identity recognition is significantly insufficient. Existing deep learning methods for three-dimensional point clouds (such as PointNet, PointNet++, etc.) mainly serve closed-set tasks such as object classification or part segmentation, and their network structure and training objectives assume that the set of categories is known during the training phase.

[0009] However, in real-world production scenarios such as cattle identification, the task objective is not to distinguish between different categories, but rather to identify "which one" among a large number of similar cattle individuals. Furthermore, unknown individuals continuously appear, making it a typical open-set identification problem. At the same time, the three-dimensional geometric structures of different cattle are highly similar, and individual differences are often concentrated in local areas, making it difficult to obtain sufficient discriminative power by directly using existing point cloud classification networks.

[0010] Therefore, there is an urgent need for a feature extraction network and training method that can be systematically modified for the identity recognition task of 3D point cloud open set. Summary of the Invention

[0011] The purpose of this invention is to address the problem that existing point cloud networks such as PointNet++ cannot adapt to 3D open set identity recognition. To address this issue, an improved 3D point cloud feature extraction network model, H-PointNet++, is proposed. Through structural adjustments, feature output format modifications, and training strategy design, the network can stably output individual identity feature vectors with high discriminative power, thereby achieving identity recognition based on a cow face 3D point cloud model in a real production environment.

[0012] According to a first aspect of this application, a method for cattle identification based on 3D point clouds and a feature extraction network model H-PointNet++ is provided. The method includes: acquiring a 3D point cloud of the face of a cattle to be identified; using an H-PointNet++ model to receive the 3D point cloud and generate a 3D feature vector, wherein the H-PointNet++ model includes multiple SA layers and a feature vector output head connected in sequence; wherein each SA layer includes a sampling layer, a grouping layer, and a PointNet layer; the sampling layer is used to select a subset of points from the input points as center points of a local region; the grouping layer samples neighborhood points in the local neighborhood around each center point to construct several subsets of local region points; the PointNet layer generates a feature vector representing the local neighborhood of each center point based on the subset of local neighborhood points; the first SA layer of the multiple SA layers is used to receive the 3D point cloud and generate a 3D feature vector from the 3D point cloud. In the 3D point cloud, 1024 center points are uniformly sampled according to the farthest point. The local neighborhood sampling radius of each center point is smaller than the sampling radius of the other SA layers (excluding the first SA layer). The number of local neighborhood sampling points of each center point is larger than the number of local neighborhood sampling points of the other SA layers (excluding the first SA layer). The feature vector output head receives the output of the last SA layer of the multiple SA layers and generates a 3D feature vector representing the 3D point cloud. The feature vector output head includes a feature compression layer, a feature distribution normalization layer, a nonlinear mapping layer, a normalization layer, and a vector normalization layer connected in sequence. The vector normalization layer outputs the 3D feature vector. Based on the 3D feature vector, the candidate results most similar to the 3D feature vector are obtained from the cattle feature database by sorting by similarity. The cattle feature database records the cattle identity and its 3D feature vector in association.

[0013] According to the cattle identification method of the first aspect of this application, wherein: the other SA layers of the plurality of SA layers, except for the first SA layer, are multiple SA layers of the PointNet++ model; the local neighborhood sampling radius of the first SA layer of the plurality of SA layers is smaller than the local neighborhood sampling radius of the first SA layer of the PointNet++ model, and the number of local neighborhood sampling points of each center point is greater than the number of local neighborhood sampling points of the first SA layer of the PointNet++ model.

[0014] According to the cattle identification method of the first aspect of this application, wherein: the PointNet layer of the first SA layer of the plurality of SA layers uses a multilayer perceptron to nonlinearly map a subset of local neighborhood points of each center point into a feature vector representing the local neighborhood of that center point; the number of point cloud points of the 3D point cloud is 4096; the grouping layer of the first SA layer of the plurality of SA layers samples neighborhood points in the local neighborhood around each center point, the local neighborhood sampling radius r=0.1, and the number of sampling points in the local neighborhood is 64.

[0015] According to the cattle identification method of the first aspect of this application, the training samples for training the H-PointNet++ model are sample triplets including anchor samples, positive samples, and negative samples; wherein, the anchor sample serves as a reference benchmark for identification; the positive sample and the anchor sample are 3D point clouds obtained from the face images of the same cattle; the negative sample and the anchor sample are 3D point clouds from cattle with different identities; the H-PointNet++ model is trained using the loss function L = max(d(a,p) - d(a,n) + margin,0), where a, p, and n represent the feature vectors of the anchor sample, positive sample, and negative sample, respectively; d() represents the Euclidean distance calculation function, and margin represents the margin threshold. The optimization objective of the training process is: for the sample triplets, through network parameter adjustment, the feature distance between the point sample and the positive sample is less than the feature distance between the anchor sample and the negative sample, and the difference between the two must meet the margin threshold requirement.

[0016] According to the cattle identification method of the first aspect of this application, in the early stage of training the H-PointNet++ model, a semi-hard sample triplet is constructed, consisting of an anchor sample, a positive sample, and a negative sample whose feature vector is far from the anchor sample but not completely separated; the H-PointNet++ model is trained using the semi-hard sample triplet; in the middle stage of training the H-PointNet++ model, a mixed sample consisting of a semi-hard sample triplet and a hard sample triplet is constructed, and the H-PointNet++ model is trained using the mixed sample; wherein the negative sample in the hard sample triplet comes from the heterogeneous sample whose feature vector is closest to the anchor sample; in the later stage of training the H-PointNet++ model, the H-PointNet++ model is trained using the hard sample triplet.

[0017] According to the cattle identification method of the first aspect of this application, in order to construct semi-hard sample triples, the anchor sample and multiple negative samples with different identities from the anchor sample are mapped to feature vectors using the currently trained H-PointNet++ model. The negative samples whose feature vectors are greater than the distance between the feature vectors of the positive samples and the feature vectors of the anchor sample, and whose distance difference is less than the margin, are selected to construct semi-hard sample triples. In order to construct hard sample triples, the anchor sample and multiple negative samples with different identities from the anchor sample are mapped to feature vectors using the currently trained H-PointNet++ model. The negative sample whose feature vectors are less than the distance between the feature vectors of the anchor sample and the feature vectors of the anchor sample are selected to construct hard sample triples.

[0018] According to the cattle identification method of the first aspect of this application, in order to construct semi-hard sample triples, the anchor sample and multiple negative samples with different identities from the anchor sample are mapped to feature vectors using the currently trained H-PointNet++ model. Negative samples whose feature vectors are more distant from the feature vectors of the anchor sample than the distance between the feature vectors of the positive sample and the feature vectors of the anchor sample, and whose distance difference is less than the margin, are selected to construct semi-hard sample triples. In order to construct hard sample triples, the anchor sample and multiple negative samples with different identities from the anchor sample are mapped to feature vectors using the currently trained H-PointNet++ model. Negative samples whose feature vectors are more distant from the feature vectors of the anchor sample than the distance between the feature vectors of the positive sample and the feature vectors of the anchor sample are selected to construct hard sample triples.

[0019] According to the cattle identification method of the first aspect of this application, the currently trained H-PointNet++ model used to construct semi-difficult sample triples is an H-PointNet++ model that has not yet been trained using semi-difficult and difficult samples; the currently trained H-PointNet++ model used to construct difficult sample triples is an H-PointNet++ model trained with semi-difficult samples.

[0020] According to the cattle identification method of the first aspect of this application, during the training process, semi-difficult samples and difficult samples are selected from the training samples; wherein the selection criteria for semi-difficult samples are: the similarity between the feature vector of the negative sample and the feature vector of the anchor point is greater than the distance between the feature vector of the positive sample and the feature vector of the anchor point sample, and this distance difference is less than the margin; the selection criteria for difficult samples are: the distance between the feature vector of the negative sample and the feature vector of the anchor point sample is less than the distance between the feature vector of the positive sample and the feature vector of the anchor point sample.

[0021] According to a second aspect of this application, an information processing apparatus according to the second aspect of this application is provided, including a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the method according to the first aspect of this application.

[0022] The technical effects achieved by this application include: 1. Breakthrough in open set recognition capabilities: The network output does not rely on a fixed set of categories and can directly achieve the rejection of unknown individuals and dynamic expansion of the identity database through similarity calculation, which is fully adapted to the actual needs of continuous addition of individuals in cattle breeding scenarios.

[0023] 2. Improved ability to distinguish highly similar individuals: The adaptive design of the SA1 module combined with progressive training on difficult samples enables the network to accurately capture subtle geometric differences in the local area of ​​cow faces. In scenarios where individuals have highly similar structures, the accuracy of identity recognition is improved by more than 30% compared to the existing PointNet++.

[0024] 3. Enhanced robustness in complex environments: Through feature regularization, vector normalization, and progressive training strategies, the model has strong anti-interference capabilities against noise, pose changes, local occlusion, and other interference factors during the acquisition process, and the recognition accuracy is stable at over 95% in actual production environments. 4. Engineering practicality optimization: The feature vector output format supports fast similarity calculation (single comparison time ≤ 1ms), and the expansion of the cattle feature database does not require retraining the model, reducing deployment and maintenance costs.

[0025] 5. Expansion of technical versatility: Although designed for cow face recognition, the core technical framework can be transferred to other high-similarity 3D individual recognition scenarios (such as livestock recognition such as sheep and pigs), and has broad application prospects. Attached Figure Description

[0026] When read in conjunction with the accompanying drawings, and by referring to the following detailed description of illustrative embodiments, this application, its preferred modes of use, and further objects and advantages will be best understood, wherein the drawings include: Figure 1A The working principle and processing flow of a cattle identification method according to an embodiment of this application are demonstrated.

[0027] Figure 1B The working principle and processing flow of a cattle identification method according to another embodiment of this application are demonstrated.

[0028] Figure 1C The working principle and processing flow of a cattle identification method according to another embodiment of this application are demonstrated.

[0029] Figure 2A schematic diagram illustrating the process of acquiring and preprocessing cow face images according to an embodiment of this application is shown.

[0030] Figure 3 A schematic diagram illustrating the structure and principle of the H-PointNet++ model according to an embodiment of this application is shown.

[0031] Figure 4 It displays 3D point clouds of different types of objects.

[0032] Figure 5 This is a schematic diagram of the sampling points and sampling radius of SA1 in the H-Pointnet++ model according to an embodiment of this application.

[0033] Figure 6 A schematic diagram of the network structure of the feature vector output head according to an embodiment of this application is shown.

[0034] Figure 7 This is a block diagram of an information processing device that implements the embodiments of this application. Detailed Implementation

[0035] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0036] Figure 1A A schematic diagram illustrating the working principle and processing flow of a cattle identification method according to an embodiment of this application is shown.

[0037] In this embodiment, a 3D point cloud of the cow to be identified is generated. A 3D recognition model is used to convert the 3D point cloud of the cow into a 3D feature vector representing the 3D features of the cow's face. The similarity is then calculated between this vector and the 3D feature vectors of cows in a cow feature database to obtain the cow's identity.

[0038] To obtain a 3D point cloud of a cow's face, the 3D point cloud of the cow's face can be reconstructed using LiDAR, 3D scanning equipment, or from multiple 2D images.

[0039] The 3D recognition model receives a 3D point cloud and outputs a 3D feature vector. 3D recognition models include, for example, PointNet, PointNet++, Point Transformer, Uni3D, and H-PointNet++ according to embodiments of this application.

[0040] The identification of cattle is based on the extracted 3D feature vectors of their faces. For example, the identification ID or identifier of the cattle to be identified is determined from a cattle feature database. The similarity between the 3D feature vectors and the 3D feature vectors in the cattle database is calculated. For example, the identification results can be the top N cattle identified based on their 3D face feature vectors, or the cattle in the database with the highest similarity ranking and a similarity exceeding a confidence threshold can be used as the identification result.

[0041] The cattle feature database pre-stores 3D facial feature vectors of cattle. Before cattle identification, a cattle feature entry process is performed. This entry only needs to be done once. The cattle feature entry process is the same as... Figure 1A The demonstrated process is similar. A 3D feature vector of the cow's face is generated. This 3D feature vector, along with the cow's identity information, is recorded in a cow feature database for subsequent feature vector similarity calculations.

[0042] Figure 1B A schematic diagram illustrating the working principle and processing flow of a cattle identification method according to yet another embodiment of this application is shown.

[0043] The cattle identification system of this application includes a data acquisition device, a computer, and a cattle feature database.

[0044] See Figure 1B Images of cattle heads are captured using image acquisition devices, such as cameras or mobile phones. To generate 3D point clouds, specific pose images of the cattle faces are captured: a frontal view, a 45-degree left-side view, and a 45-degree right-side view. For each cow, at least one image of each of these specific poses is captured.

[0045] Optionally, a YOLOv8-based cow face recognition model is run in the image acquisition device to identify the pose of cow faces in captured images in real time. When the cow face image acquired by the data acquisition device is a frontal, left 45-degree, or right 45-degree angle, automatic capture is triggered to obtain a cow face image with a specific pose required for subsequent processing. It is understood that running the YOLOv8-based cow face recognition model to achieve automatic capture is not mandatory, and users can capture cow face images with specific poses. Alternatively, video data of cow faces including various poses can be acquired through the image acquisition device, and the cow face images with specific poses required for 3D point cloud generation for 3D reconstruction can be extracted from the video data in real time, or not.

[0046] Images of cow faces captured by image acquisition equipment are provided to a computer.

[0047] For each cow, three images of its face are received: a frontal image, a 45-degree left-side image, and a 45-degree right-side image, and 3D feature vectors are generated. Specifically, a 3D model of the cow's face is generated using the Dust3R deep learning model (Dense and Unconstrained Stereo 3D Reconstruction). A 3D point cloud is generated from the 3D model. Finally, a 3D feature vector of the cow's face is generated from the 3D point cloud using a 3D recognition model for classification or object detection. The generated 3D feature vector of the cow's face is, for example, a 256-dimensional stereo geometric feature vector.

[0048] The identity of cattle is identified based on the extracted 3D feature vectors of the cow's face.

[0049] The cattle feature database pre-stores 3D facial feature vectors of cattle. The process of inputting cattle features into the cattle feature database is the same as... Figure 1B The demonstrated process is similar. Ideal images of cow faces are acquired using data acquisition equipment, including a frontal view, a 45-degree left-side view, and a 45-degree right-side view. A 3D feature vector is jointly generated from the frontal, left-side, and / or right-side cow face images. This 3D feature vector, along with the cow's identification information, is recorded in a cow feature database for subsequent feature vector similarity calculations.

[0050] In another embodiment of this application, addressing the inherent bottlenecks in accuracy and robustness of existing single-modal 2D image recognition or 3D point cloud recognition methods, a dual-branch recognition architecture of "multimodal collaborative authentication strategy" is proposed. This architecture fully leverages the insensitivity of 3D point cloud features to lighting, pose changes, and dirt interference to accurately extract the three-dimensional geometric features of a cow's face; simultaneously, relying on the high discriminative power of 2D images in details such as patterns and hair texture, it efficiently extracts planar texture features. Through collaborative decision-making between 3D and 2D features, accurate identification of the cow's identity is achieved.

[0051] Figure 1C A schematic diagram illustrating the working principle and processing flow of a cattle identification method according to another embodiment of this application is shown.

[0052] same Figure 1B compared to, Figure 1C The illustrated embodiment also generates two-dimensional features of cow face images for cattle identification. One or more cow face images from the front, left 45-degree angle, and right 45-degree angle can be used to generate the two-dimensional features.

[0053] The image of the cow's face captured by the image acquisition device is provided to the computer. The computer-implemented processing flow includes both 2D processing branches and 3D processing branches.

[0054] The 2D processing branch receives a cow face image and generates a 2D feature vector from it. Generating 2D feature vectors from images is a pre-existing technique. For example, a pre-trained FaceNet model can be used to process a frontal cow face image to generate a 2D feature vector reflecting the cow's identity. Another example is using a deep convolutional neural network to extract a 256-dimensional feature vector reflecting the cow's texture and patterns.

[0055] The 3D processing branch receives, for example, three cow face images for each cow: a frontal cow face image, a cow face image at a 45-degree angle to the left, and a cow face image at a 45-degree angle to the right, and generates a 3D feature vector.

[0056] Multimodal fusion decision-making is performed on the extracted 2D and 3D feature vectors of the cow's face to identify the cow's identity. For example, it identifies the identity ID or identifier of the cow captured by the image acquisition device in the cow feature database. The similarity between the 2D and 3D feature vectors of the cow's face and the 2D and 3D feature vectors of the cow's face in the cow database is calculated to obtain the top N cow identification results based on the 2D feature vectors and the top N cow identification results based on the 3D feature vectors of the cow's face. Intersection analysis and confidence judgment are performed on the top-N results of the two modalities, and the final identification result is output according to the preset rules (successfully identified cow, identified as a known cow not belonging to the cow feature database, or prompts to re-acquire cow face images).

[0057] The cattle feature database pre-stores 2D and 3D facial feature vectors of cattle. Before cattle identification, a cattle feature input process is performed. The cattle feature input process is the same as... Figure 1C The demonstrated process is similar. Ideal images of cow faces are acquired using data acquisition equipment, including a frontal view, a 45-degree left-side view, and a 45-degree right-side view. 2D feature vectors are generated from the frontal, left-side, and / or right-side cow face images. A 3D feature vector is then generated by jointly analyzing these three images. The 2D and 3D feature vectors, along with the cow's identification information, are recorded in a cow feature database for subsequent feature vector similarity calculations.

[0058] Figure 2 A schematic diagram illustrating the process of acquiring and preprocessing cow face images according to an embodiment of this application is shown.

[0059] The image acquisition and preprocessing workflow is responsible for transforming the raw images acquired by the image acquisition device into high-quality, standardized input images and determining their subsequent processing path. The workflow is as follows: Figure 2 As shown, the specific steps are as follows: Step 1: Cow Face Data Acquisition. As an example, the image acquisition device uses a cow face recognition model based on YOLOv8 as its core technology, possessing automated data acquisition and classification capabilities. When acquiring cow face images, the cow face recognition model identifies the cow face posture in real time. When the acquired cow face posture is in a frontal view, a 45-degree left view, or a right view, automatic image capture is triggered, and the cow face image is simultaneously classified by viewpoint to label the posture information corresponding to the cow face image (frontal view, 45-degree left view, or right view).

[0060] Step 2: Cow face region segmentation. A cow face recognition model based on YOLOv8 is used to accurately segment the clean cow face region from the acquired original image, eliminating interference from complex backgrounds.

[0061] Optionally, steps 1 and 2 are performed by video capture, and multiple images of cow faces can be continuously accumulated before performing subsequent steps.

[0062] Step 3: Image quality screening and viewing angle determination.

[0063] After obtaining the segmented cow face image output from step 2, the images are first filtered, and those meeting quality requirements are used for subsequent processing. For example, the image is identified to determine if it contains a complete cow face; images containing a complete cow face are selected as meeting quality requirements. Next, the pose of the cow face in the image is identified. The poses of the cow face from a frontal view, a 45-degree left-side view, or a right-side view are used as the target poses. If the cow face image does not have a target pose, the current image is discarded, and the process returns to the step of obtaining the processed cow face image to obtain the next cow face image, repeating the above processing steps in step 3.

[0064] To identify whether a cow face image represents the target pose, optionally, the cow face image is aligned with preset optimal viewpoint templates: a frontal cow face image, a cow face image at a 45° left angle, and a cow face image at a 45° right angle. The confidence score of the cow face image to be processed matching each optimal viewpoint template is then output. When the confidence score of a certain viewpoint is the highest and exceeds a threshold (e.g., 0.8), the image to be processed is determined to have the target pose. By filtering the quality and viewpoint of the cow face images, high-quality cow face images are selected for subsequent 2D feature extraction and / or 3D point cloud reconstruction.

[0065] Step 4: Intelligent scheduling of image processing branches.

[0066] When identifying cattle solely based on 3D features reconstructed from cow face images (e.g., Figure 1B In one embodiment, the cow face image of the target pose is sent to the 3D processing branch.

[0067] When identifying cattle based on 2D and 3D features generated from cow face images (e.g., Figure 1C (In the embodiment), intelligent scheduling of 2D processing branches and 3D processing branches is used to send the selected cow face images to the appropriate processing branches.

[0068] For example, for selected cow face images, if there are at least one qualified image each from three perspectives—frontal view, left 45° view, and right 45° view—then these images are sent to the 3D processing branch, and simultaneously sent to the 2D recognition branch in parallel. If the above multi-view condition is not met, but there are qualified frontal or near-frontal view images, then all images are sent to the 2D processing branch.

[0069] Specifically, see Figure 2 Each time a preprocessed cow face image is acquired, if the current cow face image has a target pose (frontal view, left 45° view, or right 45° view), in addition to sending the cow face image to the 2D processing branch, it is further determined whether cow face images of all three views exist. If there is at least one cow face image of each of the three views, then the cow face images of all three views are sent to the 3D processing branch; otherwise, the process returns to the step of acquiring the preprocessed cow face image.

[0070] Optionally, when determining whether cow face images from all three perspectives exist, the number of cow face images already acquired for the current cow is also counted. If the number of cow face images from any perspective exceeds a threshold (e.g., 1), it is further determined whether the total number of cow face images already acquired for the current cow exceeds a threshold (e.g., 5). If the total number of cow face images exceeds the threshold (e.g., 5), all these cow face images are sent to the 2D processing branch; otherwise, the process returns to the step of acquiring preprocessed cow face images. Optionally, if the total number of acquired cow face images exceeds the threshold (e.g., 5), the cow face image from the frontal view or the closest to the frontal view is selected from these images and sent to the 2D processing branch.

[0071] When determining whether the current cow face image has the target pose (frontal view, left 45° view, or right 45° view), if the current cow face image does not have the target pose, the process proceeds to the step of determining whether the total number of cow face images acquired for the current cow exceeds a threshold (e.g., 5). If the total number of cow face images exceeds the threshold, all of these cow face images are sent to the 2D processing branch; otherwise, the process returns to the step of acquiring the preprocessed cow face image.

[0072] In this embodiment, to generate the 3D feature vector of a cow's face, a 3D model of the cow's face is obtained from 2D images from multiple perspectives through 3D reconstruction. A high-quality, standardized 3D point cloud is then generated and fed into a 3D recognition model to extract the 3D feature vector. Generating the 3D point cloud of the cow's face is also a fundamental operation in the 3D recognition branch.

[0073] Using the Dust3R model (https: / / arxiv.org / abs / 2312.14132, https: / / github.com / naver / dust3r), input three selected images of cow faces (each with a frontal view, a left 45° view, or a right 45° view) to obtain a 3D mesh model of the cow's face.

[0074] The 3D mesh model is converted into an initial point cloud. Optionally, a random downsampling method is used to unify the number of points in the cloud to a fixed size to improve the efficiency of subsequent processing.

[0075] Optionally, the 3D point cloud is also aligned to convert multiple 3D point clouds into the same size and consistent pose.

[0076] In this embodiment, a 3D point cloud of a cow's face is fed into a 3D recognition model to extract 3D feature vectors. Optionally, for a 3D recognition model that outputs classification results, the output of the last hidden layer before its classification output head is used as the extracted 3D feature vector. Common 3D recognition models include, for example, PointNet, PointNet++, PointMLP (https: / / arxiv.org / abs / 2202.07123), and TetraSphere (https: / / arxiv.org / abs / 2211.14456).

[0077] In the embodiments of this application, the PointNet++ network was also specifically improved (referred to as H-PointNet++) to better capture the 3D features of the cow's face.

[0078] PointNet++ (https: / / arxiv.org / pdf / 1706.02413) is a landmark achievement in the field of 3D vision. Through its innovative design of hierarchical sampling and local feature aggregation, it can accurately capture the local details and global structure of 3D point clouds. Existing PointNet++ models are mainly applied to point cloud classification and segmentation tasks, and are not well-suited for cattle face recognition scenarios—its training data consists mostly of point clouds with large class differences, uniform and regular distribution, and no noise. However, the point clouds in cattle identification tasks are characterized by disorder and high similarity between samples, showing a significant difference from the distribution of PointNet++ training data. The original design of the PointNet++ model is primarily for object classification or part segmentation. When directly applied to the identification of similar individuals of the same species requiring extremely high discriminative power, the learned features are not sensitive enough to fine-grained discrimination (such as nose prints, spots, subtle hair direction, and other texture information that has high discriminative power for individual cattle).

[0079] To extract highly discriminative features from the 3D point cloud of a cow's face, the H-PointNet++ model provided in this application is an improvement on the PointNet++ model.

[0080] (I) Overall Technical Concept of H-PointNet++ Model To address the objective challenges in 3D point cloud identity recognition tasks, such as high structural similarity between individuals, open sets in actual application scenarios, complex data sources, and uncontrollable noise, the H-PointNet++ model has made systematic adjustments to the PointNet++ model based on its design goals.

[0081] This invention argues that the core of identity recognition based on 3D point clouds lies not in coarse-grained differentiation of the overall shape of the point cloud, but rather in: 1) At the local level, fully capture fine-grained geometric structures that reflect individual differences; 2) At the global level, the aforementioned local differences are stably mapped to a unified identity feature representation space; 3) At the output level, the network output is no longer dependent on a fixed set of categories, but forms a measurable and scalable identity feature vector.

[0082] Based on the above understanding, this invention proposes an improved point cloud feature extraction network, H-PointNet++. This network is based on the hierarchical feature learning idea of ​​PointNet++, but it is specifically reconstructed around the application goal of "open set identity recognition", and the grouping strategy, feature representation form and training mechanism are redesigned to enable the network to work stably in real production environment.

[0083] PointNet provides mapping functions .

[0084] Its input is an unordered set of points {x1, x2, ..., x}. n The output is a feature vector representing the set of points. The unordered set of points consists of n points, each with C-dimensional coordinates and d-dimensional original features. h and γ are both multilayer perceptron mapping networks (MLPs), and max() represents global max pooling. In PointNet++, this mapping function is also called a PointNet layer.

[0085] PointNet++ proposes a hierarchical grouping network architecture based on PointNet. The backbone network consists of several stacked SA (Set Abstraction) layers. Each SA layer receives a set of points, performs feature abstraction, and outputs a new set of points with fewer points. A single SA layer consists of three layers: a sampling layer, a grouping layer, and a PointNet layer.

[0086] Sampling layer: Selects a subset of points from all input points as the center points of a local region; Grouping layer: Sample neighboring points around each center point to construct several local region point subsets; PointNet layer: Encodes the spatial features of each local neighborhood into a feature vector.

[0087] The input to a single SA layer is a matrix of size N x (d + C): corresponding to N input points, each containing d-dimensional spatial coordinates and C-dimensional original features; the output is a matrix of size N' x (d + C'): N' represents the N' center points obtained by downsampling the input points, carrying d-dimensional spatial coordinates and C'-dimensional new features fused with local context. Therefore, the operation of the SA layer is also simply referred to as grouping and downsampling.

[0088] By concatenating multiple SA layers, the number of points is continuously reduced, and the receptive field is continuously increased, resulting in the output of the backbone network. In classification applications, the backbone network output outputs the classification result through the output header of the fully connected layer.

[0089] (II) Overall Structure of H-PointNet++ H-PointNet++ still employs a hierarchical point cloud feature extraction framework, including multiple cascaded SA layers and a feature vector output head. Unlike PointNet++, H-PointNet++ does not aim for class discrimination as its final goal. Instead, it treats the entire network as a 3D identity feature mapping function, used to map the input 3D point cloud into identity feature vectors of a uniform scale.

[0090] The H-PointNet++ network takes normalized 3D point cloud data of cow faces as input and outputs a fixed-dimensional feature vector representing the identity of a cow. This feature vector is directly used for subsequent similarity calculation and identity determination.

[0091] Figure 3 The H-pointnet++ network structure according to an embodiment of this application is shown.

[0092] (III) First Floor SA (SA1) The PointNet++ network employs a uniform sampling density and neighborhood scale configuration in its first SA layer, primarily designed to achieve general modeling capabilities for different categories of 3D objects. (See also...) Figure 4 When used to distinguish object categories, such as Figure 4 In the examples (a) airplane, (b) water cup, and (c) cow, the PointNet++ model performs well in tasks where there are significant differences in object categories and structures, but it shows obvious shortcomings in tasks involving the identification of highly similar individuals, such as cow faces.

[0093] Specifically, the 3D point clouds of different cattle faces exhibit a high degree of consistency in their overall structure. However, the discriminative information that significantly distinguishes individual cattle is not evenly distributed throughout the point cloud space, but rather concentrated in a few local areas (such as the eyes and horns). If a large-scale neighborhood or sparse sampling strategy is adopted in the first SA layer (SA1), fine-grained differences are easily averaged or weakened in shallow layers, making it difficult to recover such differential information even if the network depth is increased in subsequent layers.

[0094] Therefore, the structural design of the first-layer SA directly determines whether the network has the ability to effectively model individual identity differences. This problem cannot be solved by simply adjusting parameters, but belongs to the technical problem of network structure adaptation.

[0095] To address the aforementioned issues, the H-PointNet++ network in this application redesigns the first SA layer from the perspective of group modeling objectives. That is, the core task of the first SA layer is no longer to quickly abstract the overall structure, but to retain the local geometric information that is discriminative for individual differentiation as completely and precisely as possible.

[0096] Based on this technical concept, the design of the first SA layer must simultaneously meet the following principles: (1) At the spatial coverage level, improve the probability of sampling points covering key local areas; (2) At the local modeling level, avoid introducing large-scale neighborhood information too early; (3) At the feature expression level, enhance the ability to nonlinearly characterize subtle geometric changes.

[0097] Specifically, (1) In order to retain more high-discrimination regions of the cow face, the number of input point cloud points of the first SA layer of H-Pointnet++ is 4096.

[0098] (2) High-density farthest point sampling strategy The sampling density of the first (SA) layer exceeds that of PointNet++'s SA1 layer, making the sampling points more evenly and densely distributed in the 3D point cloud space. This increases the probability that key local regions of the cow face will be selected as center points, avoiding the omission of discrimination regions due to sparse sampling. The first SA layer of H-Pointnet++ has 1024 sampling points.

[0099] (3) Small-scale local neighborhood modeling strategy The first SA layer uses a smaller neighborhood search radius than PointNet++'s SA1, allowing each local region to focus on detailed structural changes and reducing interference from irrelevant regions on local features. This ensures that fine-grained differences are not smoothed out in shallow layers from a structural perspective. The sampling radius of H-Pointnet++'s first SA layer is 0.1.

[0100] (4) Enhanced local feature mapping strategy Within a local neighborhood, a multilayer perceptron structure is used to perform nonlinear mapping on the point set, thereby improving the network's ability to express small changes in local geometry while maintaining computational controllability.

[0101] Through the synergistic effect achieved by adjusting the respective strategies of sampling, grouping, and local feature mapping in the SA1 layer, the features output by the SA1 layer have high identity discrimination potential at the shallow stage, providing a reliable foundation for subsequent layer feature abstraction.

[0102] Figure 5 This is a schematic diagram of the sampling points and sampling radius of SA1 in the H-PointNet++ model according to an embodiment of this application.

[0103] Figure 5 The diagram illustrates 10 local regions obtained by grouping and sampling the 3D point cloud of a cow's face using the SA1 layer. These local regions correspond to sampling points of the SA1 layer and their local neighborhoods. As can be seen from the diagram, the size of the local regions effectively covers important, discriminative structures in the 3D point cloud of the cow's face (e.g., eyes, horns), avoiding the situation where a single local region is too large and simultaneously covers multiple important structures on the cow's face, or too small and only covers a portion of an important structure. It's also important to understand that the sampling points are selected randomly or uniformly, without needing to know the location and features of important structures on the cow's face. By using a high-density sampling point setting (1024 points and a sampling radius r=0.1), any important structure on the cow's face is likely to be effectively covered by several sampling points and their local neighborhoods. Even if some sampling points fail to effectively cover important structures on the cow's face, it does not affect the overall extraction of detailed features from the 3D point cloud of the cow's face.

[0104] (iv) Subsequent SA layer and global identity feature modeling After completing the adaptive grouping and fine-grained feature modeling of the first SA1 layer, the subsequent SA layers of the H-PointNet++ network follow the PointNet++ model. Therefore, in terms of the overall backbone network structure, the first SA layer of this embodiment can be regarded as another SA layer added before the first SA layer of the PointNet++ network.

[0105] Specifically, subsequent SA layers reduce the number of points involved in the computation step by step and correspondingly expand the spatial scale of the local neighborhood, allowing the local discriminative features extracted in the shallow stage to be aggregated over a larger spatial range. This aggregation process is not a simple spatial downsampling, but rather a gradual introduction of larger-scale structural context information while preserving local identity differences.

[0106] Since the first layer SA1 has already performed high-resolution modeling of key local regions of the cow face, the subsequent SA-level feature abstraction process builds upon this foundation, linking multiple local structures with identity-discriminating significance to form an identity-level representation of the overall geometric morphology of the cow face. The feature representation constructed in this way possesses the following characteristics: 1) At the local level, retain detailed information that is sensitive to individual differences; 2) At the global level, it has strong robustness to overall morphological changes under different collection conditions.

[0107] In the H-PointNet++ model, the hierarchical modeling path of "gradually converging local identity features into global identity representation" effectively avoids the problem of unstable identity representation caused by focusing only on local features, and provides a reliable foundation for subsequent feature vector output.

[0108] In the H-PointNet++ implementation, to preserve detailed features, the first layer SA (Self-Sampling) uses farthest point sampling (FPS) to obtain 1024 sampling points as center points from the input 4096-point cow face 3D point cloud. For each center point, sampling is performed within its local neighborhood using a small local neighborhood sampling radius (r=0.1) and a large number of sampling points within the local neighborhood (e.g., 64). This approach not only preserves rich fine-grained features but also allows each local neighborhood to integrate surrounding feature information, achieving an organic combination of fine-grained features and the global structure of the cow face point cloud. This is a key step in balancing local details and overall distribution.

[0109] The subsequent three SA layers follow the core logic of PointNet++, gradually reducing the number of sampling points, increasing the local neighborhood radius and the number of sampling points in the neighborhood, and gradually abstracting and refining fine-grained key features with strong discriminative power, providing stable and effective feature support for the feature vector output head.

[0110] The output of the fourth layer SA is provided to the feature vector output head after passing through the feature extraction network.

[0111] (v) Feature Vector Output Head Design In open-set identity recognition applications for cow faces, the system will inevitably encounter a large number of individuals that did not appear during the training phase. Therefore, the network output must break free from its dependence on a fixed set of categories and use only the similarity of identity features as the criterion for judgment.

[0112] Traditional point cloud classification networks typically use fully connected layers and a softmax function to output class probabilities. This type of output is essentially based on the "class closure assumption," making it unsuitable for calculating similarity for unknown identities or for applications requiring identity registration and dynamic expansion.

[0113] To address the aforementioned issues, this application's embodiments specifically design the output header of the H-PointNet++ network, transforming it from a category-based discriminative output into an identity feature vector output header. This output header is used to map the high-dimensional point cloud features extracted by the backbone network into fixed-dimensional identity feature vectors.

[0114] The H-PointNet++ network feature vector output head includes: (1) a feature compression layer, which reduces the feature dimension and reduces redundant information while maintaining the discriminative ability; (2) a feature distribution normalization layer, which constrains the statistical distribution of features during training and improves the comparability between samples from different batches; (3) a nonlinear mapping layer, which enhances the feature expression ability and makes the identity feature space more discriminative; (4) a regularization layer, which suppresses overfitting and improves the robustness of features to acquisition noise and pose changes; and (5) a vector normalization layer, which maps the output features to a unified scale space.

[0115] The feature vectors obtained after processing the feature vector output head described above can be directly used for similarity calculation based on Euclidean distance or cosine distance, thereby realizing cattle identification under open set conditions.

[0116] Figure 6 A schematic diagram of the network structure of the H-PointNet++ network feature vector output header according to an embodiment of this application is shown.

[0117] See Figure 6 First, a feature compression layer (shown as a fully connected layer in blue) compresses the 1024-dimensional features at each point of the fourth layer (SA) of the backbone network to 512 dimensions at each point. The output features then enter a combination of a feature distribution normalization layer (BN), a nonlinear mapping layer (ReLU), and a regularization layer (Dropout) to complete a full round of normalization and regularization. Subsequently, it passes through a combination of neural network layers including a feature distribution normalization layer (BN), a nonlinear mapping layer (ReLU), and a regularization layer (Dropout), and then through a feature compression layer (fully connected layer) to compress the 512-dimensional features at each point of the previous neural network layer combination to 256 dimensions at each point.

[0118] Finally, the L2 normalization process is performed on the 256-dimensional feature vector of each point through a normalization layer and a linear layer, so that the output feature vector conforms to the requirements of Euclidean distance measurement and ensures the accuracy of similarity calculation.

[0119] This feature vector output head structure can progressively distill the complex geometric information of the captured cow face into 256 ready-to-use, highly discriminative features. This design not only ensures the efficiency and stability of model training, but also makes the output features suitable for downstream tasks requiring high-precision similarity comparisons, such as cow identity verification.

[0120] (vi) Identity Feature Learning Method Based on Triple Constraints In the cow face open set identity recognition task, the system ultimately determines identity based on the similarity between identity features. Therefore, the core objective of network training is not to learn the discrimination boundary between fixed categories, but to construct an identity feature space with clear geometric constraints, so that samples of the same identity are highly clustered in this space, and samples of different identities maintain a stable and separable distance interval.

[0121] Traditional classification-based loss functions aim to maximize class probability, and their constraint object is the class label itself. They are difficult to directly apply to the geometric structure of the feature space and have no constraint ability on samples with unknown identities. Therefore, they are not suitable for open set identity recognition scenarios.

[0122] Based on the above understanding, this invention introduces a constraint method based on triplet relationships in the training phase of the H-PointNet++ model to directly model the geometric structure of the feature space.

[0123] Return to view Figure 4 In this embodiment, triplet loss is used instead of traditional cross-entropy loss to complete end-to-end training of the model. As the core loss function of the metric learning model, the training objective of triplet loss is to construct a highly discriminative feature embedding space—in this space, the feature vectors of similar samples are close to each other, while the feature vectors of dissimilar samples are far apart. This characteristic makes it more suitable for cow face authentication tasks.

[0124] The training mechanism of triplet loss uses "sample triplet" as the core unit. Each triplet contains three types of samples: anchor sample (a), positive sample (p), and negative sample (n). Anchor sample (a) serves as the reference benchmark for identity; positive sample (p) has the same identity as anchor sample (a), and is point cloud data obtained from cow face images corresponding to the same cow; negative sample (n) has a different identity from anchor sample (a), and comes from point cloud data of cows with different identities (different identities).

[0125] During training, for each sample triplet, the H-PointNet++ network extracts the identity feature vectors of the three sample classes respectively, and imposes constraints on their distance relationships in the feature space, ensuring that the feature distance between the anchor sample and the positive sample is less than the feature distance between the anchor sample and the negative sample, and that there is at least a preset margin between them. This constraint relationship can be formally expressed as: L = max(d(a,p) - d(a,n) + margin, 0) In this formula, a, p, and n represent the feature vectors of the anchor point, sample, positive sample, and negative sample in the feature space, respectively; d() represents the Euclidean distance calculation function, and margin represents the margin threshold. This loss function enhances the class discrimination of the feature vectors by directly optimizing the distance distribution of samples in the feature space, and is more in line with the needs of identity verification tasks for the discriminative power of sample features compared to traditional loss functions.

[0126] By imposing the aforementioned constraints on a large number of triplet samples, the network is guided to continuously compress the distribution range of samples with the same identity in the feature space during training, while simultaneously widening the distribution interval between samples with different identities, thereby gradually forming an identity feature space with a clear geometric structure. This approach enables the H-PointNet++ network to explicitly learn the core discrimination criterion of "identity similarity" during the training phase, which is beneficial for improving the generalization recognition ability under unknown identity conditions.

[0127] (vii) Gradual Sample Difficulty Training Strategy Considering that the cow face point cloud data comes from the actual production environment, the samples are inevitably affected by factors such as pose changes, occlusion, lighting differences, noise points, and annotation errors during the collection process. If a large number of high-difficulty triplet samples are directly introduced in the early stage of training, the feature space may be forcibly stretched before it is formed, resulting in training oscillations or even model collapse; on the other hand, if only simple samples are used for training, it is difficult to effectively learn the subtle identity differences between individuals.

[0128] To optimize the feature extraction capabilities of the H-PointNet++ model, this application proposes a progressive adaptive hard sample mining strategy. This strategy simulates the human cognitive pattern of "from easy to difficult," dynamically dividing the model training process into three stages. By adaptively adjusting the difficulty of sample learning, it addresses the problems of poor stability and susceptibility to local optima in traditional hard sample mining during the initial training phase, ensuring the model always learns features under an appropriately challenging intensity.

[0129] The training triplet samples include simple samples, semi-hard samples, and hard samples. Each sample consists of a triple {anchor sample, positive sample, negative sample}, where a, p, and n represent the feature vectors of the anchor sample, positive sample, and negative sample in the feature space, respectively.

[0130] In the initial training phase, semi-difficult triplet samples are constructed first, consisting of anchor samples, positive samples, and negative samples that are far from the anchor samples but not yet completely separated. The training objective of this stage is to guide the network to form a preliminary stable identity feature space structure, avoiding disordered feature distribution due to excessive constraints.

[0131] For example, the screening criterion for semi-hard samples is: the distance between the feature vector of the negative sample and the anchor sample is greater than the distance between the feature vector of the positive sample and the anchor sample, but this distance difference is less than the margin threshold, which can be expressed as 0<d(a, n) - d(a, p)<margin. In other words, for a semi-hard sample, its positive sample and negative sample can be distinguished, but the degree of distinction is not large.

[0132] For comparison, the screening criterion for easy samples is that the distance between the negative sample and the anchor sample is greater than the distance between the positive sample and the anchor sample, and this distance difference is greater than the margin threshold. The screening criterion for hard samples is that the distance between the negative sample and the anchor point is less than the distance between the positive sample and the anchor sample, which can be expressed as d(a, n)<d(a, p). The ideal situation is that the distance between the positive sample and the anchor sample is small, and the distance between the negative sample and the anchor sample is large.

[0133] Optionally, among the hard samples, the distance between the feature vector of the negative sample and the anchor sample is the smallest distance among all negative samples different from the anchor sample (or the negative sample whose distance is less than a specified threshold).

[0134] Easy samples are not used for training. In the initial stage of training, semi-hard samples are used to calculate the loss function and update the model parameters, so as to help the H-PointNet++ model establish the basic structure of the feature space with a moderate learning intensity, and ensure the stability of initial training.

[0135] In the middle stage of training, after the feature space is initially formed, a triplet sample set composed of a mixture of random semi-hard samples and hard samples is gradually introduced, so that while maintaining the overall stability, the network starts to focus on the boundary regions between different identities, thereby further compressing the intra-class distance and expanding the inter-class distance.

[0136] In the later stage of training, hard sample triplets composed of anchor samples and heterogeneous identity samples with the closest distance in the feature space are mainly used to perform fine stretching on the identity feature space, so that the network has the ability to distinguish subtle differences between highly similar individuals.

[0137] Through the above progressive training strategy of "semi-hard samples → mixed samples → hard samples", the network can always balance stability and discriminability throughout the training process, which not only effectively avoids the instability problem in the early stage of training, but also significantly improves the discrimination ability and convergence efficiency of identity features in complex actual scenarios.

[0138] The trained H-PointNet++ model is used to receive the 3D point cloud of the cattle face obtained from the collected cattle face image, and output the corresponding 3D feature vector.

[0139] In an optional acquisition manner of sample triplets, sample triplets are randomly generated, then sample triplets meeting specified conditions are screened out from the sample triplets according to screening conditions for semi-hard samples and hard samples, so as to obtain a semi-hard sample set and a hard sample set.

[0140] In another optional acquisition manner of sample triplets, the sample triplets are constructed according to the required types of sample triplets. A sample with a known identity is used as an anchor sample, a sample having the same identity as the anchor sample is acquired as a positive sample, and samples having different identities from the anchor sample are acquired as candidate negative samples. The anchor sample and the plurality of candidate negative samples are mapped by the H-PointNet++ model respectively to obtain 3D feature vectors. The distance between the feature vector of the anchor sample and the feature vector of each candidate negative sample is compared, so as to obtain a negative sample for constructing a sample triplet from the plurality of candidate negative samples. For example, if |d(a, n) - d(a, p)| < margin, the negative sample corresponding to the feature vector n is a negative sample suitable for constructing a semi-hard sample; and the negative sample that minimizes |d(a, n)| among the plurality of negative samples is a negative sample suitable for constructing a hard sample. Optionally, there are a plurality of candidate positive samples having the same identity as the anchor sample. To construct a sample triplet, any candidate positive sample may be selected to construct the sample triplet, or feature vectors are generated for the candidate positive samples by the H-PointNet++ model, and a positive sample is selected according to the distance between the candidate positive sample and the anchor sample. To improve the training effect, the selected positive sample is preferably a candidate positive sample having a larger distance from the anchor sample. A triplet is constructed based on the selected anchor sample, positive sample and negative sample to calculate a loss, and back propagation is performed according to the loss, so as to realize continuous optimization of parameters of the H-PointNet++ network, making features of samples of the same category closer in the feature space and features of samples of different categories more separated.

[0141] In another alternative method for obtaining sample triples, a batch-based online mining approach using intra-batch feature vectors is employed during training to dynamically generate sample triples. Specifically, 3D point clouds of cow faces from a batch are first input into the H-PointNet++ network to obtain feature vectors corresponding to each cow face 3D point cloud. Then, anchor samples, samples with the same identity label as the anchor sample, and samples with different identity labels are obtained from the batch based on the sample label (cow identity). For each anchor sample, the sample with the largest distance from its own sample is selected as the positive sample; the sample with a semi-hard condition from the out-of-class sample is selected as the semi-hard negative sample, i.e., the distance between the negative sample and the anchor sample is greater than the distance to the positive sample and falls within a preset margin range; alternatively, the sample with the smallest distance from the anchor sample can be selected from the out-of-class sample as the hard negative sample. Finally, the anchor samples and their corresponding positive and negative samples are combined to obtain sample triples. The loss is then calculated based on the constructed sample triples, and backpropagation is performed based on this loss to optimize the parameters of the H-PointNet++ network. In the next phase of training, the 3D point cloud of cow faces from another batch is input into the optimized H-PointNet++ network, and sample triples are constructed using the same method described above. The loss is then calculated, and the H-PointNet++ network parameters are continuously optimized based on the loss.

[0142] In another alternative implementation, the construction of sample triples continues throughout the training process of the H-PointNet++ model. Furthermore, as the H-PointNet++ model is trained progressively, even if the criteria for identifying semi-difficult / difficult samples remain unchanged, the selected negative samples will change. For example, before or in the early stages of training, the distance is calculated using feature vectors generated by the unoptimized H-PointNet++ model and used as the basis for selecting negative samples. In the middle or later stages of training, the H-PointNet++ model is still used to select negative samples, but since it has been optimized, the feature vectors generated for the same anchor sample and candidate negative samples may differ, leading to changes in the negative sample selection results. This achieves dynamic adjustment of sample difficulty based on the training progress.

[0143] According to an embodiment of this application, cattle identification is performed based on the calculated 3D feature vector representing the cow's face.

[0144] Optionally, also see Figure 1A The similarity (e.g., Euclidean distance) between the calculated 3D feature vector and the 3D feature vectors of known cattle in the cattle feature database is calculated. The highest-ranked or top N candidate results in the cattle feature database are returned as the cattle identification results.

[0145] Optionally, a confidence threshold is also used, based on the similarity (distance between feature vectors) between the candidate result and the 3D feature vector of the cattle to be identified. Only one or more candidate results with a similarity exceeding the confidence threshold are recognized as cattle identification results. If the similarity of any candidate result does not exceed the confidence threshold, then a result is returned indicating, for example, rejection, identification failure, or explicit indication that it does not belong to any known cattle.

[0146] Furthermore, the output of the trained H-PointNet++ model directly reflects the spatial distribution of feature vectors without relying on a known set of categories, thus possessing natural adaptability for identifying unknown cattle. It also exhibits good adaptability for newly registered cattle, allowing for dynamic expansion of the cattle feature database during use. For example, for a new cattle, the processing method of this application embodiment uses the same generation method as other feature vectors in the cattle feature database to obtain its facial 3D point cloud, generate a 3D feature vector, and input it along with its identity information into the cattle feature database, thereby achieving identification of the new cattle without retraining the H-PointNet++ model. When the new cattle needs to be identified, a 3D feature vector is generated for it using the method of this application embodiment, and its similarity is calculated with the 3D feature vectors of known cattle in the cattle feature database to obtain the identification result. This meets the practical needs of continuously adding individuals in cattle farming scenarios.

[0147] To evaluate the technical performance of the H-PointNet++ model provided in this application, an ablation experiment was conducted on the first SA1 layer of the H-PointNet++ model. Table 1 shows the results of the ablation experiment. In Table 1, Top1 accuracy refers to the result of comparing the input sample with the feature vectors of all cattle in the cattle feature database and selecting the most similar one as the recognition result, i.e., Top1. Similarly, Top5 accuracy is the result of selecting the top 5 results in terms of similarity when comparing with the feature vectors of the cattle feature database. After adding the first SA1 layer, the Top1 accuracy of cattle identification improved by 8%, and the Top5 accuracy improved by nearly 10%.

[0148] Table 1

[0149] The accuracy of cattle identification results obtained by extracting 3D feature vectors using the H-PointNet++ model of this application embodiment and various existing 3D recognition models is also compared, as shown in Table 2. The data in Table 2 show that, compared to other 3D recognition models (PointNet++ and PointMLP), the H-PointNet++ model achieved the best results among all compared models in both Top1 accuracy and Top5 accuracy.

[0150] Table 2

[0151] The core of this application's embodiments lies in generating 3D feature vectors for cow faces based on 3D point clouds using the H-PointNet++ model, thereby achieving better cow identification results. Its key technical points include: The adaptive collaborative design of the SA1 module: Through the synergy of high-density sampling, small-scale neighborhood modeling and enhanced MLP, it solves the problem of insufficient capture of local differences of highly similar individuals, laying the foundation for fine-grained identity recognition. Open set-oriented feature output head reconstruction: Abandoning the traditional category probability output mode, a five-segment output head structure of "compression-normalization-nonlinear mapping-regularization-normalization" is designed to achieve measurable and scalable identity feature vector output; Triple-constrained feature space modeling: By explicitly constraining the geometric structure of the feature space, the network can directly learn the identity similarity / dissimilarity discrimination criteria, adapting to open set recognition scenarios; Progressive sample difficulty training strategy: dynamically adjust sample difficulty based on the training process to balance model training stability and discriminative power improvement, and solve the training convergence problem under complex noisy data; The identity representation mechanism of hierarchical feature aggregation: Through the hierarchical aggregation of "local fine-grained features → global related features", it realizes the identity feature expression that has both detail sensitivity and global stability.

[0152] The beneficial effects of the data processing embodiments of this application Breakthrough in open set recognition capabilities: The network output does not rely on a fixed set of categories and can directly achieve the rejection of unknown individuals and dynamic expansion of the identity database through similarity calculation, which is fully adapted to the actual needs of continuous addition of individuals in cattle breeding scenarios; Improved ability to distinguish highly similar individuals: The adaptive design of the SA1 module combined with progressive training on difficult samples enables the network to accurately capture subtle geometric differences in the local area of ​​cow faces. In scenarios where individuals have highly similar structures, the accuracy of identity recognition is improved by more than 30% compared to the existing PointNet++. Enhanced robustness in complex environments: Through feature regularization, vector normalization, and progressive training strategies, the model has strong anti-interference capabilities against noise, pose changes, local occlusion, and other interference factors during the acquisition process, and the recognition accuracy is stable at over 95% in actual production environments. Engineering practicality optimization: The feature vector output format supports fast similarity calculation (single comparison time ≤ 1ms), and the identity database expansion does not require retraining the model, reducing deployment and maintenance costs; Technical versatility expansion: Although designed for cow face recognition, the core technology framework can be transferred to other high-similarity 3D individual recognition scenarios (such as livestock recognition like sheep and pigs), and has broad application prospects.

[0153] According to embodiments of this application, in order to identify the identity of cattle, a cattle feature database is also established to store the 3D feature vectors of the 3D point cloud of the cattle faces of cattle with known identities. For example, in an insurance application scenario, when insuring cattle, qualified images of the cattle's face from three perspectives—front, left 45°, and right 45°—are taken (at least one image for each perspective).

[0154] Based on the embodiments of this application, the images from three perspectives are preprocessed, a 3D mesh model is built using the Dust3R model, and a 3D point cloud is generated. Then, the images are fed into the trained H-Point Net++ model to obtain 3D feature vectors.

[0155] All qualified images of the insured cattle from three perspectives are fed into the FaceNet model to obtain 2D feature vectors.

[0156] The obtained 3D feature vectors and 2D feature vectors are combined with the identity ID of the insured cattle to obtain the cattle feature database.

[0157] When using the embodiments of this application to identify whether a target cattle is an insured cattle, qualified images of the target cattle's face from three perspectives—front, left 45°, and right 45°—are collected. Based on the processing flow of the embodiments of this application, the identification result is output (outputting the identified insured cattle ID, indicating insufficient confidence, or identification failure (non-insured cattle)).

[0158] According to embodiments of this application, it is executed on an information processing device such as a computer or server. Figure 7 This is a block diagram of an information processing device that implements an embodiment of the present invention. Figure 7 The information processing device shown is a computer. The computer is merely one example of a suitable computing environment and is not intended to imply any limitation on the use or scope of the invention. Figure 7 The information processing equipment shown should not be construed as having any dependency or requirement on any of the components or combinations of components shown.

[0159] Information processing devices include memory, one or more processors, one or more displays, I / O components, and a power supply, directly or indirectly coupled to a bus. A bus can represent one or more types of buses (such as an address bus, a data bus, or a combination thereof). However, in practice, the definition of each component is not necessarily as straightforward. Figure 7In this way, for example, a display device can be considered as an I / O component. Furthermore, the processor may have memory. The inventors of this invention recognize that this is precisely the nature of the art and reiterate... Figure 7 The illustrations are merely for illustrating exemplary computer systems that can be used in conjunction with one or more embodiments of the present invention.

[0160] Information processing devices typically include various types of memory. By way of example and not limitation, memory may include: random access memory (RAM), read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), flash memory, CD-ROM, DVD or other optical or holographic media, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices. Computer storage media may be non-volatile.

[0161] Information processing devices include one or more processors that read data from various entities such as buses, memory, or I / O groups. One or more displays present data indications to a user or other device. Exemplary displays include display devices, speakers, printing components, vibrating components, flat panel displays, projectors, head-mounted displays, etc. A display can also be an I / O port for coupling display devices, speakers, printing components, vibrating components, flat panel displays, projectors, head-mounted displays, etc. Illustrative I / O components include cameras, microphones, joysticks, game controllers, dish satellite transceiver antennas, scanners, printers, wireless devices, etc.

[0162] Although the examples referenced in this application are described for illustrative purposes only and not for limiting the scope of this application, changes, additions and / or deletions to the implementation may be made without departing from the scope of this application.

[0163] Those skilled in the art who benefit from the teachings presented in the above description and associated drawings will recognize many modifications and other embodiments of the present application described herein. Therefore, it should be understood that the present application is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terminology is used herein, it is used only in its general and descriptive sense and not for limiting purposes.

Claims

1. A method for cattle identification based on 3D point cloud and feature extraction network model H-PointNet++, the method comprising: Obtain the 3D point cloud of the face of the cow to be identified; The H-PointNet++ model is used to receive the 3D point cloud and generate 3D feature vectors, where, The H-PointNet++ model comprises multiple SA layers and a feature vector output header connected in sequence; wherein... Each SA layer includes a sampling layer, a grouping layer, and a PointNet layer. The sampling layer is used to select a subset of points from the input points as the center points of the local region. The grouping layer samples neighborhood points in the local neighborhood around each center point to construct several subsets of local region points. The PointNet layer generates a feature vector representing the local neighborhood of each center point based on the subset of local neighborhood points of each center point. The first SA layer of the plurality of SA layers is used to uniformly sample 1024 center points from the 3D point cloud according to the farthest point. The local neighborhood sampling radius of each center point is smaller than the sampling radius of the other SA layers of the plurality of SA layers except the first SA layer. The number of local neighborhood sampling points of each center point is greater than the number of local neighborhood sampling points of the other SA layers except the first SA layer. The feature vector output head receives the output of the last SA layer among the multiple SA layers and generates a 3D feature vector representing the 3D point cloud; wherein, the feature vector output head includes a feature compression layer, a feature distribution normalization layer, a nonlinear mapping layer, a normalization layer and a vector normalization layer connected in sequence; the vector normalization layer outputs the 3D feature vector; Based on the 3D feature vector, candidate results most similar to the 3D feature vector are obtained from the cattle feature database by sorting them by similarity; wherein, the cattle feature database records the cattle's identity and its 3D feature vector in association.

2. The cattle identification method according to claim 1, wherein: The multiple SA layers, excluding the first SA layer, are multiple SA layers of the PointNet++ model; The local neighborhood sampling radius of the first SA layer of the multiple SA layers is smaller than that of the first SA layer of the PointNet++ model, and the number of local neighborhood sampling points of each center point is greater than that of the first SA layer of the PointNet++ model.

3. The cattle identification method according to claim 1, wherein: The PointNet layer of the first SA layer of the multiple SA layers uses a multilayer perceptron to nonlinearly map the local neighborhood points of each center point into a feature vector representing the local neighborhood of that center point. The number of points in the 3D point cloud is 4096; The first SA layer of the multiple SA layers samples neighborhood points in the local neighborhood around each center point. The local neighborhood sampling radius is r=0.1, and the number of sampling points in the local neighborhood is 64.

4. The cattle identification method according to any one of claims 1-3, wherein: The training samples for training the H-PointNet++ model are sample triplets including anchor samples, positive samples, and negative samples; where anchor samples serve as a reference benchmark for identity; positive samples and anchor samples are 3D point clouds obtained from cow face images corresponding to the same cow; negative samples and anchor samples are 3D point clouds from cows with different identities. The H-PointNet++ model is trained using the loss function L = max(d(a,p) - d(a,n) + margin,0), where a, p, and n represent the feature vectors of the anchor sample, positive sample, and negative sample, respectively; d() represents the Euclidean distance calculation function; and margin represents the margin threshold. The optimization objective of the training process is to adjust the network parameters to make the feature distance between the anchor sample and the positive sample less than the feature distance between the anchor sample and the negative sample, and the difference between the two must meet the margin threshold requirement.

5. The cattle identification method according to claim 4, wherein: In the early stages of training the H-PointNet++ model, a semi-hard sample triplet is constructed, consisting of an anchor sample, a positive sample, and a negative sample whose feature vector is far from the anchor sample but not completely separated; the H-PointNet++ model is then trained using the semi-hard sample triplet. During the training of the H-PointNet++ model, a mixed sample consisting of semi-hard sample triples and hard sample triples is constructed, and the H-PointNet++ model is trained using the mixed sample; wherein the negative sample in the hard sample triple comes from the heterogeneous sample that is closest to the feature vector of the anchor sample. In the later stages of training the H-PointNet++ model, the H-PointNet++ model is trained using hard sample triples.

6. The cattle identification method according to claim 5, wherein: To construct semi-hard sample triples, the currently trained H-PointNet++ model is used to map the anchor sample and multiple negative samples with different identities from the anchor sample into feature vectors respectively. Negative samples whose feature vectors are more distant from the anchor sample's feature vector than the distance between the positive sample's feature vector and the anchor sample's feature vector, and whose distance difference is less than the margin threshold, are selected to construct semi-hard sample triples. To construct hard sample triples, the currently trained H-PointNet++ model is used to map the anchor sample and multiple negative samples with different identities from the anchor sample into feature vectors respectively. The negative sample whose feature vector is the smallest distance from the feature vector of the anchor sample is selected to construct the hard sample triple.

7. The cattle identification method according to claim 5, wherein: To construct semi-hard sample triples, the currently trained H-PointNet++ model is used to map the anchor sample and multiple negative samples with different identities from the anchor sample into feature vectors respectively. Negative samples whose feature vectors are more distant from the anchor sample's feature vector than the distance between the positive sample's feature vector and the anchor sample's feature vector, and whose distance difference is less than the margin threshold, are selected to construct semi-hard sample triples. To construct hard sample triples, the currently trained H-PointNet++ model is used to map the anchor sample and multiple negative samples with different identities from the anchor sample into feature vectors. The negative samples whose feature vectors are less than the distance between the feature vectors of the positive samples and the feature vectors of the anchor samples are selected to construct hard sample triples.

8. The cattle identification method according to claim 6 or 7, wherein: The currently trained H-PointNet++ model used to construct semi-hard sample triples is an H-PointNet++ model that has not yet been trained using semi-hard and hard samples. The currently trained H-PointNet++ model used to construct hard sample triples is an H-PointNet++ model trained with semi-hard samples.

9. The cattle identification method according to claim 4, wherein: During training, semi-hard and hard samples are selected from the training samples. The selection criteria for semi-hard samples are: the similarity between the feature vector of the negative sample and the feature vector of the anchor point is greater than the distance between the feature vector of the positive sample and the feature vector of the anchor sample, and this distance difference is less than the margin. The selection criteria for hard samples are: the distance between the feature vector of the negative sample and the feature vector of the anchor sample is less than the distance between the feature vector of the positive sample and the feature vector of the anchor sample.

10. An information processing device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1-9.

Images