Method and system for 3D modeling of pig based on single-view depth camera image

Abstract

The invention provides a pig 3D modeling method and system based on a single-view depth camera image, relates to the field of 3D modeling, and solves the technical problem that group statistical modeling and individual form comparison cannot be accurately performed due to the fact that 3D point cloud data of different samples have attitude differences, scale changes and fuzzy corresponding relations of anatomical structures in the prior art. The method comprises the steps of performing semantic alignment and spatial alignment on 3D point cloud data of each sample in a 3D point cloud data set to obtain an aligned 3D point cloud data set, and analyzing average shape point cloud data B and a deformation feature vector V; inputting the aligned 3D point cloud data set into a deep learning network Net model, and outputting a feature vector C; and reconstructing according to the average shape point cloud data B, the deformation feature vector V and the feature vector C to obtain a complete 3D point cloud data model M of the pig. The method is used in the pig 3D modeling process.

Description

Technical Field

[0001] This application relates to the field of 3D modeling, and in particular to a method and system for 3D modeling of pigs based on single-view depth camera images. Background Technology

[0002] In modern intelligent farming systems, accurately acquiring the three-dimensional body phenotype of pigs is crucial for achieving efficient breeding, health assessment, and growth management. In recent years, depth cameras have gradually become an important tool for animal 3D data acquisition due to their low cost and ease of deployment. However, single-view depth images can only capture partial surface information of the pig's body, and are affected by shooting angle, pig posture, and individual scale differences, resulting in raw 3D point clouds with serious occlusion, inconsistent poses, and semantically disordered point sequences.

[0003] Current technologies fall short in addressing this issue: on the one hand, traditional point cloud registration methods cannot establish cross-sample anatomical structure correspondences, resulting in "the same index point representing different body parts"; on the other hand, multi-view reconstruction or high-precision scanning schemes are costly and complex to operate, making them difficult to scale up in actual farming environments; and purely data-driven deep learning models often lack modeling of statistical regularities in population morphology, making it difficult to learn biologically meaningful deformation features without complete 3D annotations. Therefore, there is an urgent need for a pig 3D modeling method that can achieve anatomical semantic consistency, pose robustness, and strong interpretability under single-view, low-cost, and unannotated conditions to support the intelligent upgrading of animal husbandry. Summary of the Invention

[0004] This application provides a method and system for 3D modeling of pigs based on single-view depth camera images, which solves the technical problem that existing technologies cannot accurately perform group statistical modeling and individual morphological comparison due to differences in pose, scale, and ambiguity in the correspondence of anatomical structures in different sample 3D point cloud data.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, a method for 3D modeling of pigs based on single-view depth camera images is provided, including: Acquire 3D point cloud data of multiple groups of pig samples to obtain several 3D point cloud datasets; perform semantic alignment and spatial alignment on the 3D point cloud data of each sample in the 3D point cloud dataset to obtain aligned 3D point cloud datasets. Based on the aligned 3D point cloud dataset, the average shape point cloud data B and the deformation feature vector V are analyzed; the average shape point cloud data B represents the benchmark point cloud model of the typical three-dimensional morphology of the pig population; the deformation feature vector V represents the projection coefficient of the morphological deviation of each sample's 3D point cloud data relative to the average shape point cloud data B in the principal component space. The aligned 3D point cloud dataset is input into the deep learning network Net model, and the output feature vector C is generated. The complete 3D point cloud data model M of the pig is reconstructed according to the formula M=B+C@V; where @ represents matrix multiplication.

[0006] Based on the above technical solutions, in the method for 3D modeling of pigs based on single-view depth camera images provided in this application, accurately acquiring the three-dimensional body shape features of pigs is crucial for breeding, health assessment, and growth monitoring in intelligent farming. However, raw 3D point cloud data often suffers from problems such as inconsistent point order, arbitrary poses, and severe occlusion. If directly used for modeling, it will lead to blurred average morphology of the group and distorted individual deformation information, making it difficult to support reliable phenotypic analysis. Therefore, it is necessary to establish a 3D modeling method that can effectively separate common structures from individual differences. This solution uses two-stage alignment to perform semantic alignment to unify the anatomical semantics of each point cloud, such as ensuring that the i-th point of all samples is the "tail root," and performs spatial alignment to eliminate rigid body interference caused by rotation and translation, thereby obtaining a high-quality, structurally consistent aligned point cloud dataset. On this basis, the average shape point cloud data B is calculated as a benchmark model of the typical morphology of the pig group, and the deformation feature vector V is extracted through principal component analysis, that is, the projection coefficient of the morphological deviation of each sample relative to the average shape point cloud data B in the principal component space. Aligned point clouds are input into a pre-trained deep learning network Net, which outputs a low-dimensional feature vector C. A complete 3D pig model is then efficiently generated using the linear reconstruction formula (M=B+C@V). This method requires no multi-view data or manual annotation, employs a self-supervised learning strategy, and combines computational efficiency, biological interpretability, and reconstructive controllability. It can stably extract phenotypic semantic features from single-view depth images, providing reliable and practical technical support for smart farming.

[0007] In conjunction with the first aspect above, in one possible implementation, the semantic alignment method for the 3D point cloud data of each sample includes: Select any set of 3D point cloud data from the 3D point cloud dataset and label it as the target point cloud data; use a pre-trained point cloud registration network model to register each set of 3D point cloud data in the 3D point cloud dataset with the target point cloud data respectively, to obtain a point correspondence index set between each set of 3D point cloud data and the target point cloud data. Where i is the index of the element in the index set, i = 0, 1, ..., m, and m is a positive integer; the i-th element in the index set I. This represents the index of the point in the 3D point cloud data to be aligned that has the same semantics as the i-th point in the target point cloud data; The points in each group of 3D point cloud data are reordered according to the index set, so that each 3D point cloud data has the same semantics at the same index position, resulting in a semantically aligned 3D point cloud dataset.

[0008] In conjunction with the first aspect above, in one possible implementation, the spatial alignment method for the 3D point cloud data of each sample includes: The rigid transformation parameters of semantically aligned 3D point cloud data relative to the target point cloud data are estimated by a coarse alignment network model. All 3D point cloud data are aligned to a unified coordinate system centered on the target point cloud data to obtain a spatially aligned 3D point cloud dataset. The rigid transformation parameters include 3D rotation matrices and translation vectors.

[0009] In conjunction with the first aspect above, in one possible implementation, the construction of the coarse-aligned network model includes: Obtain a training dataset, which includes multiple pairs of labeled 3D point cloud data samples, each pair containing a source point cloud data and a target point cloud data. Preprocess the source point cloud data and target point cloud data in each sample pair; A neural network architecture is constructed, comprising a shared weighted key feature extraction module, a global feature aggregation module, and a rigid body transformation parameter regression module connected in sequence. Based on the training dataset, the neural network framework is trained end-to-end with the goal of minimizing the alignment loss function. The alignment loss function is used to optimize the degree of overlap between the source point cloud data and the target point cloud data after prediction transformation, and guide the coarse alignment network to learn the correct spatial alignment ability. After training, the network parameters are fixed to obtain a coarsely aligned network model for estimating arbitrary pig 3D point cloud data relative to target point cloud data.

[0010] Fixed network parameters refer to keeping the values ​​of all learnable parameters in the neural network constant after the model training is completed.

[0011] In conjunction with the first aspect above, in one possible implementation, the construction of the alignment loss function includes: Based on the rigid body transformation parameters, the preprocessed 3D point cloud data is spatially transformed to obtain rotated point cloud data. Calculate the cosine similarity of the angle between each point in the rotated point cloud data and the corresponding point in the target point cloud data, and average the cosine similarity to obtain the alignment loss function value. : ; Where N is the number of sampling points, and n is the index of the sampling point. Let n be the position vector of the nth point in the target point cloud data. Let be the position vector of the nth point in the rotated point cloud data.

[0012] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the average shape point cloud data B includes: The aligned 3D point cloud dataset is constructed as a matrix Amr of {K,M}, where K is the number of 3D point cloud data samples and M is the coordinate dimension of each sample after flattening. The column mean of the matrix Amr is calculated in the K sample dimensions to obtain the mean vector of length M, which is labeled as the average shape point cloud data B.

[0013] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the deformation feature vector V includes: The difference between each 3D point cloud data in the aligned 3D point cloud dataset and the average shape point cloud data B is calculated to obtain a decentralized deviation point cloud matrix; principal component analysis is performed on the deviation point cloud matrix to extract the eigenvectors corresponding to the top Z largest eigenvalues, forming a principal component matrix; the deviation point cloud matrix and the principal component matrix are multiplied to obtain a deformation feature matrix; each row of the deformation feature matrix is ​​a deformation feature vector V of a sample.

[0014] In conjunction with the first aspect above, in one possible implementation, the method for obtaining the feature vector C includes: The aligned 3D point cloud dataset is input into a pre-built and trained deep learning network Net model, which includes a convolutional feature extraction module, a global context aggregation module, and a fully connected regression head, to predict low-dimensional semantic features related to pig body size.

[0015] The deep learning network Net model performs forward inference on the 3D point cloud data and outputs a feature vector C, which is used to characterize the morphological or body shape semantic features of the pig from the current perspective.

[0016] In conjunction with the first aspect above, in one possible implementation, the construction of the deep learning network Net model includes: Several aligned 3D point cloud data and corresponding feature vectors C are obtained from historical data and integrated into training and validation datasets. The deep learning network Net is trained end-to-end with the optimization objective of minimizing the absolute value of the distance between the reconstructed 3D point cloud data and the 3D point cloud data of the original input model. The distance parameter is minimized by a ladder iterative minimization algorithm. After training, a deep learning network model Net is obtained, which is used to extract feature vector C from pig depth images.

[0017] The deep learning network Net model performs forward inference on the 3D point cloud data and outputs a feature vector C, which is used to characterize the morphological or body shape semantic features of the pig from the current perspective.

[0018] Secondly, this application provides a system for 3D modeling of pigs based on single-view depth camera images, comprising: a processing module and a reconstruction module; wherein, the processing module is used to acquire 3D point cloud data of multiple sets of pig samples to obtain several 3D point cloud datasets; semantically align and spatially align the 3D point cloud data of each sample in the 3D point cloud dataset to obtain aligned 3D point cloud datasets; based on the aligned 3D point cloud datasets, analyze the average shape point cloud data B and the deformation feature vector V; the average shape point cloud data B represents a benchmark point cloud model of the typical three-dimensional morphology of the pig population; input the aligned 3D point cloud datasets into a deep learning network Net model, and output feature vector C; the reconstruction module is used to reconstruct a complete 3D point cloud data model M of the pig according to the formula M=B+C@V.

[0019] This application provides a method and system for 3D modeling of pigs based on single-view depth camera images. It enables high-precision, interpretable 3D reconstruction of the pig body using only a single depth image, significantly reducing hardware costs and deployment complexity. The method first backprojects the depth image into an initial 3D point cloud and estimates its rigid body transformation relative to the group's average pose using a coarse alignment network, achieving spatial normalization. A unified point correspondence is established by combining a pre-constructed semantically aligned point cloud dataset, ensuring anatomical consistency. The average shape point cloud data B and principal component deformation feature vector V are learned from the aligned data, and an end-to-end deep learning network model Net is trained to regress a low-dimensional feature vector C from the normalized point cloud. A complete, occluded 3D model of the pig body is reconstructed based on the average shape point cloud data B, deformation feature vector V, and feature vector C. The entire process eliminates the need for multi-view acquisition, manual annotation, or complex optimization iterations, offering advantages such as self-supervision, computational efficiency, and strong generalization ability. The reconstructed results not only boast high geometric fidelity, but also allow the feature vector C to be directly correlated with key phenotypic indicators, supporting subsequent applications such as intelligent breeding, weight estimation, and health monitoring. This system is suitable for automated phenotypic acquisition in actual farming environments, providing a low-cost, highly robust 3D vision solution for the digital and intelligent transformation of animal husbandry.

[0020] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0021] Figure 1 This application provides an architecture diagram of a method for 3D modeling of pigs based on single-view depth camera images, as provided in an embodiment of the present application. Figure 2 This application provides a schematic flowchart of a method for semantic and spatial alignment of 3D point cloud data. Figure 3 This is a schematic diagram illustrating a method for constructing a coarsely aligned network model, as provided in an embodiment of this application. Detailed Implementation

[0022] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] The method for 3D modeling of pigs based on single-view depth camera images provided in this application embodiment can be applied to a system for 3D modeling of pigs based on single-view depth camera images. The system includes: a processing module and a reconstruction module. The processing module is used to acquire 3D point cloud data of multiple sets of pig samples to obtain several 3D point cloud datasets; the 3D point cloud data of each sample in the 3D point cloud dataset is semantically aligned and spatially aligned to obtain aligned 3D point cloud datasets. Based on the aligned 3D point cloud dataset, the average shape point cloud data B and the deformation feature vector V are analyzed; the average shape point cloud data B represents the benchmark point cloud model of the typical three-dimensional morphology of the pig population. The aligned 3D point cloud dataset is input into the deep learning network Net model, and the output feature vector C is generated. The reconstruction module is used to reconstruct a complete 3D point cloud data model M of a pig according to the formula M=B+C@V.

[0024] To address the technical problem in existing technologies where differences in pose, scale, and anatomical structure correspondence among different 3D point cloud data samples prevent accurate group statistical modeling and individual morphological comparison, this application provides a method for 3D modeling of pigs based on single-view depth camera images. The method includes: acquiring 3D point cloud data from multiple groups of pig samples to obtain several 3D point cloud datasets; performing semantic and spatial alignment on the 3D point cloud data of each sample in the 3D point cloud datasets to obtain aligned 3D point cloud datasets; analyzing the average shape point cloud data B and deformation feature vector V based on the aligned 3D point cloud datasets; the average shape point cloud data B represents a benchmark point cloud model of the typical three-dimensional morphology of the pig population; the deformation feature vector V represents the projection coefficient of the morphological deviation of each sample's 3D point cloud data relative to the average shape point cloud data B in the principal component space; and inputting the aligned 3D point cloud datasets into a deep learning network Net. The model outputs a feature vector C. A complete 3D point cloud data model M of the pig is reconstructed according to the formula M=B+C@V, where @ represents matrix multiplication. Based on this, the technical solution unifies the 3D point cloud data of multiple pigs through semantic and spatial alignment, constructing an average shape point cloud data B representing the typical morphology of the group. Principal component analysis is used to extract the projection coefficients of the morphological deviation of each sample relative to the average shape point cloud data B in the low-dimensional principal component space as the deformation feature vector V. Then, combined with the feature vector C output by the deep learning network Net, a complete and interpretable 3D pig body model is efficiently reconstructed using the formula (M=B+C@V). This method does not require multi-view or manual annotation, achieving end-to-end mapping from single-view depth images to accurate phenotypic features in a self-supervised manner. It combines computational efficiency, biological interpretability, and compatibility with downstream tasks, providing reliable and scalable technical support for smart breeding, health monitoring, and other smart farming applications.

[0025] like Figure 1 As shown in the embodiment of this application, a method for 3D modeling of pigs based on single-view depth camera images is provided, comprising: S201. Acquire multiple sets of depth images of pigs using a single-view depth camera. For each depth image, convert it into 3D point cloud data according to the camera intrinsic parameters to obtain several 3D point cloud datasets.

[0026] It should be noted that several independent 3D point cloud datasets are provided, each representing the 3D point cloud data of a single pig or the same pig at different times or poses. This data can be used for applications such as training deep learning models, shape analysis, and behavior monitoring. Each 3D point cloud dataset typically contains thousands to millions of points, each with its spatial location (X, Y, Z coordinates) and possible additional information such as color and reflectivity.

[0027] S202. Perform semantic alignment and spatial alignment on the 3D point cloud data of each sample in the 3D point cloud dataset to obtain an aligned 3D point cloud dataset.

[0028] S203. Based on the aligned 3D point cloud dataset, analyze the average shape point cloud data B and the deformation feature vector V.

[0029] It should be noted that the average shape point cloud data B is used as a benchmark reference for pig 3D morphology modeling; the deformation feature vector V represents the projection coefficient of the morphological deviation of each sample's 3D point cloud data relative to the average shape point cloud data B in the principal component space.

[0030] S204. Input the aligned 3D point cloud dataset into the deep learning network Net model and output the feature vector C.

[0031] S205. Reconstruct the complete 3D point cloud data model M of the pig using the formula M=B+C@V. Here, @ represents matrix multiplication.

[0032] Based on the above technical solutions, this application provides a method for 3D modeling of pigs using single-view depth camera images. In smart farming, accurate quantification of pig 3D phenotypes is crucial for achieving intelligent breeding, health assessment, and growth monitoring. However, raw 3D point cloud data suffers from problems such as disordered point order, varied poses, and severe occlusion. Direct modeling leads to blurred average morphology, distorted individual differences, and difficulty in extracting stable and interpretable body shape features. Therefore, a 3D modeling method that can decouple commonalities and variations and support efficient reconstruction and semantic analysis is urgently needed. This solution adopts an integrated "alignment-modeling-reconstruction" technical path: semantic alignment ensures that all samples have the same anatomical semantics at the same index, and spatial alignment eliminates rigid body pose interference, thereby obtaining a high-quality aligned dataset. On this basis, average shape point cloud data B is constructed as the population benchmark, and principal component analysis is used to extract deformation feature vector V, compressing high-dimensional morphological differences into low-dimensional, interpretable projection coefficients. Furthermore, a deep learning network Net regresses feature vector C from the single-view input, and the formula (M=B+C@V) is used to achieve linear reconstruction of the complete 3D model. This approach offers significant advantages: first, it eliminates the need for multi-view or manual annotation, driving training through self-supervised methods; second, the feature vector C and the deformation feature vector V possess clear geometric and biological meanings, supporting phenotypic parameter inversion; and third, it achieves efficient reconstruction with controllable structure, facilitating deployment in real-world farming scenarios. Overall, this method realizes an end-to-end mapping from raw observations to quantifiable, reconstructable, and generalizable 3D phenotypic features, providing core technological support for the digital transformation of animal husbandry.

[0033] In one possible implementation of the embodiments of this application, such as Figure 2 As shown, the above S201 can be specifically implemented through the following S301, S302, S303, S304 and S305, which are explained in detail below: The steps for semantic alignment of 3D point cloud datasets are as follows: S301. Select any one set of 3D point cloud data from multiple sets and mark it as the target point cloud data; S302. Using a pre-trained point cloud registration network model, register each group of 3D point cloud data with the target point cloud data to obtain a point correspondence index set between each group of 3D point cloud data and the target point cloud data. .

[0034] Where i is the index of the element in the index set, i = 0, 1, ..., m, and m is a positive integer; It should be noted that the i-th element in index set I This represents the index of a point in the 3D point cloud data to be aligned that has the same semantic meaning as the i-th point in the target point cloud data; for example... =98 indicates that the 98th point of the 3D point cloud data to be aligned corresponds to the 35th point of the target point cloud data.

[0035] S303. Reorder the points in all 3D point cloud data according to the index set, so that each 3D point cloud data has the same semantics at the same index position, and obtain a semantically aligned 3D point cloud dataset.

[0036] It should be noted that the original 3D point clouds often have arbitrary point order: the 100th point in point cloud A might be the "tip of the nose," while the 100th point in point cloud B might be the "left ear." Directly averaging the unaligned point clouds leads to a blurred and distorted average shape, such as nose + ear → a "ghosting" structure; Principal Component Analysis (PCA) fails because each dimension mixes different semantics. Semantic alignment establishes point correspondences, such as using registration networks or template matching, and reorders all point clouds so that all point clouds represent the same geometric / anatomical semantics at index i, such as all being the "tail root."

[0037] The steps for spatial alignment of 3D point cloud datasets are as follows: S304. Estimate the rigid body transformation parameters of the semantically aligned 3D point cloud data relative to the target point cloud data using a coarse alignment network model; wherein the rigid body transformation parameters include the 3D rotation matrix and translation vector.

[0038] S305. Align all 3D point cloud data to a unified coordinate system centered on the target point cloud data to obtain a spatially aligned 3D point cloud dataset.

[0039] It should be noted that the scanning postures of different pigs vary: some have their heads facing left, some are lying on their sides, and some are standing at a higher position. These differences are due to rigid body transformations (rotation + translation) and are unrelated to actual body shape variations. Without correction, PCA will misclassify "head facing left vs. head facing right" as important deformation patterns, contaminating true biological characteristics such as body length and backfat thickness. Spatial alignment unifies each point cloud to the same coordinate system by estimating the optimal rigid body transformation of each point cloud relative to a reference, such as the average shape or standard template.

[0040] The differences between point clouds only reflect non-rigid deformation, that is, changes in actual size, rather than shooting angle or placement.

[0041] Therefore, semantic alignment ensures "point-to-point semantic consistency," while spatial alignment ensures "overall pose consistency." Only by completing both steps simultaneously can we separate real biological body shape variations from 3D point clouds of multiple pigs, thereby supporting accurate modeling, analysis, and intelligent applications.

[0042] Based on the above technical solutions, in the intelligent analysis of 3D phenotypic features in pigs, the raw point cloud data suffers from two major interfering factors: semantic confusion in point order and inconsistency in spatial pose. The former leads to misalignment of point indices at the same anatomical location in different individuals, such as "nose tip" and "ear" being in the same 100th position, blurring the average shape and rendering PCA ineffective. The latter introduces irrelevant rigid body transformations due to differences in scanning pose (such as head tilt or lying on one's side), misclassifying non-biological variations as deformation features, severely contaminating the body shape modeling results. If these issues are not addressed, subsequent deformation feature extraction, phenotypic prediction, or model generation will lose their biological significance. This solution adopts a two-stage strategy of "semantic alignment first, then spatial alignment": a pre-trained point cloud registration network is used to establish a cross-sample point correspondence index set, and the indexes are reordered to achieve semantic consistency; then, a coarse alignment network is used to estimate rigid body transformations, placing all point clouds in the same coordinate system. This method accurately separates semantic structure and spatial pose, ensuring that subsequent statistical modeling only reflects real non-rigid deformations, effectively guaranteeing the accuracy of the average shape B and the deformation feature vector V.

[0043] One possible implementation is, such as Figure 3 As shown, S304 also includes the construction process of the coarse-aligned network model, which is implemented through the following S401 to S405: S401. Obtain the training dataset. The training dataset includes multiple pairs of labeled pig 3D point cloud data samples. Each sample pair contains a source point cloud data and a target point cloud data.

[0044] For example, for a sample pair, the source point cloud data may represent the 3D scan result of a pig in a specific pose, while the target point cloud data is the ideal or standard form in that pose.

[0045] S402. Preprocess the source point cloud data and target point cloud data in each sample pair, that is, perform centroid alignment and translation on the source point cloud and target point cloud in each pair to eliminate global position differences and make all point clouds share the same reference origin.

[0046] Specifically, the mean of the target point cloud data is calculated as the center point, and the centers of the remaining 3D point cloud data are shifted to the center point: ; Where X0 is the original 3D point cloud data, X is the translated 3D point cloud data, and mean(X0) represents the mean of X0. It should be noted that this step is to eliminate global translation bias, ensuring that all 3D point cloud data share the same reference origin. This facilitates subsequent rotation matching and avoids rotation estimation errors caused by positional offsets. Without translation, a pig on the left and a pig on the right might be misjudged as having different rotations when they are actually only in different positions.

[0047] S403. Construct a neural network architecture, which includes a shared weight key feature extraction module, a global feature aggregation module, and a rigid body transformation parameter regression module connected in sequence.

[0048] S404. Based on the training dataset, train the neural network framework end-to-end with the goal of minimizing the alignment loss function; The alignment loss function is obtained as follows: S4041. Based on rigid body transformation parameters, perform spatial transformation on the translated 3D point cloud data to obtain rotated point cloud data. S40411. Construct a first orthogonal projection camera with fixed parameters, orthogonally project the translated 3D point cloud data into single-view 3D point cloud data, and obtain the projected image SX. It should be noted that this step compresses 3D point cloud data into a 2D image format, adapting it to traditional CNN processing, while retaining the main geometric contour information for training a coarse alignment network model. This achieves a dimensionality reduction representation from 3D to 2D, reducing computational complexity.

[0049] S40412. Sample N non-coplanar point indices from the translated 3D point cloud data to form a set. Sample corresponding points from the target point cloud data according to the index set to obtain the point cloud data set. , used to establish point correspondence.

[0050] S40413. Input the projected image SX into the coarse alignment network model to obtain the 3D rotation matrix R; S40414. Rotate the translated 3D point cloud data according to the 3D rotation matrix to obtain the rotated 3D point cloud data XR: X Based on the index set, sample N points from the XR to form... .

[0051] S40415. Calculate the cosine similarity of the angle between each point in the rotated 3D point cloud data and the corresponding point in the target point cloud data, and average the cosine similarity to obtain the alignment loss function value. : ; Where N is the number of sampling points, and n is the index of the sampling point. This is the position vector of the nth point in the basic point cloud data. This is the position vector of the corresponding point in the rotated point cloud data.

[0052] It should be noted that the alignment loss function is used to optimize the degree of overlap between the source point cloud data and the target point cloud data after the prediction transformation, guiding the coarse alignment network to learn the correct spatial alignment ability.

[0053] S405. After training, fix the network parameters to obtain a coarse alignment network model for estimating arbitrary pig 3D point cloud data relative to target point cloud data.

[0054] It should be noted that fixing network parameters means that after the model training is completed, the values ​​of all learnable parameters in the neural network, such as convolutional kernel weights, fully connected layer weights, biases, etc., are no longer updated, so that they remain constant, thereby changing the model from the "training state" to the "inference state".

[0055] For example, suppose we have a new 3D point cloud dataset of pigs that needs spatial alignment. Using a pre-trained coarse alignment network model, we input this 3D point cloud dataset, and after steps such as feature extraction, global feature aggregation, and rigid body transformation parameter regression, we finally output the optimal rigid body transformation parameters of this point cloud dataset relative to the target point cloud dataset. Applying these parameters to perform a spatial transformation on this point cloud dataset achieves spatial alignment with the target point cloud dataset.

[0056] Based on the above technical solutions, in 3D phenotypic analysis of pigs, different individuals or the same pig often exhibit arbitrary postures in different scans, such as head tilted to the left or lying on its side, resulting in significant spatial pose differences in point cloud data. If directly used for modeling, these rigid body transformations will severely interfere with the recognition of true body shape variations, such as body length and backfat thickness, blurring the average shape, distorting deformation features, and even causing deep learning models to fail. Therefore, it is necessary to eliminate the influence of posture and achieve spatial alignment. This solution adopts a technical path of "coarse alignment network + orthogonal projection + cosine angle loss": translation is eliminated by centroid normalization; the 3D point cloud is orthogonally projected into a 2D image to adapt to an efficient CNN architecture; the rotation matrix is ​​predicted end-to-end using a neural network; and the consistency of the orientation of corresponding points (mean cosine angle loss) is used as the optimization objective. This method avoids the problems of sensitivity to initial values ​​and slow convergence of traditional ICP algorithms, and can guide reasonable rotation without precise point correspondence. This scheme is computationally efficient, robust, and can be trained end-to-end. It can also effectively decouple rigid body posture from non-rigid deformation, laying a reliable geometric foundation for subsequent semantic alignment, statistical modeling, and phenotypic feature extraction.

[0057] In one possible implementation of this application embodiment, the above-mentioned S203 can be specifically described as follows: The method for obtaining the average shape point cloud data B is as follows: The aligned 3D point cloud dataset is constructed as a matrix Amr of {K,M}; where K is the number of 3D point cloud data samples and M is the coordinate dimension of each sample after flattening. It should be noted that each 3D point cloud data sample has M dimensions. If each point contains x, y, and z, then there are N points in total, M = 3N. Each 3D point cloud data sample is flattened into a row vector and stacked into K... M matrix.

[0058] Calculate the column mean of matrix AMR across the K sample dimensions. That is, for each spatial location, take the mean of its coordinates across all K samples to obtain a mean vector of length M, which is labeled as the average shape point cloud data B.

[0059] The method for obtaining the deformation eigenvector V is as follows: Calculate the difference between each 3D point cloud data in the aligned 3D point cloud dataset and the average shape point cloud data B to obtain a decentralized deviation point cloud matrix; perform principal component analysis on the deviation point cloud matrix to extract the eigenvectors corresponding to the top Z largest eigenvalues ​​and form the principal component matrix; perform matrix multiplication between the deviation point cloud matrix and the principal component matrix to obtain the deformation feature matrix; each row of the deformation feature matrix is ​​the deformation feature vector V of a sample.

[0060] Based on the above technical solutions, in the intelligent analysis of 3D phenotypic characteristics in pigs, although the original point cloud data has been semantically and spatially aligned, it still contains complex individual morphological differences. Directly using it for modeling would result in high dimensionality, redundancy, and difficulty in interpretation. Without extracting commonalities and variational structures, deep learning models are difficult to generalize, and body shape assessment lacks biological basis. Therefore, there is an urgent need for a mathematical representation method that can decouple group commonalities from individual differences. This solution adopts a statistical modeling paradigm of "average shape + principal component deformation": by aligning the point cloud to construct a matrix and calculating the column mean, the average shape point cloud data is obtained, accurately depicting the typical geometric structure of the group; principal component analysis (PCA) is used to extract the dominant deformation pattern (principal component) from the decentralized bias, and each sample is projected onto this low-dimensional subspace to obtain the deformation feature vector. This method has significant advantages: First, it is highly efficient in dimensionality reduction, compressing tens of thousands of dimensional point clouds into tens of dimensional interpretable features; second, it has clear semantics, with each deformation feature vector V corresponding to a specific biological trait (such as body length and back height); third, it is controllable in reconstruction, supporting accurate reconstruction of an individual's 3D morphology from the average shape point cloud data B and the deformation feature vector V; and fourth, it has strong compatibility, with the deformation feature vector V serving as a supervisory signal to train deep networks, enabling end-to-end prediction from single-view images to phenotypic parameters.

[0061] In one possible implementation of this application embodiment, the above-mentioned S204 can be specifically described as follows: The aligned 3D point cloud dataset is input into a pre-built and trained deep learning network Net model, which includes a convolutional feature extraction module, a global context aggregation module, and a fully connected regression head. It should be noted that the convolutional feature extraction module is used to extract local geometric features point by point from the input 3D point cloud. It captures the spatial structure information (such as curvature, edge, surface orientation, etc.) of each point and its neighborhood through point cloud-specific convolutional operations (such as MLP or edge convolution), and converts the original coordinates into a high-dimensional local feature representation with semantic meaning.

[0062] Global Context Aggregation Module: This module is responsible for fusing the local features of all points and using symmetric aggregation functions such as max pooling and attention mechanisms to generate a global feature vector that is independent of the point order, thereby encoding the overall morphological semantics of the whole pig (such as body proportions, torso outline, etc.) and realizing context perception from local to global.

[0063] Fully connected regression head: This module consists of several fully connected layers that map the global feature vector into a low-dimensional feature vector C, which is used to characterize the key body shape or morphological semantic features of the pig from the current perspective. The feature vector C can be used as a deformation coefficient in 3D reconstruction, or it can be directly used for downstream tasks such as phenotypic prediction or individual identification.

[0064] The construction of the deep learning network Net model is as follows: Several aligned 3D point cloud data and corresponding feature vectors C are obtained from historical data, and training and validation datasets are constructed. The deep learning network Net is trained end-to-end with the optimization objective of minimizing the absolute value of the distance between the reconstructed 3D point cloud data and the 3D point cloud data of the original input model. The distance parameter is minimized by a ladder iterative minimization algorithm. After training, a deep learning network model Net is obtained, which is used to extract feature vector C from pig depth images.

[0065] The 3D point cloud data is forward inferred by the deep learning network Net model, and the output feature vector C is used to represent the morphological or body shape semantic features of the pig from the current viewpoint.

[0066] Based on the above technical solutions, in intelligent farming and phenomics, traditional manual measurement of pig body size is inefficient and highly subjective, while direct analysis using raw 3D point clouds faces problems such as high dimensionality, redundancy, limited viewpoints, and occlusion, making it difficult to achieve automated, quantifiable, and accurate phenotypic extraction. Therefore, there is an urgent need for a method that can automatically learn and output biologically meaningful low-dimensional morphological features from single-view observations. This solution proposes an end-to-end feature extraction framework based on the deep learning network Net: it uses convolutional modules to capture local geometric details and models the overall morphological semantics through global aggregation, outputting a low-dimensional feature vector C from the regression head. During training, the reprojection error (e.g., absolute distance) of the reconstructed point cloud under the observation viewpoint is used as a self-supervised signal, eliminating the need for manual annotation of deformation parameters. The network can learn the true body structure simply by aligning the point cloud with the raw depth data. This technical solution has significant advantages: First, it eliminates the reliance on multi-view or complete 3D annotation, making it suitable for actual aquaculture scenarios; second, the feature vector C has clear geometric interpretability and can be directly used to predict key phenotypic indicators such as body length and backfat thickness; third, the model structure conforms to the disorder of point clouds and the local-global perception law, has strong generalization ability, and achieves efficient mapping from single-view depth images to quantifiable, reconstructable, and interpretable 3D phenotypic features.

[0067] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.

Claims

1. A method for 3D modeling of pigs based on single-view depth camera images, characterized in that, include: Acquire 3D point cloud data of multiple groups of pig samples to obtain several 3D point cloud datasets; Semantic alignment and spatial alignment are performed on the 3D point cloud data of each sample in the 3D point cloud dataset to obtain an aligned 3D point cloud dataset. Based on the aligned 3D point cloud dataset, the average shape point cloud data B and the deformation feature vector V are analyzed; the average shape point cloud data B represents the benchmark point cloud model of the typical three-dimensional morphology of the pig population; the deformation feature vector V represents the projection coefficient of the morphological deviation of each sample's 3D point cloud data relative to the average shape point cloud data B in the principal component space. The aligned 3D point cloud dataset is input into the deep learning network Net model, and the output feature vector C is generated. The complete 3D point cloud data model M of the pig is reconstructed according to the formula M=B+C@V; where @ represents matrix multiplication.

2. The method for 3D modeling of pigs based on single-view depth camera images according to claim 1, characterized in that, The semantic alignment method for the 3D point cloud data of each sample includes: Select any set of 3D point cloud data from the 3D point cloud dataset and label it as the target point cloud data; use a pre-trained point cloud registration network model to register each set of 3D point cloud data in the 3D point cloud dataset with the target point cloud data respectively, to obtain a point correspondence index set between each set of 3D point cloud data and the target point cloud data. Where i is the index of the element in the index set, i = 0, 1, ..., m, and m is a positive integer; the i-th element in the index set I. This represents the index of the point in the 3D point cloud data to be aligned that has the same semantics as the i-th point in the target point cloud data; The points in each group of 3D point cloud data are reordered according to the index set, so that each 3D point cloud data has the same semantics at the same index position, resulting in a semantically aligned 3D point cloud dataset.

3. The method for 3D modeling of pigs based on single-view depth camera images according to claim 1, characterized in that, The spatial alignment method for the 3D point cloud data of each sample includes: The rigid transformation parameters of semantically aligned 3D point cloud data relative to the target point cloud data are estimated by a coarse alignment network model. All 3D point cloud data are aligned to a unified coordinate system centered on the target point cloud data to obtain a spatially aligned 3D point cloud dataset. The rigid transformation parameters include 3D rotation matrices and translation vectors.

4. The method for 3D modeling of pigs based on single-view depth camera images according to claim 3, characterized in that, The construction of the coarse-aligned network model includes: Obtain a training dataset, which includes multiple pairs of labeled 3D point cloud data samples, each pair containing a source point cloud data and a target point cloud data. Preprocess the source point cloud data and target point cloud data in each sample pair; A neural network architecture is constructed, comprising a shared weighted key feature extraction module, a global feature aggregation module, and a rigid body transformation parameter regression module connected in sequence. Based on the training dataset, the neural network framework is trained end-to-end with the goal of minimizing the alignment loss function. The alignment loss function is used to optimize the degree of overlap between the source point cloud data and the target point cloud data after prediction transformation, and guide the coarse alignment network to learn the correct spatial alignment ability. After training, a coarse alignment network model is obtained for estimating arbitrary pig 3D point cloud data relative to target point cloud data.

5. The method for 3D modeling of pigs based on single-view depth camera images according to claim 4, characterized in that, The construction of the alignment loss function includes: Based on the rigid body transformation parameters, the preprocessed 3D point cloud data is spatially transformed to obtain rotated point cloud data. Calculate the cosine similarity of the angle between each point in the rotated point cloud data and the corresponding point in the target point cloud data, and average the cosine similarity to obtain the alignment loss function value. : ； Where N is the number of sampling points, and n is the index of the sampling point. Let n be the position vector of the nth point in the target point cloud data. Let be the position vector of the nth point in the rotated point cloud data.

6. The method for 3D modeling of pigs based on single-view depth camera images according to claim 1, characterized in that, The method for obtaining the average shape point cloud data B includes: The aligned 3D point cloud dataset is constructed as a matrix Amr of {K,M}, where K is the number of 3D point cloud data samples and M is the coordinate dimension of each sample after flattening. The column mean of the matrix Amr is calculated in the K sample dimensions to obtain the mean vector of length M, which is labeled as the average shape point cloud data B.

7. The method for 3D modeling of pigs based on single-view depth camera images according to claim 1, characterized in that, The method for obtaining the deformation feature vector V includes: The difference between each 3D point cloud data in the aligned 3D point cloud dataset and the average shape point cloud data B is calculated to obtain a decentralized deviation point cloud matrix; principal component analysis is performed on the deviation point cloud matrix to extract the eigenvectors corresponding to the top Z largest eigenvalues, forming a principal component matrix; the deviation point cloud matrix and the principal component matrix are multiplied to obtain a deformation feature matrix; each row of the deformation feature matrix is ​​a deformation feature vector V of a sample.

8. The method for 3D modeling of pigs based on single-view depth camera images according to claim 1, characterized in that, The method for obtaining the feature vector C includes: The aligned 3D point cloud dataset is input into a pre-built and trained deep learning network Net model, which includes a convolutional feature extraction module, a global context aggregation module, and a fully connected regression head, to predict low-dimensional semantic features related to pig body size. The deep learning network Net model performs forward inference on the 3D point cloud data and outputs a feature vector C, which is used to characterize the morphological or body shape semantic features of the pig from the current perspective.

9. A method for 3D modeling of pigs based on single-view depth camera images according to claim 8, characterized in that, The construction of the deep learning network Net model includes: Several aligned 3D point cloud data and corresponding feature vectors C are obtained from historical data and integrated into training and validation datasets. The deep learning network Net is trained end-to-end with the optimization objective of minimizing the absolute value of the distance between the reconstructed 3D point cloud data and the 3D point cloud data of the original input model. The distance parameter is minimized by a ladder iterative minimization algorithm. After training, a deep learning network model Net is obtained, which is used to extract feature vector C from pig depth images.

10. A system for 3D modeling of pigs based on single-view depth camera images, operating based on the method for 3D modeling of pigs based on single-view depth camera images as described in any one of claims 1-9, characterized in that, Includes a processing module and a reconstruction module; The processing module is used to acquire 3D point cloud data of multiple groups of pig samples to obtain several 3D point cloud datasets; and to perform semantic alignment and spatial alignment on the 3D point cloud data of each sample in the 3D point cloud dataset to obtain aligned 3D point cloud datasets. Based on an aligned 3D point cloud dataset, we analyze the average shape point cloud data B and the deformation feature vector V. The average shape point cloud data B represents a benchmark point cloud model of the typical three-dimensional morphology of the pig population. The aligned 3D point cloud dataset is input into the deep learning network Net model, and the output feature vector C is generated. The reconstruction module is used to reconstruct a complete 3D point cloud data model M of a pig according to the formula M=B+C@V.

Images