Pig three-dimensional point cloud completion method and system for complex environment

Abstract

The application discloses a pig three-dimensional point cloud completion method and system for complex environment, and on the basis of an AdaPoinTr architecture, a frequency decoupling Mamba-Transformer hybrid module is designed for the first half of an encoder: firstly, farthest point sampling dimension reduction is performed on features, and three-dimensional Morton bit interleaving sorting is adopted to sort sampling center points to retain pig curved surface spatial continuity; then, low-frequency global structure is extracted through bidirectional Mamba scanning, interpolation recovery is performed to the original resolution, and high-frequency local details are obtained by subtracting the original features; scaling and offset parameters of high-frequency features are modulated through FiLM under the condition of low-frequency features, high-frequency and low-frequency features are adaptively fused through a gating mechanism, and the point cloud completion over-smoothing problem is relieved; the second half of the encoder retains the Transformer structure to finely depict local geometry. The application adopts a pre-training and pig data fine-tuning two-stage transfer learning strategy, adapts to a complex noise scene of a pig farm, and can accurately recover three-dimensional point clouds of missing pig bodies, limbs and other parts.

Description

Technical Field

[0001] This invention relates to 3D point cloud processing and smart agriculture technology, specifically to a method and system for completing 3D point clouds of pigs in complex environments, which is particularly suitable for large-area defective point cloud data caused by movement and occlusion in agricultural scenarios. Background Technology

[0002] Point clouds, as a data representation in three-dimensional space, can accurately describe the geometric structure and surface details of objects, playing a crucial role in numerous practical applications such as autonomous driving, 3D reconstruction, industrial inspection, and smart agriculture. However, due to the inherent disorder, sparsity, and irregularity of 3D point cloud data, the analysis and processing of point clouds are extremely challenging. In actual acquisition processes, limitations imposed by sensor accuracy, object surface characteristics, and factors such as occlusion and lighting in complex environments often result in point cloud data that is sparse, incomplete, and fragmented, failing to accurately reflect the complete structural information of the target object and thus posing significant difficulties for subsequent analysis and processing. For example, non-contact livestock body size measurement has become an active research direction in smart agriculture, but traditional measurement methods rely on manual tools such as measuring tapes and rulers. These methods are labor-intensive, time-consuming, and often stressful for animals, resulting in incomplete point clouds. Occlusion, animal movement, and limited camera perspective lead to missing regions in the point cloud, directly affecting the accuracy of key point localization and body size measurement.

[0003] In recent years, significant progress has been made in deep learning-based 3D point cloud completion methods. Point cloud completion aims to reconstruct complete 3D geometry from incomplete local point clouds, holding significant value in occluded scene understanding and 3D object reconstruction. Early deep learning-based methods, such as 3D-EPN, obtained completion results by voxelizing incomplete shapes and using 3D convolutional neural networks to predict complete occupancy on a voxel mesh. While these methods offer intuitive topological representation, the cost of memory and computation increases cubically with voxel resolution, making it difficult to recover fine geometric details at high resolutions. Another type of multi-view method, such as 3D-R2N2, renders the 3D shape as a depth map or multi-view image, completes the completion in the 2D domain, and then maps it back to 3D space. However, this type of method is limited by the complexity of viewpoint selection and multi-view fusion, and suffers from insufficient modeling of geometric detail consistency.

[0004] With the introduction of deep network architectures such as PointNet and PointNet++, which can directly process raw point cloud data, the field of point cloud completion has seen a significant breakthrough. Yuan et al.'s PCN was the first to introduce an encoder-decoder framework into point cloud completion tasks, achieving coarse-grained reconstruction of missing regions. Subsequently, methods such as FoldingNet and TopNet further improved the geometric regularity of completed point clouds by refining the decoder structure. Methods such as PF-Net and GRNet further introduced multi-scale feature extraction and 3D voxel convolution, enhancing the ability to generate local details. However, the aforementioned encoder-decoder architectures are inherently limited by the local receptive field of convolution operations, making it difficult to effectively capture non-local long-distance geometric dependencies in point clouds, and their computational efficiency is limited when processing large-scale unordered point sets.

[0005] Following the tremendous success of the Transformer architecture in natural language processing and 2D image processing, its powerful global attention mechanism has been rapidly adopted for 3D point cloud processing. Transformer-based methods, such as PoinTr, AdaPoinTr, SnowFlakeNet, PMP-Net++, SeedFormer, AnchorFormer, ProxyFormer, LAKe-Net, FB-Net, SVDFormer, and GeoFormer, have significantly enhanced their ability to jointly model global geometry and local details by introducing attention mechanisms, achieving excellent performance on public datasets. Methods like PoinTr utilize farthest point sampling and K-nearest neighbor extraction to extract local features, and leverage the Transformer encoder to predict missing point proxies, achieving high-precision completion. However, while the Transformer's self-attention mechanism effectively captures long-distance dependencies, its computational complexity is proportional to the square of the input sequence length, exhibiting quadratic computational complexity. This creates a significant efficiency bottleneck when processing high-resolution point clouds or large-scale scenes, limiting its application in resource-constrained or real-time-critical scenarios.

[0006] Recently proposed Sequence State Space (SSM) models, such as Mamba, offer an efficient paradigm for long sequence modeling that differs from self-attention. Mamba models sequences of arbitrary length in near-linear time complexity through learnable state space recursion, balancing strong global dependency modeling capabilities with good computational efficiency. In point cloud and 3D vision, works such as PointMamba, PointCloudMamba, and Mamba3D have attempted to introduce Mamba into point cloud analysis tasks. These methods transform point clouds into one-dimensional sequences using spatial filling curves or consistent traversal serialization, and utilize the Mamba encoder for global modeling, maintaining competitive performance while reducing computational complexity. In point cloud completion, frameworks such as 3DMambaComplete have attempted to use Mamba, but their feature extraction designs have not fully leveraged Mamba's advantages, and there is still room for improvement in completion accuracy under large-scale inputs. Existing methods in point cloud completion often tend towards a single architecture, failing to fully explore the potential of the collaborative work between Transformer and Mamba, and still exhibiting significant limitations and shortcomings, as follows:

[0007] First, the completion accuracy is insufficient for large-area, highly incomplete data. In agricultural scenarios, point clouds of targets such as pigs (e.g., live pigs) often have large areas of missing data due to motion, occlusion, and viewpoint limitations, with a degree of incompleteness far exceeding that of general datasets (such as ShapeNet and Completion3D). Existing methods based on a single Transformer or Mamba architecture struggle to effectively utilize limited available geometric information to recover the complete global shape structure when faced with such extreme incompleteness, resulting in a significant decrease in completion accuracy.

[0008] Second, there is insufficient utilization of prior knowledge regarding the biological morphology of specific targets. Targets such as pigs have unique biological morphological constraints, and their body shape, posture, etc., have clear prior knowledge. General point cloud completion models lack specialized modeling for this kind of prior knowledge, making it difficult to effectively restore the target's unique local details and overall shape during the completion process. This results in completion results that are geometrically unreasonable or do not conform to the target's biological characteristics.

[0009] Third, training data is scarce in real-world scenarios. Under realistic data collection conditions, it is difficult to obtain complete ground truth point clouds of targets that are precisely paired with incomplete point clouds, resulting in an extreme scarcity of paired training data. Existing methods heavily rely on training paradigms based on large-scale paired datasets, and under conditions of limited data, the generalization ability and completion performance of the models are severely constrained.

[0010] Fourth, the integration of Transformer and Mamba has limitations. Although some studies have attempted to integrate Transformer and Mamba, such as completely replacing self-attention and replacing the encoder entirely with Mamba parallel insertion, this integration method has two shortcomings: First, there is an inherent contradiction between the disorder of point clouds and Mamba's modeling method that relies on sequence order. Although Transformer can use the self-attention mechanism to establish global relationships between points to overcome disorder, Mamba uses a state-space recursive approach to model sequence information, emphasizing the continuous propagation of sequence context, which makes it easy for local geometric information to be lost during sequence propagation. Second, existing integration strategies simply parallelize or stack the two, failing to achieve deep coupling and complementary advantages between them, making it difficult to effectively control computational overhead while ensuring the quality of completion. Summary of the Invention

[0011] Purpose of the Invention: The purpose of this invention is to address the shortcomings of existing technologies and provide a method and system for completing 3D point clouds of pigs in complex environments. By employing 3D Morton code spatial sorting, bidirectional Mamba scanning, frequency decomposition and FiLM gating fusion mechanism, and a progressive injection strategy, this invention alleviates the over-smoothing problem in the point cloud completion process while preserving global structural modeling capabilities. Furthermore, it achieves efficient transfer of general point cloud completion capabilities to specific pig domains through a two-stage transfer learning strategy and can also be applied to the morphological reconstruction of other live animal targets.

[0012] Technical solution: The present invention provides a method for completing 3D point clouds of pigs in complex environments, comprising the following steps:

[0013] Step 1: Obtain the original cloud data of the defects in the pigs that need to be supplemented;

[0014] Step 2: Process the original incomplete cloud data using the DGCNN grouping module to obtain aggregated features;

[0015] The DGCNN grouping module here adopts the feature extraction structure of AdaPoinTr. It extracts the local geometric features of the pig point cloud layer by layer by combining local K-nearest neighbor graph convolution with progressive farthest point sampling, providing basic feature support for subsequent encoding and decoding to reconstruct the missing parts.

[0016] Step 3: Normalize the aggregated features. First, normalize the three-dimensional coordinates of the points and perform 10-bit integer quantization. Then, generate Morton codes by interleaving along the XYZ axes. Globally sort the feature sequence according to the Morton codes to obtain one-dimensional serialized features with local continuity in three-dimensional space. ;

[0017] Step 4: Construct a Transformer-Mamba hybrid coding network to serialize the one-dimensional features of the pig point cloud. Inputting the trained Transformer-Mamba hybrid coding network yields an enhanced feature sequence that integrates global structure and local geometric details. The Transformer-Mamba hybrid coding network is based on an improved AdaPoinTr architecture, combining the Mamba selective state-space mechanism with the Transformer local feature modeling mechanism. It incorporates a progressive Laplace-Mamba injection strategy, a bidirectional scanning mechanism, and a frequency decoupling module combined with FiLM modulation. Specifically, it includes the following sub-steps:

[0018] Step 4.1, for Perform farthest point sampling for dimensionality reduction to obtain a set of low-resolution center points and their corresponding features; sort the low-resolution center point set by Morton bit interleaving according to three-dimensional coordinates; set up two independent Mamba instances, and perform forward and reverse scanning on the sorted low-resolution center point feature sequence respectively; after concatenating the outputs of the forward and reverse scanning in the channel dimension, restore the original feature dimension through linear projection to obtain low-resolution enhanced features.

[0019] Step 4.2: Restore the low-resolution enhanced features to the original point cloud resolution using nearest neighbor distance weighted interpolation to obtain the low-frequency component representing the overall outline and pose of the pig. Subtract the low-frequency component residual from the input features obtained in step 4.1 to obtain the high-frequency component that characterizes local anatomical details. ; with low-frequency components As a condition, scaling and offset factors are generated through a two-layer MLP to optimize high-frequency components. Affine modulation and normalization are performed to obtain the high-frequency modulation characteristics. ; low frequency components With modulation high-frequency characteristics After concatenation, a linear layer generates gated weights, which are then used to adjust the weights. and Weighted fusion and residual connection are performed to output frequency-coordinated enhanced features;

[0020] Step 4.3: Execute the progressive Laplace-Mamba injection strategy, insert the Laplace-MambaMixer module only in the first half of the encoder layer, and retain the original Transformer self-attention structure in the second half of the encoder layer. The layered collaborative modeling of global structure and local geometric details outputs a refined enhanced feature sequence.

[0021] Step 5: Based on the cross-attention decoding mechanism, decode and reconstruct the refined enhanced feature sequence to generate a complete pig prediction point cloud.

[0022] Furthermore, the specific processing method for obtaining aggregated features in step 2 is as follows: construct a local graph based on K nearest neighbors and extract initial local geometric features, perform dimensionality reduction step by step through two farthest point samplings, and repeatedly perform graph convolution feature aggregation based on K nearest neighbors at the center point after each dimensionality reduction, gradually expanding the feature channel dimension, and outputting the coordinates of the downsampled center point and its corresponding aggregated features.

[0023] Furthermore, the detailed method for the normalization process in step 3 is as follows: First, perform minimum boundary and maximum boundary normalization on the three-dimensional coordinates of the pig defect cloud;

[0024] Then, the normalized 3D coordinates are quantized to 10-bit integer order; the quantized 3D integer coordinates are interleaved along the XYZ axes to generate Morton codes corresponding to each point cloud point.

[0025] Next, the aggregated feature sequence is globally sorted according to the Morton code to obtain a normalized one-dimensional serialized feature with continuity in three-dimensional surface space. Since the point coordinates generated by the DGCNN grouping module remain unchanged throughout the encoder processing, the Morton sort index results calculated independently by each layer of the encoder are consistent.

[0026] Furthermore, in the Transformer-Mamba hybrid coding network base, a Mamba selective state-space mechanism is constructed based on the Discrete State-Space Model (SSM). The SSM discrete formula is as follows:

[0027] ;

[0028] ;

[0029] , Let be the hidden states at time t and time t-1, respectively. Let be the discrete input at time t. Let be the discrete output at time t. , and This is the discrete parameter matrix obtained by zero-order preserved discretization of continuous parameters. and The following conditions must be met:

[0030] ;

[0031] ;

[0032] Let A be the time step parameter, B be continuous parameter matrices, and I be the identity matrix.

[0033] To achieve parallel training, the SSM recursive form is equivalently transformed into a global convolutional form:

[0034] ;

[0035] ;

[0036] This represents the global convolution kernel generated from the state-space parameters. Given the input sequence, The output sequence is represented by *, which indicates a convolution operation. The sequence length;

[0037] The Mamba selective state-space mechanism designs some state parameters as input-dependent functions, satisfying the following equation:

[0038] ;

[0039] Mamba employs the SelectiveScan algorithm to achieve parallel computation of state recursion, maintaining a linear time complexity of O(N). Compared to the existing Transformer's O(N²) complexity, it significantly improves the efficiency and scalability of long sequence point cloud modeling.

[0040] Furthermore, the execution process of the bidirectional scanning mechanism in step 4.1 is as follows:

[0041] Two independent Mamba instances are set up to perform forward and reverse scans on the low-resolution center point feature sequences obtained by sampling and dimensionality reduction of the farthest point and sorting by Morton. The forward scan captures the preceding context information of the sequence, and the reverse scan supplements the following context information missing in the forward scan.

[0042] Then, the output sequences of the forward and reverse scans are concatenated along the channel dimension and mapped back to the original feature dimension C via linear projection, satisfying:

[0043] ;

[0044] The feature sequence after Morton sorting; This is a sequence reversal operation; To perform a forward scan of the Mamba instance, capture the sequence preorder context; To perform a reverse scan of the Mamba instance, process the reverse sequence to supplement the post-sequence context; The concatenation is done along the channel dimension, resulting in a 2C feature dimension. The linear projection layer maps 2C-dimensional features back to the original dimension C; O represents the output features of the bidirectional scanning module.

[0045] LayerNorm normalization is first applied to the input features, and residual connections are used for scanning and projection operations. Random depth regularization is introduced during the training phase. Since the point cloud coordinates remain unchanged during the encoding process, the Morton sorting index results calculated independently by each encoding layer are consistent. Morton sorting adopts three-dimensional XYZ axis positional interleaving encoding, avoiding the defect of two-dimensional Z-curve and Hilbert curve sorting that destroys the spatial proximity of three-dimensional point clouds, and accurately preserves the continuity of the three-dimensional curved surface space of pigs.

[0046] Furthermore, the detailed process of obtaining the frequency co-enhancement features in step 4.2 is as follows:

[0047] First, the input point cloud features (Specifically referring to one-dimensional serialization features) Sparse point set obtained by sampling FPS at the farthest point and corresponding sparse features N is the number of points in the point cloud, and C is the original feature dimension;

[0048] ;

[0049] ;

[0050] r is the downsampling factor;

[0051] Morton bit interleaving sorting is performed according to three-dimensional coordinates, and low-resolution enhanced features are obtained by bidirectional Mamba scanning as described in claim 4; the low-resolution enhanced features are restored to the original number of points N by nearest neighbor distance weighted interpolation to obtain low-frequency components;

[0052] High-frequency components are extracted using residual subtraction. : ;

[0053] The frequency decoupling operation described above is performed in the complete C-dimensional feature space, explicitly separating high and low frequency features to avoid the loss of local details; it solves the problem of excessive smoothing in point cloud completion from the root cause.

[0054] Next, FiLM cross-frequency modulation and fusion are used, the specific process of which includes:

[0055] Low frequency components Input two-layer MLP to generate modulation parameter scaling factor offset factor ,Right now ;

[0056] Subsequently, modulation parameters were used to modulate the high-frequency components. Element-wise affine transformation and normalization yield the modulation high-frequency characteristics. : ;

[0057] For element-wise multiplication, For layer normalization;

[0058] modulated high-frequency features Compared with the original low-frequency characteristics The weights are concatenated and then processed through a linear layer to generate a gate.

[0059] ;

[0060] It is the Sigmoid activation function. For linear projection layers;

[0061] The original feature dimensions are restored through linear mapping, resulting in the feature fusion outcome. :

[0062] ;

[0063] Feature fusion results With input point cloud features Perform residual connections to obtain the final output features of the module. :

[0064] ;

[0065] By adaptively guiding high-frequency local detail features with low-frequency global features, the information conflict problem of high- and low-frequency feature fusion is resolved, and the overall morphological consistency of pig point cloud and the integrity of local edge details are ensured simultaneously.

[0066] Furthermore, the details of step 4.3, the progressive Laplace-Mamba injection strategy, are as follows:

[0067] Let L be the total number of encoder layers, then the number of layers injected into the Laplace-Mamba module is... : ;

[0068] This is a floor function;

[0069] No. The feature update rule for each coding layer is:

[0070] ;

[0071] For the first Layer output features; For Laplace-MambaMixer module operations, For Transformer coded block operations;

[0072] Only before Layered parallel embedding of Mamba modules and Transformer self-attention modules, then The layer retains only the Transformer structure; through a layered design of shallow global modeling and deep detail optimization, the error cascading problem of serial pipeline coding is avoided, and efficient collaboration between global structural modeling and local geometric representation is achieved.

[0073] Furthermore, a two-stage transfer learning training strategy is implemented on the Transformer-Mamba hybrid encoding network.

[0074] The first stage involves general pre-training, selecting a public 3D point cloud benchmark dataset as training samples, and fully training the Transformer-Mamba hybrid coding network to enable the network to learn the basic representation capabilities of the 3D geometric structure, surface morphology, and edge details of general objects, and to complete the initialization of the network's basic parameters.

[0075] The second stage involves domain fine-tuning. This involves collecting depth images of pigs in real pig farming scenarios, generating defective point cloud data through coordinate transformation, and constructing a fine-tuning dataset. Based on the fine-tuning dataset, the pre-trained network is iteratively fine-tuned to adapt to specific biological geometric features, trunk surface morphology, and noise distribution characteristics of the real acquisition environment, overcoming the problem of scarce labeled point cloud data for pigs in agricultural scenarios.

[0076] Furthermore, the network training employs a composite loss function, which is obtained by weighted fusion of the fine point cloud chamfer distance loss, the center point chamfer distance loss, and the denoising branch chamfer distance loss.

[0077] The formula for chamfer distance CD loss is:

[0078] ;

[0079] P represents the predicted point cloud set, Q represents the true point cloud set, and N and M represent the number of points in the predicted and true point clouds, respectively. The distance is the Euclidean distance with a 2-norm.

[0080] Since the decoding module adopts an auxiliary denoising mechanism to enhance the feature prediction capability of the decoder, the denoised query point and the center point are generated synchronously to obtain a complete point cloud based on the denoised query point; loss optimization is applied to this part of the point cloud to ensure that the network can still obtain effective optimization through high-quality input points in the early stage of training when the center point prediction is not accurate.

[0081] For predicting the bijective mapping function from the point cloud to the real point cloud;

[0082] The final composite loss function is: ;

[0083] To balance the weight hyperparameters, the optimal values ​​were determined through ablation experiments. By employing a layered design of shallow global modeling and deep detail optimization, the error cascading problem of serial pipeline coding was avoided, achieving efficient collaboration between global structural modeling and local geometric representation.

[0084] Furthermore, the progressive completion process of the 3D point cloud is visualized by using Mitsuba and Open3D tools to perform 3D visualization rendering on the completed 3D point cloud of the pig, intuitively displaying the completion effect in different areas.

[0085] This invention also discloses a three-dimensional point cloud completion system for pigs in complex environments, including a point cloud feature extraction module, a point cloud normalization module, a hybrid encoding module, a cross-attention decoding and reconstruction module, and a training optimization module;

[0086] The point cloud feature extraction module has a built-in DGCNN grouping unit that receives the point cloud of pig defects. It extracts the downsampled center point and its corresponding local geometric features layer by layer by combining local K-nearest neighbor graph convolution with progressive farthest point sampling.

[0087] The point cloud normalization module performs coordinate normalization, 10-bit quantization, 3D Morton encoding and global sorting on the weighted aggregated features, and outputs spatially continuous one-dimensional serialized features.

[0088] The hybrid coding module incorporates a progressive Laplace-Mamba injection unit, a bidirectional Mamba scanning unit, a Laplace frequency decoupling unit, and a FiLM cross-frequency modulation unit to construct a Transformer-Mamba hybrid coding network and output a refined and enhanced feature sequence.

[0089] The cross-attention decoding and reconstruction module decodes the enhanced feature sequence to generate a complete pig prediction point cloud.

[0090] The training and optimization module performs two-stage transfer learning training, iteratively optimizes network parameters based on a composite loss function, and outputs the optimal pig point cloud completion model.

[0091] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0092] 1. The Transformer-Mamba hybrid coding network of the present invention achieves complementary division of labor for features: the Mamba branch is dedicated to modeling the low-frequency global topology, while the Transformer branch focuses on extracting high-frequency local geometric details. The two work together to complete feature representation. Unlike the pseudo-bidirectional modeling achieved by the comparative patents that rely solely on Bi-SSM channel flipping, the present invention achieves true decoupling and complementarity between global and local features, resulting in higher modeling accuracy.

[0093] 2. This invention adds a LaplacePointMixer frequency decoupling module, which extracts low-frequency contours through dynamic neighborhood aggregation and separates high-frequency details through residual subtraction, and performs operations on the complete feature dimension throughout the process; it avoids the defects of local surface and edge detail loss caused by the maximum pooling operation used in the comparison patent from the root, and greatly improves the reconstruction effect of fine structures such as pig limbs and torso wrinkles.

[0094] 3. This invention uses three-dimensional XYZ bit interleaved Morton code sorting, abandoning the two-dimensional Z curve and Hilbert space filling curve used in the patent, avoiding the defect of two-dimensional sorting destroying the spatial proximity of three-dimensional point cloud, accurately preserving the spatial continuity of the three-dimensional curved surface of pigs, and making the spatial feature serialization more reasonable.

[0095] 4. This invention proposes a dedicated transfer learning strategy adapted to 3D reconstruction of pigs. In the first stage, the network is pre-trained on a public benchmark dataset to learn the basic geometric representations of general object surfaces and edges. In the second stage, the network is fine-tuned using a defect cloud dataset generated by converting depth images from real pig farms. This completes the adaptation of general geometric capabilities to the domain-specific biological features and breeding noise distribution of pigs, solving the practical problem of the scarcity of high-quality labeled point cloud data in agricultural scenarios and reducing the cost of data collection. Attached Figure Description

[0096] Figure 1 This is a schematic diagram of the overall network structure of the present invention.

[0097] Figure 2 This is a schematic diagram of bidirectional scanning in this invention.

[0098] Figure 3 This is a schematic diagram of the Laplace-MambaMixer module structure in this invention.

[0099] Figure 4 This is an example diagram of a pig defect cloud used for testing in an embodiment of the present invention.

[0100] Figure 5 For the purposes of this invention, Figure 4 The image shows a visualization of the completed incomplete cloud.

[0101] Figure 6This is a visualization of the results of this invention on the PCN dataset. Detailed Implementation

[0102] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0103] like Figure 1 and Figure 3 As shown, the present invention provides a method for completing 3D point clouds of pigs in complex environments, comprising the following steps:

[0104] Step 1: Obtain the original cloud data of the defects in the pigs that need to be supplemented;

[0105] Step 2: Process the original residual point cloud data using the DGCNN grouping module: First, perform a linear transformation on the input point cloud coordinates to obtain initial features. Construct a local graph based on K-nearest neighbors and extract local geometric features through graph convolutional layers. Then, perform two farthest point sampling dimensionality reductions on the point cloud. Repeat the graph convolutional feature extraction based on K-nearest neighbors at the center point after each dimensionality reduction, gradually expanding the feature channel dimension. Finally, output the downsampled center point coordinates and their corresponding aggregated features.

[0106] Step 3: Normalize the aggregated features. First, normalize the three-dimensional coordinates of the points and perform 10-bit integer quantization. Then, generate Morton codes by interleaving along the XYZ axes. Globally sort the feature sequence according to the Morton codes to obtain one-dimensional serialized features with local continuity in three-dimensional space. ;

[0107] Step 4: Construct a Transformer-Mamba hybrid coding network to serialize the one-dimensional features of the pig point cloud. Inputting the trained Transformer-Mamba hybrid coding network yields an enhanced feature sequence that integrates global structure and local geometric details. The Transformer-Mamba hybrid coding network is based on an improved AdaPoinTr architecture, combining the Mamba selective state-space mechanism with the Transformer local feature modeling mechanism. It incorporates a progressive Laplace-Mamba injection strategy, a bidirectional scanning mechanism, and a frequency decoupling module combined with FiLM modulation. Specifically, it includes the following sub-steps:

[0108] Step 4.1, for Perform farthest point sampling for dimensionality reduction to obtain a set of low-resolution center points and their corresponding features; sort the low-resolution center point set by Morton bit interleaving according to three-dimensional coordinates; set up two independent Mamba instances, and perform forward and reverse scanning on the sorted low-resolution center point feature sequence respectively; after concatenating the outputs of the forward and reverse scanning in the channel dimension, restore the original feature dimension through linear projection to obtain low-resolution enhanced features.

[0109] Step 4.2: Restore the low-resolution enhanced features to the original point cloud resolution using nearest neighbor distance weighted interpolation to obtain the low-frequency component representing the overall outline and pose of the pig. ;Will and Subtracting the residuals yields the high-frequency components characterizing local anatomical details. ; with low-frequency components As a condition, scaling and offset factors are generated through a two-layer MLP to optimize high-frequency components. Affine modulation and normalization are performed to obtain ;Will and After concatenation, a linear layer generates gated weights, which are then used to adjust the values ​​of the concatenated data. Weighted fusion and residual connection are performed to output frequency-coordinated enhanced features;

[0110] Step 4.3: Execute the progressive Laplace-Mamba injection strategy, inserting the Laplace-MambaMixer module only in the first half of the encoder layer, while retaining the original Transformer self-attention structure in the second half of the encoder layer, and outputting the enhanced feature sequence.

[0111] Step 5: Decode and reconstruct the enhanced feature sequence based on the cross-attention decoding mechanism to generate a complete pig prediction point cloud.

[0112] The system for implementing the aforementioned method for completing 3D point clouds of pigs in complex environments includes a point cloud feature extraction and fusion module, a point cloud normalization module, a hybrid encoding module, a cross-attention decoding and reconstruction module, and a training and optimization module. The point cloud feature extraction module incorporates a DGCNN grouping unit, receives residual pig point clouds, and extracts downsampled center points and their local geometric features. The point cloud normalization module performs coordinate normalization, 10-bit quantization, 3D Morton encoding, and global sorting on the aggregated features, outputting spatially continuous one-dimensional serialized features. The hybrid encoding module incorporates a progressive Laplace-Mamba injection unit, a bidirectional Mamba scanning unit, a frequency decomposition unit, and a FiLM cross-frequency modulation and gating fusion unit to construct a Transformer-Mamba hybrid encoding network, processing the one-dimensional serialized features and outputting an enhanced feature sequence. The cross-attention decoding and reconstruction module receives the enhanced feature sequence, decodes it to generate a complete predicted pig point cloud. The training and optimization module performs two-stage transfer learning training, iteratively optimizing network parameters based on a composite loss function, and outputting the optimal pig point cloud completion model.

[0113] In the encoding phase of the above process, a Mamba-Transformer hybrid module was designed. This module utilizes the Transformer to capture fine-grained local relationships while leveraging Mamba's selection mechanism and linear complexity to efficiently model the global structure. The LaplacePointMixer module is also involved. It explicitly decomposes the low-frequency morphological components and high-frequency detail components of point cloud features using the graph Laplacian operator and introduces a feature linear modulation mechanism to adaptively enhance high-frequency features based on low-frequency features, fundamentally alleviating the over-smoothing problem in sequence modeling.

[0114] In the decoding stage described above, a cross-attention mechanism is used to gradually reconstruct the complete pig point cloud by combining center point information. Experiments on PCN, ShapeNet-55, and ShapeNet-34 benchmark datasets show that the present invention outperforms or matches existing technologies in metrics such as F-score, and exhibits significant efficiency advantages when processing large-scale point clouds. Furthermore, the present invention transfers the pre-trained model to a self-built pig point cloud dataset for cross-domain fine-tuning, verifying its generalization ability in real agricultural scenarios and providing effective technical support for non-contact measurement of the three-dimensional morphology of pigs in smart pig farms.

[0115] To verify the effectiveness of the technical solution of this invention, systematic experiments were conducted on multiple publicly available benchmark datasets. For synthetic datasets, ShapeNet55 and ShapeNet34 were selected as the main evaluation scenarios. For real-world scanning datasets, the KITTI dataset was introduced to examine the model's generalization ability under actual acquisition noise and occlusion conditions. Simultaneously, ablation experiments were designed to progressively analyze the contribution of each key module in the network model of this invention to the overall performance, and comparative experiments were used to verify the advantages of this method compared to existing technical solutions.

[0116] In the evaluation of synthetic datasets, this invention uses Chamfer Distance (CD) and F-Score as core quantitative indicators, which comprehensively measure the model output from two dimensions: reconstruction accuracy and completion completeness.

[0117] Chamfer distance measures the geometric deviation of the overall shape by calculating the mean of the bidirectional nearest neighbor distances between the predicted point cloud and the ground truth point cloud. Its expression is defined as:

[0118] ;

[0119] The F-score combines precision and recall to provide a unified score for the overall quality of point cloud completion. The calculation method is as follows: ;

[0120] in, and At a given distance threshold The precision and recall are calculated using the F-score, which ranges from [0,1]. A higher F-score indicates better completion performance.

[0121] On the real-world scanning dataset KITTI, due to the lack of precise ground truth annotations for each point, supervised metrics such as CD cannot be directly calculated. Therefore, this invention introduces Minimum Matching Distance (MMD) and Fidelity Error (FD) to evaluate the completion results. MMD measures the minimum chamfer distance between the predicted point cloud and shape samples in the training set, reflecting the reasonableness of the generated shape; FD measures the average nearest neighbor distance between the input sparse points and the completed output, defined as follows: ;

[0122] Where S1 is the input point set and S2 is the completed output point set. The smaller the FD, the higher the degree of preservation of the original input by the completed result, and the stronger the geometric consistency between the two.

[0123] All training and testing tasks in this embodiment are deployed on a computing platform equipped with a single NVIDIA Tesla V100 GPU. At the software level, a PyTorch-based deep learning development environment is built using Docker container technology. In the specific model training configuration, the iteration cycle of the entire network is set to 300 epochs, and the batch size is constant at 16. The Adam algorithm is used to perform gradient optimization of the network weights, and the initial learning rate is specified as 1×10⁻⁶. -4 A tiered learning rate scheduling strategy was adopted—the learning rate was reduced to one-tenth of its current value every 20 training epochs. To ensure the fairness and reproducibility of the experimental results, all baseline methods were retrained and evaluated under the same software and hardware conditions.

[0124] This embodiment uses a self-built pig body point cloud dataset as the benchmark for model performance evaluation. This dataset was constructed using multi-view depth camera acquisition and point cloud registration technology, with the data collected from live pigs, yielding a total of 132 pig body point cloud datasets. The dataset covers various natural postures of pigs, including standing and walking, reflecting the complex geometric features and morphological changes of pigs in the actual production environment. To generate paired partial and complete point clouds, the complete point cloud was obtained through multi-view acquisition and registration fusion, while the partial point clouds were generated online. Specifically, a viewpoint was selected, and the n points farthest from that viewpoint were removed. The results were then iteratively downsampled from the farthest points to 2048 points, simulating point cloud gaps caused by factors such as fence occlusion, limited sensor viewpoints, and mutual occlusion among pigs in real-world farming scenarios. During training, multiple local views and missing patterns were randomly generated to enhance the generalization ability and robustness of the completion network.

[0125] Compared to existing synthetic datasets such as ShapeNet, the pig dataset obtained in this embodiment is derived from a real-world farming environment, better meeting the needs of pig point cloud completion in practical application scenarios. It serves as an ideal performance benchmark for point cloud completion tasks in precision animal husbandry. Figure 4 The pig point cloud in the image needs to be completed. Figure 5 The completion results obtained by this invention are visually demonstrated. Figure 5 This invention demonstrates its effectiveness in accurately predicting missing parts of 3D objects with a reduced number of points, thereby improving accuracy, more precisely restoring details, and better preserving the original incomplete structure, highlighting the advantages of the Mamba network in feature extraction capabilities in point cloud completion tasks.

[0126] The point cloud completion method and network model of this invention can also be applied to other scenarios, such as... Figure 6As shown, to ensure fairness and consistency with existing networks, this embodiment also adopts the L1 norm distance proposed by the classic point cloud completion network PCN as the evaluation metric. Furthermore, the present invention is compared with various classic and state-of-the-art methods, including Adapointr, to comprehensively evaluate the model performance, as shown in Tables 1 and 2. Despite using the same decoding module, the network of the present invention achieves superior completion results in most categories due to the powerful feature extraction capabilities of the Mamba network. In terms of overall average metrics, it significantly outperforms other networks. Specifically, the average CD of the network model of the present invention... L1 It scored 6.44, significantly lower than its best competitor ODGNet (0.06 lower).

[0127] Table 1

[0128] PCN 17.175 0.968 55.470 0.347 Pointr 13.005 0.640 29.689 0.515 SnowflakeNet 12.145 0.593 31.149 0.567 Adapointr 9.441 0.393 22.796 0.644 This invention 9.219 0.376 21.886 0.646

[0129] Table 2 shows the L1CD results on the PCN dataset, with all prices multiplied by a factor of 1000.

[0130] FoldingNet 9.49 15.80 12.61 15.55 16.41 15.97 13.65 14.99 14.31 PCN 5.50 22.70 10.63 8.70 11.00 11.34 11.68 8.59 9.64 GRNet 6.45 10.37 9.45 9.41 7.96 10.51 8.44 8.04 8.83 Pointr 4.75 10.47 8.68 9.39 7.75 10.93 7.78 7.29 8.38 SnowflakeNet 4.29 9.16 8.08 7.89 6.07 9.23 6.55 6.40 7.21 PMP-Net 4.39 9.96 8.53 8.09 6.06 9.82 7.17 6.52 7.56 Adapointr 3.68 8.82 7.47 6.85 5.47 8.35 5.80 5.76 6.53 3dMambaComplete 3.86 9.12 7.72 7.41 5.73 9.04 6.29 6.09 6.91 ODGNet 3.77 8.77 7.56 6.84 5.09 8.47 5.84 5.66 6.50 This invention 3.55 8.80 7.45 6.72 5.13 8.31 5.84 5.71 6.44

[0131] This implementation also included stepwise ablation experiments to quantify the contribution of each module in the model of this invention. As shown in Table 3, baseline model I is the original AdaPoinTr model without modification. Variant II adds a Morton bidirectional Mamba scan module (using a single xyz axis sequence, including both forward and reverse scan directions) and FiLM modulation to the baseline, but does not introduce frequency decoupling, to verify the combined effect of Mamba scan and FiLM modulation. Variant III introduces a Morton bidirectional scan and Laplace frequency decomposition to the baseline, but removes FiLM modulation to evaluate the contribution of frequency decoupling separately. The complete model of this invention further adds FiLM modulation to Variant III, conditionally modulating the decomposed low-frequency and high-frequency features separately to achieve optimal performance. Each configuration only changes specific components relative to the control group, thus allowing performance differences to be explicitly attributed to individual design choices.

[0132] Table 3 Performance comparison of different variants on the PCN dataset

[51]

[0133] I × × × 6.52 0.186 II √ × √ 6.48 0.187 III √ √ × 6.53 0.187 This invention √ √ √ 6.44 0.180

[0134] To verify the impact of different Morton encoding strategies on point cloud completion performance, the embodiments employed single-view Morton encoding and three-view Morton encoding experiments. As shown in Table 4, both methods were trained for 300 epochs under the same dataset, training strategy, and network configuration, with only the generation method of the Morton sequence being changed. Single-view Morton encoding constructs a spatial sequence based solely on the original coordinates, while three-view Morton encoding generates Morton sequences along three orthogonal directions and performs feature fusion. The experimental results are shown in Table 4.

[0135] Table 4

[0136] Three-view Morton coding 0.846 6.479 0.182 24.292 Single-view Morton coding 0.847 6.443 0.180 23.991

[0137] As shown in Table 4, single-view Morton coding outperforms three-view Morton coding in all four evaluation metrics. Compared with the three-view scheme, single-view Morton coding reduces CDL1, CDL2, and EMD by 0.56%, 1.10%, and 1.24%, respectively, indicating that using a single Morton sort can obtain a more stable point cloud sequence representation.

Claims

1. A method for completing 3D point clouds of pigs in complex environments, characterized in that, Includes the following steps: Step 1: Obtain the original cloud data of the defects in the pigs that need to be supplemented; Step 2: Process the original incomplete cloud data using the DGCNN grouping module to obtain aggregated features; Step 3: Normalize the aggregated features. First, normalize the three-dimensional coordinates of the points and perform 10-bit integer quantization. Then, generate Morton codes by interleaving along the XYZ axes. Globally sort the feature sequence according to the Morton codes to obtain one-dimensional serialized features with local continuity in three-dimensional space. ; Step 4: Construct a Transformer-Mamba hybrid coding network to serialize the one-dimensional features of the pig point cloud. Inputting the trained Transformer-Mamba hybrid coding network yields an enhanced feature sequence that integrates global structure and local geometric details. The Transformer-Mamba hybrid coding network is based on an improved AdaPoinTr architecture, combining the Mamba selective state-space mechanism with the Transformer local feature modeling mechanism. It incorporates a progressive Laplace-Mamba injection strategy, a bidirectional scanning mechanism, and a frequency decoupling module combined with FiLM modulation. Specifically, it includes the following sub-steps: Step 4.1, for Perform farthest point sampling for dimensionality reduction to obtain a low-resolution set of center points and corresponding features; The low-resolution center point set is sorted by Morton bit interleaving according to three-dimensional coordinates; two independent Mamba instances are set up, and forward and reverse scanning are performed on the sorted low-resolution center point feature sequence respectively. The outputs of forward and reverse scanning are concatenated in the channel dimension and then linearly projected to restore the original feature dimension to obtain the low-resolution enhanced features. Step 4.2: Restore the low-resolution enhanced features to the original point cloud resolution using nearest neighbor distance weighted interpolation to obtain the low-frequency component representing the overall outline and pose of the pig. Subtract the low-frequency component residual from the input features obtained in step 4.1 to obtain the high-frequency component that characterizes local anatomical details. ; with low-frequency components As a condition, scaling and offset factors are generated through a two-layer MLP to optimize high-frequency components. Affine modulation and normalization are performed to obtain the high-frequency modulation characteristics. ; low frequency components With modulation high-frequency characteristics After concatenation, a linear layer generates gated weights, which are then used to adjust the weights. and Weighted fusion and residual connection are performed to output frequency-coordinated enhanced features; Step 4.3: Execute the progressive Laplace-Mamba injection strategy, inserting the Laplace-MambaMixer module only in the first half of the encoder layer, while retaining the original Transformer self-attention structure in the second half of the encoder layer, and outputting the enhanced feature sequence. Step 5: Decode and reconstruct the enhanced feature sequence based on the cross-attention decoding mechanism to generate a complete pig prediction point cloud.

2. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, The specific processing method for obtaining aggregated features in step 2 is as follows: construct a local graph based on K nearest neighbors and extract initial local geometric features. Then, perform dimensionality reduction step by step by sampling the farthest point twice. Repeatedly perform graph convolution feature aggregation based on K nearest neighbors at the center point after each dimensionality reduction, gradually expand the feature channel dimension, and output the coordinates of the downsampled center point and its corresponding aggregated features.

3. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, The detailed method for step 3 normalization is as follows: First, perform minimum boundary and maximum boundary normalization on the three-dimensional coordinates of the pig defect cloud; Then, the normalized 3D coordinates are quantized to 10-bit integer order; the quantized 3D integer coordinates are interleaved along the XYZ axes to generate Morton codes corresponding to each point cloud point. Next, the aggregated feature sequence is globally sorted according to the Morton code to obtain a normalized one-dimensional serialized feature with continuity in three-dimensional surface space. ; Since the point coordinates generated by the DGCNN grouping module remain unchanged throughout the encoder processing, the Morton sort index results calculated independently by each layer of the encoder are consistent.

4. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, In the Transformer-Mamba hybrid coding network base, a Mamba selective state-space mechanism is constructed based on the Discrete State-Space Model (SSM). The SSM discrete formula is: ； ； , Let be the hidden states at time t and time t-1, respectively. Let be the discrete input at time t. Let be the discrete output at time t. , and This is the discrete parameter matrix obtained by zero-order preserved discretization of continuous parameters. and The following conditions must be met: ； ； Let A be the time step parameter, B be continuous parameter matrices, and I be the identity matrix. To achieve parallel training, the SSM recursive form is equivalently transformed into a global convolutional form: ； ； This represents the global convolution kernel generated from the state-space parameters. Given the input sequence, The output sequence is represented by *, which indicates a convolution operation. The sequence length; The Mamba selective state-space mechanism designs some state parameters as input-dependent functions, satisfying the following equation: .

5. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, The execution process of the bidirectional scanning mechanism in step 4.1 is as follows: Two independent Mamba instances are set up to perform forward and reverse scans on the low-resolution center point feature sequences obtained by sampling and dimensionality reduction of the farthest point and sorting by Morton. The forward scan captures the preceding context information of the sequence, and the reverse scan supplements the following context information missing in the forward scan. Then, the output sequences of the forward and reverse scans are concatenated along the channel dimension and mapped back to the original feature dimension C via linear projection, satisfying: ； The feature sequence after Morton sorting; This is a sequence reversal operation; To perform a forward scan of the Mamba instance, capture the sequence preorder context; To perform a reverse scan of the Mamba instance, process the reverse sequence to supplement the post-sequence context; The concatenation is done along the channel dimension, resulting in a 2C feature dimension. The linear projection layer maps 2C-dimensional features back to the original dimension C; O represents the output features of the bidirectional scanning module. LayerNorm normalization is first applied to the input features, and residual connections are used for scanning and projection operations. Random depth regularization is introduced during the training phase.

6. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, Step 4.2, the detailed process of obtaining frequency co-enhancement features, is as follows: First, the input point cloud features Sparse point set obtained by sampling FPS at the farthest point and corresponding sparse features N is the number of points in the point cloud, and C is the original feature dimension; ； ； ; This is the downsampling factor; Morton bit interleaving sorting is performed according to three-dimensional coordinates, and low-resolution enhanced features are obtained by bidirectional Mamba scanning as described in claim 4; the low-resolution enhanced features are restored to the original number of points N by nearest neighbor distance weighted interpolation to obtain low-frequency components. ; High-frequency components are extracted using residual subtraction. : ; Next, FiLM cross-frequency modulation and fusion are used, the specific process of which includes: Low frequency components Input two-layer MLP to generate modulation parameter scaling factor offset factor ,Right now ; Subsequently, modulation parameters were used to modulate the high-frequency components. Element-wise affine transformation and normalization yield the modulation high-frequency characteristics. : ; For element-wise multiplication, For layer normalization; modulated high-frequency features Compared with the original low-frequency characteristics The weights are concatenated and then generated through a linear layer to form a gated weight. ； It is the Sigmoid activation function. For linear projection layers; Based on gating weights and Weighted fusion is performed to obtain the feature fusion result. ; ； Feature fusion results With input point cloud features Perform residual connections to obtain the final output features of the module. : 。 7. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, The details of step 4.3, the progressive Laplace-Mamba injection strategy, are as follows: Let L be the total number of encoder layers, then the number of layers injected into the Laplace-Mamba module is... : ; This is a floor function; No. The feature update rule for each coding layer is: ； For the first Layer output features; For Laplace-MambaMixer module operations, For Transformer coded block operations; Only before Layered parallel embedding of Mamba modules and Transformer self-attention modules, then The layer retains only the Transformer structure.

8. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, A two-stage transfer learning training strategy was implemented for the Transformer-Mamba hybrid coding network. The first stage involves general pre-training, selecting publicly available 3D point cloud benchmark datasets as training samples to fully train the Transformer-Mamba hybrid coding network, enabling the Transformer-Mamba hybrid coding network to learn the representation capabilities of general object 3D geometric structure, surface morphology, and edge details. The second stage involves domain fine-tuning, collecting depth images of pigs in real pig farming scenarios, generating defective cloud data through coordinate transformation, and constructing a fine-tuning dataset. The pre-trained network was iteratively fine-tuned based on the fine-tuning dataset to adapt to the biological geometric features, trunk surface morphology and noise distribution characteristics of the actual collection environment of the pig. The network training uses a composite loss function, which is obtained by weighted fusion of fine point cloud chamfer distance loss, center point chamfer distance loss and denoising branch chamfer distance loss; The formula for chamfer distance CD loss is: ； P represents the predicted point cloud set, Q represents the true point cloud set, and N and M represent the number of points in the predicted and true point clouds, respectively. The distance is the Euclidean distance with a 2-norm. The final composite loss function is: ; To balance the weight hyperparameters.

9. The method for completing 3D point clouds of pigs in complex environments according to claim 1, characterized in that, The process of progressively completing a 3D point cloud is visualized by using Mitsuba and Open3D tools to perform 3D visualization rendering on the completed 3D point cloud of a pig, intuitively showing the completion effect in different areas.

10. A 3D point cloud completion system for pigs in complex environments, characterized in that, It includes a point cloud feature extraction module, a point cloud normalization module, a hybrid encoding module, a cross-attention decoding and reconstruction module, and a training optimization module; the point cloud feature extraction module has a built-in DGCNN grouping unit, which receives the residual point cloud of pigs and extracts the downsampled center point and its local geometric features; The point cloud normalization module performs coordinate normalization, 10-bit quantization, 3D Morton encoding and global sorting on the aggregated features, and outputs spatially continuous one-dimensional serialized features. The hybrid coding module incorporates a progressive Laplace-Mamba injection unit, a bidirectional Mamba scanning unit, a frequency decomposition unit, and a FiLM cross-frequency modulation and gating fusion unit to construct a Transformer-Mamba hybrid coding network, which outputs an enhanced feature sequence after processing one-dimensional serialization features. The cross-attention decoding and reconstruction module receives the enhanced feature sequence and decodes it to generate a complete pig prediction point cloud. The training and optimization module performs two-stage transfer learning training, iteratively optimizes network parameters based on a composite loss function, and outputs the optimal pig point cloud completion model.

Images