An Automatic Measurement Method for Muscovy Duck Body Size Based on Multimodal Fusion and Parametric Mesh Reconstruction

Abstract

This invention relates to the field of Muscovy duck body size measurement, specifically to an automatic body size measurement method for Muscovy ducks based on multimodal fusion and parametric mesh reconstruction. This invention simultaneously acquires six channels of data (RGB, depth, and infrared) using two binocular depth cameras (side-view and top-view). Infrared-guided depth completion and dual-view point cloud fusion are then used to obtain a complete point cloud. A multimodal feature encoder extracts fused features, and shape and posture parameters are decoded using depth-sensing attention decoding. A three-dimensional mesh is generated by combining this with a statistical shape model of the Muscovy duck, and posture differences are eliminated through posture normalization. Finally, seven body size parameters are geometrically measured on a standard posture mesh, and body weight is estimated using a volume density model. This invention upgrades body size detection from regression prediction to a three-dimensional reconstruction and geometric measurement paradigm, significantly improving measurement accuracy and consistency. This invention is applicable to Muscovy duck body size measurement.

Description

Technical Field

[0001] This invention relates to the field of Muscovy duck body size measurement, specifically to an automatic measurement method for Muscovy duck body size based on multimodal fusion and parametric mesh reconstruction. Background Technology

[0002] Measuring the body size of Muscovy ducks is a core requirement for precision poultry farming management. Morphological indicators such as weight, body length, keel length, semi-submersion length, chest width, chest depth, neck length, and shank length directly reflect the growth and development status of Muscovy ducks and are important bases for nutritional regulation, breeding selection, and slaughter decisions. Traditional manual measurement methods require capturing and keeping the ducks still, which is time-consuming, labor-intensive, and can cause stress to the ducks, affecting their subsequent growth and development.

[0003] With the development of deep learning and computer vision technologies, image-based non-contact body size measurement methods have gradually emerged. Related technical approaches mainly include: 2D image feature regression methods (using convolutional neural networks to extract features from images and then directly regress body size values), 2D-3D fusion-based keypoint detection methods (detecting keypoints on 2D images and then projecting them onto a point cloud surface to calculate body size), and direct measurement methods based on point clouds.

[0004] In the poultry sector, due to the greater freedom of movement, more complex feather occlusion, and lower economic value of poultry compared to large livestock, existing work is extremely scarce.

[0005] In the field of 3D mesh reconstruction, parametric animal shape models (such as SMAL for tetrapods and AVES for birds) combined with high-capacity Vision Transformer encoders and Transformer decoders have made significant progress in multi-species pose and shape estimation, but have not yet been applied to the scenario of accurate measurement of poultry body size.

[0006] The existing technology mainly has the following problems:

[0007] There is a lack of a three-dimensional geometric measurement paradigm for body size measurement;

[0008] The top-view depth information is missing, and the 3D contour recovery is incomplete.

[0009] Infrared modes are not being utilized, and deep voids are difficult to repair;

[0010] The single-view point cloud occlusion is severe, and the surface coverage is incomplete.

[0011] There is a lack of a shape model specifically for Muscovy ducks, and the spatial adaptability of general animal shapes is insufficient.

[0012] The lack of a pose normalization mechanism results in insufficient consistency in cross-pose measurements.

[0013] Weight prediction lacks physical constraints and has insufficient interpretability;

[0014] The relevant 3D reconstruction methods do not have the ability to output both body size and weight. Summary of the Invention

[0015] The purpose of this invention is to overcome the shortcomings of the prior art and provide an automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction. This solves the problem that existing animal 3D reconstruction methods only output posture parameters and cannot directly serve the needs of precise breeding and measurement of Muscovy ducks.

[0016] The present invention achieves the above objectives by adopting the following technical solution: The present invention provides an automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction, comprising the following steps:

[0017] S1. Simultaneously acquire RGB images, depth images and infrared images of the mallard from the side view and top view using the first binocular depth camera and the second binocular depth camera to form six channels of raw multimodal data;

[0018] S2. Perform infrared-guided depth completion on the depth image to obtain a dense depth map; backproject the side-view and top-view dense depth maps into point clouds respectively, and obtain a complete three-dimensional point cloud covering the surface of the Muscovy duck after registration and fusion.

[0019] S3. Features are extracted from the RGB image, dense depth map and infrared image respectively using a multimodal feature encoder, and then fused through a progressive cross-modal attention mechanism to obtain a unified multimodal feature representation;

[0020] S4. Based on the multimodal feature representation, the shape parameters and posture parameters of the Muscovy duck are decoded by a depth-sensing attention Transformer; a preliminary 3D mesh is generated using an offline constructed statistical shape model of the Muscovy duck based on the shape parameters and posture parameters; the preliminary 3D mesh is refined by graph convolution vertex refinement using the complete 3D point cloud to obtain a fine 3D mesh.

[0021] S5. Replace the posture parameters of the fine three-dimensional mesh with predefined standard standing posture parameters, and regenerate the standard posture mesh using the Muscovy duck statistical shape model.

[0022] S6. On the standard posture grid, measure the body size parameters directly using geometric methods; calculate the volume and body surface area based on the standard posture grid, and estimate the weight using a physically interpretable density model.

[0023] Furthermore, the infrared-guided depth completion specifically includes:

[0024] A sparse convolutional encoder is used to extract features of the effective depth region in the depth image, and a standard convolutional encoder is used to extract features of the infrared image. The two features are fused through a cross-modal attention mechanism with depth features as queries and infrared features as keys and values. After decoding, a dense depth map is output. The training loss function is composed of L1 loss and edge alignment loss on effective pixels.

[0025] Furthermore, the dense depth maps from the side and top views are back-projected into point clouds, and after registration and fusion, a complete 3D point cloud covering the surface of the Muscovy duck is obtained, specifically including:

[0026] Before backprojection, the depth image is segmented using RGB images to retain only the foreground region of the duck; the dense depth maps of the side view and top view are backprojected to generate side view point clouds and top view point clouds respectively; the top view point cloud is transformed to the side view coordinate system using offline calibrated extrinsic parameters, and then iterative nearest point fine registration is performed; the two point clouds are merged and voxel downsampling is performed to obtain the complete point cloud.

[0027] Furthermore, the multimodal feature encoder employs a two-stage progressive fusion:

[0028] The first stage enhances the RGB features with depth features to eliminate depth direction blur; the second stage supplements the edge contour information unrelated to illumination with infrared features; among them, the RGB, depth and infrared features are extracted by independent VisionTransformer encoders.

[0029] Furthermore, the depth-sensing attention Transformer injects a depth space bias into the decoder's self-attention calculation: a positive bias is assigned to the image patch position belonging to the Muscovy body region, and a negative infinite bias is assigned to the background region, so that the decoder focuses on the features of the Muscovy body region.

[0030] Furthermore, the offline construction process of the statistical shape model of the Muscovy duck includes:

[0031] Poisson surface reconstruction was performed on the collected Muscovy duck point cloud to obtain the original mesh; using the individual mesh whose body shape is closest to the average value as a template, non-rigid iterative nearest-point registration was performed on all sample meshes to obtain a topologically consistent mesh set; principal component analysis was performed on the vertices of the registered mesh to extract the set number of principal components as shape basis, and the mean of all sample vertices was used as the average vertex coordinates to complete the offline construction of the Muscovy duck statistical shape model.

[0032] Furthermore, the process of generating a preliminary 3D mesh using the offline-constructed statistical shape model of the Muscovy duck based on the shape and posture parameters specifically includes:

[0033] The statistical shape model of the Muscovy duck includes the average vertex coordinates of the Muscovy duck and multiple shape bases extracted from a large amount of Muscovy duck point cloud data through principal component analysis. Based on the decoded shape parameters, the multiple shape bases are weighted according to their corresponding shape parameters and superimposed on the average vertex coordinates to generate individualized vertex coordinates. Then, through linear blending skinning, the skeleton joints are driven to rotate according to the decoded posture parameters, thereby generating a preliminary three-dimensional mesh.

[0034] Furthermore, the predefined standard standing posture parameters are determined by taking the element-wise median of all duck posture parameters in the training set as the standard standing posture.

[0035] In step S5, the shape parameters are kept unchanged, the posture parameters are replaced with the standard standing posture, and the standard posture mesh is regenerated using the statistical shape model of the Muscovy duck, so that the same Muscovy duck can obtain consistent body size measurement results after normalization under different collection postures.

[0036] Furthermore, the loss function for graph convolution vertex thinning consists of three components: Chamfer distance, surface smoothing loss, and normal vector consistency loss.

[0037] The fine 3D mesh is further optimized by a dual-view consistency constraint. This constraint uses a differentiable renderer to render the fine mesh onto the side and top planes respectively, compares it with the real segmented contour and depth map, and applies a geometric consistency loss consisting of a weighted sum of contour loss, depth loss and Chamfer distance loss.

[0038] Furthermore, the body size parameters include seven items, which are directly measured from the standard attitude mesh using the following geometric methods:

[0039] Body oblique length: the Euclidean distance between the tip of the beak and the farthest point of the tail;

[0040] Keel length and semi-submersible length: geodesic distance along the grid surface;

[0041] Chest width and chest depth: Cut a mesh section perpendicular to the spine at the chest joint and take the maximum transverse width and maximum longitudinal height of the section profile;

[0042] Neck length: The sum of the Euclidean distances between adjacent joints on the cervical skeletal articular chain;

[0043] Tibia length: the Euclidean distance between the knee and ankle joints of the skeleton.

[0044] The beneficial effects of this invention are as follows:

[0045] This invention constructs a 3D mesh for Muscovy ducks, standardizes the mesh to a standard standing posture, and then calculates body size parameters on the standard posture mesh using a geometric measurement module. Unlike existing technologies that directly regress image or point cloud features into body size values ​​using neural networks, the body size results of this invention are derived from geometric quantities such as geodesic distance, cross-sectional width, cross-sectional depth, and joint distance on the 3D mesh. This fundamentally eliminates the systematic errors caused by end-to-end black-box regression and improves the interpretability and optimizability of body size measurement results.

[0046] This invention employs dual RGB-D cameras for simultaneous acquisition from both side-view and top-view perspectives, enabling the acquisition of not only RGB images but also depth and infrared images from the top-view perspective. Through the top-view depth map, this invention can obtain realistic three-dimensional structural information of the duck's back, hips, upper chest edge, and body width. Furthermore, the dual-view consistency constraint uses the top-view depth map as a geometric supervision signal, making the reconstructed mesh more closely resemble the actual duck body morphology in the body width and back contour directions. This improves the measurement accuracy of body size parameters such as chest width, chest depth, and semi-submersible length, overcoming the fundamental deficiency of existing technologies that completely lack three-dimensional information from the top-view perspective.

[0047] This invention designs an infrared-guided depth completion method. Leveraging the pixel alignment between infrared and depth images, it incorporates edge, contour, and texture information from the infrared image into the depth completion process. For depth holes caused by feather edges, abrupt changes in body curvature, and near-field blind spots in Muscovy ducks, infrared images can provide more stable boundary guidance information. This improves the continuity and edge accuracy of the depth map, significantly increasing the depth hole repair rate and providing higher-quality depth input for subsequent point cloud generation, dual-view fusion, and 3D mesh reconstruction.

[0048] This invention generates point clouds by back-projecting side-view and top-view depth maps respectively, and obtains a fused point cloud from both perspectives through extrinsic parameter matrix alignment and ICP fine registration. Compared with existing technologies that rely solely on a single side-view point cloud, this invention can supplement the 3D information of the duck's back, abdomen, contralateral body surface, and local leg areas, improving the point cloud's body surface coverage and reducing surface defects caused by single-view occlusion. This fused point cloud is further used for GCN vertex refinement, subjecting mesh vertices to a more complete body surface point cloud constraint, reducing the fitting error between the mesh and the actual duck body surface.

[0049] This invention utilizes an offline-constructed statistical shape model of Muscovy ducks. It extracts the average shape and main shape variation directions from 3D mesh data of Muscovy duck samples to represent the body shape differences between individual ducks. Unlike existing technologies that use a universal shape space for multiple species, the Muscovy duck statistical shape model can specifically model the breast morphology, hip width, keel region, shank structure, and overall body proportions of Muscovy ducks. Each principal component direction of the shape basis corresponds to the actual body shape variation patterns of Muscovy ducks, improving the accuracy of 3D mesh reconstruction in adapting to Muscovy duck body shapes and providing a more accurate 3D geometric basis for subsequent body size measurements.

[0050] This invention transforms 3D meshes of Muscovy ducks in different postures during data collection into a standard standing posture. This process preserves the shape parameters of the individual ducks, standardizing only the posture parameters, thereby eliminating the influence of head and neck extension / retraction, changes in body orientation, standing posture, and leg posture on body size measurements. As a result, the body size results obtained for the same Muscovy duck under different collection postures are completely consistent. In particular, it improves the measurement consistency of body size indicators closely related to posture changes, such as semi-submersible length, body oblique length, tibia length, and keel length, and reduces the variance in body size measurements of the same individual under different postures.

[0051] This invention accurately calculates the volume and surface area of ​​Muscovy ducks using a standard-pose closed mesh and estimates their weight by combining physical characteristics such as body size index and equivalent density. The core physical term is that weight is approximately equal to equivalent density multiplied by volume. Unlike existing technologies that directly regress weight from image features or fused features (black box regression), this invention's weight estimation is based on parameters with clear physical meaning, such as three-dimensional volume and surface area, making the predictions clear and interpretable. The model based on physical characteristics demonstrates stronger generalization ability when sample distribution, collection environment, or Muscovy duck body size changes.

[0052] This invention outputs seven body size parameters—oblique length, keel length, semi-submersible length, chest width, chest depth, neck length, and tibia length—from a standard posture 3D mesh. Furthermore, it outputs an estimated body weight based on mesh volume, body surface area, and body size characteristics. This invention not only completes 3D shape reconstruction but also directly converts the reconstruction results into body size and weight data usable in aquaculture production. Attached Figure Description

[0053] Figure 1 This is a flowchart of an automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction, provided by an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0055] This invention provides an automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction, such as... Figure 1 As shown, it includes the following steps:

[0056] S1. Data Acquisition: Simultaneously acquire RGB images, depth images, and infrared images of the mallard duck from the side and top views using the first and second binocular depth cameras to form six channels of raw multimodal data.

[0057] In one embodiment of the invention, two Gemini 335L cameras are installed on a duck collection box. The side-view camera is fixed above a 45° inclined plane, and the top-view camera is fixed 1.0m directly above the box. The two cameras achieve frame-level precise synchronization triggering through a hardware synchronization interface, synchronously outputting six data channels per frame: a side-view RGB image. Side view depth map Side-view infrared image And the corresponding top-down three-way data , , The infrared and depth maps are generated by the same binocular sensor system, with naturally aligned pixels that require no additional calibration.

[0058] This embodiment collects multimodal visual data from no fewer than 500 Muscovy ducks. Each duck is captured in 3 to 5 different poses within the acquisition box, ultimately yielding no fewer than 1500 complete sets of six-channel multimodal data. Simultaneously, seven body dimensions and weight of each duck are manually measured as ground truth values. Weight is measured using an electronic scale; body length, keel length, neck length, and semi-diving length are measured using a flexible measuring tape; and chest width, chest depth, and shank length are measured using electronic calipers. All data are divided into training, validation, and test sets in an 8:1:1 ratio. The training set is used for model training, the validation set for hyperparameter tuning, and the test set for final performance evaluation. All pose data from the same duck belong to the same subset to avoid data leakage.

[0059] In an alternative embodiment of the present invention, the Orbbec Gemini 335L camera described above can be replaced with other RGB-D cameras capable of simultaneously outputting RGB, depth, and infrared modes, such as the Microsoft Azure Kinect DK or the Intel RealSense D455. When replacing the camera, it is necessary to ensure that the infrared and depth maps of the selected camera are naturally aligned and that multi-camera hardware synchronous triggering acquisition is supported. If the selected camera does not have hardware synchronization capabilities, a software timestamp alignment scheme can be used as an alternative, but this will introduce a certain time deviation, slightly affecting the accuracy of subsequent point cloud registration.

[0060] S2. Data preprocessing: Infrared-guided depth completion is performed on the depth image to obtain a dense depth map; the dense depth maps from the side view and top view are back-projected into point clouds, and after registration and fusion, a complete three-dimensional point cloud covering the surface of the Muscovy duck is obtained.

[0061] In one embodiment of the present invention, infrared-guided depth completion is specifically as follows:

[0062] The original depth maps from the side and top views are processed as follows: Holes with zero pixel values ​​exist in areas such as the edges of the duck feathers and near-field blind spots in the original depth maps. These holes are repaired using infrared images. Specifically, the depth branch uses a sparse convolutional encoder to extract features from the effective depth region. The infrared branch uses a standard convolutional encoder to extract edge texture features from the infrared image. The two feature streams are fused through cross-modal attention, with deep features as the query and infrared features as the key and value:

[0063] ;

[0064] The fused features are upsampled by the decoder to output a dense depth map. The loss function consists of two terms: effective pixel L1 loss and edge alignment loss.

[0065] ;

[0066] Where Ω represents the set of effective depth pixels. For the image gradient operator, λ=0.1.

[0067] In an alternative embodiment of the present invention, the aforementioned cross-modal attention fusion mechanism can be replaced by other fusion strategies such as convolutional decoding after feature channel concatenation, spatial propagation network (SPN), or guided filtering. The edge alignment term in the loss function can be replaced by structural similarity loss (SSIM) or gradient domain loss. All of the above alternative solutions can achieve the basic function of guided depth completion while ensuring pixel alignment between the infrared image and the depth map.

[0068] In one embodiment of the present invention, the dual-view point cloud registration and fusion is specifically as follows:

[0069] Before backprojection, background segmentation is performed on the depth map using the acquired RGB images. Only the depth pixels corresponding to the foreground region of the mallard duck are retained before backprojection to avoid background points from the acquisition box being mixed into the point cloud. The side-view and top-view completed depth maps are then backprojected to generate 3D point clouds. and Using the extrinsic transformation matrix calibrated offline The top-view point cloud was transformed to the side-view coordinate system, and then ICP fine registration was performed to eliminate residual calibration errors. Finally, the two point clouds were merged and 3mm voxel downsampling was performed to obtain the complete Muscovy duck body surface point cloud. .

[0070] S3. Feature Encoding: Features are extracted from the RGB image, dense depth map and infrared image respectively using a multimodal feature encoder, and then fused through a progressive cross-modal attention mechanism to obtain a unified multimodal feature representation.

[0071] In one embodiment of the present invention, a multimodal feature encoder processes side-view and top-view RGB images, completed depth images, and infrared images, respectively. The RGB images, completed depth images, and infrared images are each processed by an independent VisionTransformer encoder to extract features. , , (Each dimension is N×d, where N is the number of patches and d is the feature dimension).

[0072] A two-stage, gradual integration approach is adopted:

[0073] The first stage enhances RGB features with depth features to eliminate depth direction blur in monocular images:

[0074] ;

[0075] The second stage supplements the illumination-independent edge contour information with infrared features:

[0076] ;

[0077] Fusion features The data is then fed into subsequent processing steps. In addition, the class token f output by ViT is used for body size prior constraints, classifying the ducks into weight classes. Through supervised contrastive learning, the characteristics of ducks in the same body size class are similar, while the characteristics of ducks in different classes are different, thus enhancing the decoder's ability to perceive body size differences.

[0078] In an alternative embodiment of the present invention, the above encoder structure can be replaced by the following alternatives: First, a single Vision Transformer with shared weights can be used to process the channel stitching input of the three images, which can reduce the number of parameters; Second, the Vision Transformer can be replaced with other visual encoders such as Swin Transformer and ConvNeXt; Third, the progressive two-stage cross-modal fusion can be replaced with a single-stage scheme of direct stitching of three features followed by linear projection fusion, thereby achieving a simpler fusion process.

[0079] S4. Accurate Reconstruction of the Muscovy Duck 3D Mesh: Based on multimodal feature representation, the shape and posture parameters of the Muscovy duck are decoded through a depth-sensing attention Transformer; an offline-constructed statistical shape model of the Muscovy duck is used to generate a preliminary 3D mesh based on the shape and posture parameters; the preliminary 3D mesh is refined by graph convolution vertex refinement using the complete 3D point cloud to obtain a fine 3D mesh.

[0080] In one embodiment of the present invention, the depth-aware attention Transformer decoding is specifically as follows:

[0081] Spatial attention bias is constructed using the completed depth map: a positive bias is assigned to the patch positions belonging to the Muscovy body region, and a negative infinite bias is assigned to the background region. This bias is then injected into the self-attention calculation of the Transformer decoder, enabling the decoder to focus on the Muscovy body region features and suppress background interference.

[0082] ;

[0083] The decoder output features are used by three independent MLP regression heads to predict shape parameters. Attitude parameters and global translation .

[0084] In one embodiment of the present invention, the offline construction of the statistical shape model of the Muscovy duck is specifically as follows:

[0085] Poisson surface reconstruction was performed on the collected Muscovy duck point clouds to obtain the original mesh. Using the individual mesh whose body size is closest to the average as a template, non-rigid ICP registration was performed on all sample meshes to obtain a topologically consistent mesh set. Principal component analysis was performed on the vertices of the registered mesh, and the top 10 principal components were extracted as shape basis. The average vertex coordinates are taken as the mean of all sample vertices. Complete the Duck-SSM construction.

[0086] In an alternative embodiment of the present invention, when sufficient data is collected, the shape space described above for linear principal component analysis can be replaced with a nonlinear shape space constructed by a variational autoencoder (VAE) to better model the nonlinear variation law of the Muscovy duck's body shape. The number of shape basis K can be flexibly adjusted between 8 and 20 according to the cumulative variance explained by the actual dataset, with the recommended value for K being the number of principal components that retain more than 90% of the cumulative variance.

[0087] In one embodiment of the present invention, the process for generating a preliminary mesh is as follows:

[0088] Average vertex coordinates of Muscovy ducks And 10 shape bases extracted from a large amount of Muscovy point cloud data via PCA Composition. Given shape parameters Generate individual vertex coordinates:

[0089] ;

[0090] Then, the linear hybrid skin is based on the attitude parameters. Drive the joints of the skeleton to rotate, generating a preliminary 3D mesh. .

[0091] In one embodiment of the present invention, graph convolution vertex thinning is specifically as follows:

[0092] The initial mesh generated from the offline constructed statistical shape model of Muscovy ducks lacks true 3D constraints per vertex. A graph convolutional network is used to fuse the point cloud. Predict the position offset for each vertex To obtain a fine mesh The refinement loss consists of three components: Chamfer distance, surface smoothing loss, and normal vector consistency loss, ensuring that the mesh accurately fits the real surface of the duck and has a smooth surface.

[0093] In an alternative embodiment of the present invention, the above-mentioned graph convolutional network (GCNN) thinning method can be replaced by the following schemes: First, PointNet++ is used to replace the GCNN to perform feature aggregation on the vertex neighborhood point cloud before predicting the offset; Second, the vertex-by-vertex offset prediction is replaced by implicit neural surface thinning, i.e., training a small MLP that takes vertex coordinates as input and outputs a signed distance function value, and moves the vertices to the zero level set position through gradient descent; Third, when there is sufficient data, the GCNN thinning step can be omitted, and the Duck-SSM parameters can be directly driven to the accurate result by relying on the dual-view consistency constraint. The Chamfer distance term in the thinning loss can be replaced by the one-way point-to-plane distance (Point-to-PlaneDistance), which has a faster convergence speed when the mesh surface is relatively smooth.

[0094] In one embodiment of the present invention, the dual-view consistency constraint is specifically as follows:

[0095] Using a differentiable renderer Rendered separately onto the side and top view planes, compared with the actual segmented contours and depth maps, and a dual-view geometric consistency constraint loss is applied. :

[0096] ;

[0097] Side-view constraints control the shape accuracy in the length and height directions, while top-view constraints control the shape accuracy in the width direction, forming a geometric complement.

[0098] In an alternative embodiment of the invention, the aforementioned differentiable renderer can be replaced with other differentiable rendering frameworks that support gradient backpropagation, such as PyTorch3D or Kaolin. The cross-entropy loss in the contour consistency loss can be replaced with Dice loss or IoU loss, which provides better optimization stability when the segmentation boundaries are complex. The dual-view consistency constraint can be jointly trained with graph convolutional refinement to jointly optimize the fine mesh.

[0099] S5. Replace the posture parameters of the fine three-dimensional mesh with predefined standard standing posture parameters, and regenerate the standard posture mesh using the Muscovy duck statistical shape model.

[0100] In one embodiment of the present invention, attitude normalization is specifically as follows:

[0101] When collecting data on Muscovy duck postures, these postures are random, and direct measurement can introduce inconsistencies. The element-wise median of all Muscovy duck posture parameters in the training set is defined as the standard standing posture. Preserve shape parameters Remain unchanged, replace the attitude parameters with Regenerate the standard standing pose mesh:

[0102] ;

[0103] This operation only changes the duck's posture without altering its body shape, ensuring that the body size measurements of the same duck are completely consistent regardless of its collection posture.

[0104] In an alternative embodiment of the invention, standard standing posture parameters The definition can be replaced by the following alternatives: First, domain experts can manually specify the angles of each joint according to the standard body size measurement specifications for Muscovy ducks; second, select a number of samples with the highest posture scores from the training set and take the average of their posture parameters as the mean. All of the above alternatives can achieve the core objective of attitude normalization, among which the median approach is the most robust to abnormal attitude samples and the simplest to implement in engineering.

[0105] S6. Output: Body size parameters are directly measured on the standard attitude mesh using geometric methods; volume and body surface area are calculated based on the standard attitude mesh, and body weight is estimated using a physically interpretable density model.

[0106] In one embodiment of the invention, seven body scales are measured from standard attitude mesh geometry. The measurements are taken directly using pure geometric algorithms, without relying on any neural network predictions.

[0107] Body oblique length : The Euclidean distance from the tip of the mouth to the farthest point of the tail.

[0108] Long keel Semi-submersible captain The shortest path along the geodesic distance on the grid surface is found on the grid edge graph using Dijkstra's algorithm.

[0109] chest width , chest deep Cut a mesh section perpendicular to the spine at the thoracic joint of the skeleton, and take the maximum transverse width and maximum longitudinal height of the section profile.

[0110] Neck length The sum of the Euclidean distances between adjacent joints in the cervical skeletal articular chain.

[0111] tibia length : The Euclidean distance from the knee joint to the ankle joint of the skeleton.

[0112] Volume and surface area calculation: The volume V of the Muscovy duck is accurately calculated based on the divergence theorem of closed mesh, and the surface area S is obtained by summing the areas of all triangular facets.

[0113] Physically Interpretable Weight Estimation: Constructing Physical Eigenvectors Weight regression through lightweight three-layer MLP Its core physical terms are:

[0114] ;

[0115] equivalent density By implicitly learning from training data, the network automatically captures the influence of factors such as the duck's feeding status and the dryness or wetness of its feathers on the volume-weight relationship, which has stronger physical interpretability compared to the original black-box regression method.

[0116] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. An automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction, characterized in that, Includes the following steps: S1. Simultaneously acquire RGB images, depth images and infrared images of the mallard from the side view and top view using the first binocular depth camera and the second binocular depth camera to form six channels of raw multimodal data; S2. Perform infrared-guided depth completion on the depth image to obtain a dense depth map; backproject the side-view and top-view dense depth maps into point clouds respectively, and obtain a complete three-dimensional point cloud covering the surface of the Muscovy duck after registration and fusion. S3. Features are extracted from the RGB image, dense depth map and infrared image respectively using a multimodal feature encoder, and then fused through a progressive cross-modal attention mechanism to obtain a unified multimodal feature representation; The multimodal feature encoder employs a two-stage progressive fusion: The first stage enhances RGB features with depth features to eliminate depth direction blur, as follows: ； In the formula, Indicates RGB-deep fusion characteristics. Represents RGB image features, Representing depth image features, Represents the query vector. Represents the key vector. Represents a value vector. Represents the cross-attention function; The second stage supplements the illumination-independent edge contour information with infrared features, as follows: ； In the formula, This represents the final multimodal feature representation. Indicates infrared image features; Among them, RGB image features, depth image features, and infrared image features are extracted by independent VisionTransformer encoders; S4. Based on the multimodal feature representation, the shape parameters and posture parameters of the Muscovy duck are decoded by a depth-sensing attention Transformer; a preliminary 3D mesh is generated using an offline constructed statistical shape model of the Muscovy duck based on the shape parameters and posture parameters; the preliminary 3D mesh is refined by graph convolution vertex refinement using the complete 3D point cloud to obtain a fine 3D mesh. S5. Replace the posture parameters of the fine three-dimensional mesh with predefined standard standing posture parameters, and regenerate the standard posture mesh using the Muscovy duck statistical shape model. S6. On the standard posture grid, measure the body size parameters directly using geometric methods; calculate the volume and body surface area based on the standard posture grid, and estimate the weight using a physically interpretable density model.

2. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The infrared-guided depth completion specifically includes: A sparse convolutional encoder is used to extract features of the effective depth region in the depth image, and a standard convolutional encoder is used to extract features of the infrared image. The two features are fused through a cross-modal attention mechanism with depth features as queries and infrared features as keys and values. After decoding, a dense depth map is output. The training loss function is composed of L1 loss and edge alignment loss on effective pixels.

3. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The dense depth maps from the side and top views are back-projected into point clouds, and after registration and fusion, a complete 3D point cloud covering the surface of the Muscovy duck is obtained, specifically including: Before backprojection, the depth image is segmented using RGB images to retain only the foreground region of the duck; the dense depth maps of the side view and top view are backprojected to generate side view point clouds and top view point clouds respectively; the top view point cloud is transformed to the side view coordinate system using offline calibrated extrinsic parameters, and then iterative nearest point fine registration is performed; the two point clouds are merged and voxel downsampling is performed to obtain the complete point cloud.

4. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The depth-sensing attention Transformer injects a depth space bias into the decoder's self-attention calculation: it assigns a positive bias to the image patch position belonging to the Muscovy body region and a negative infinite bias to the background region, so that the decoder focuses on the features of the Muscovy body region.

5. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The offline construction process of the statistical shape model of the Muscovy duck includes: Poisson surface reconstruction was performed on the collected Muscovy duck point cloud to obtain the original mesh; using the individual mesh whose body shape is closest to the average value as a template, non-rigid iterative nearest-point registration was performed on all sample meshes to obtain a topologically consistent mesh set; principal component analysis was performed on the vertices of the registered mesh to extract the set number of principal components as shape basis, and the mean of all sample vertices was used as the average vertex coordinates to complete the offline construction of the Muscovy duck statistical shape model.

6. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The process of generating a preliminary 3D mesh using an offline-constructed statistical shape model of a Muscovy duck based on the shape and posture parameters specifically includes: The statistical shape model of the Muscovy duck includes the average vertex coordinates of the Muscovy duck and multiple shape bases extracted from a large amount of Muscovy duck point cloud data through principal component analysis. Based on the decoded shape parameters, the multiple shape bases are weighted according to their corresponding shape parameters and superimposed on the average vertex coordinates to generate individualized vertex coordinates. Then, through linear blending skinning, the skeleton joints are driven to rotate according to the decoded posture parameters, thereby generating a preliminary three-dimensional mesh.

7. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The predefined standard standing posture parameters are determined by taking the element-wise median of all duck posture parameters in the training set as the standard standing posture. In step S5, the shape parameters are kept unchanged, the posture parameters are replaced with the standard standing posture, and the standard posture mesh is regenerated using the statistical shape model of the Muscovy duck, so that the same Muscovy duck can obtain consistent body size measurement results after normalization under different collection postures.

8. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The loss function for graph convolution vertex thinning consists of three components: Chamfer distance, surface smoothing loss, and normal vector consistency loss. The fine 3D mesh is further optimized by a dual-view consistency constraint. This constraint uses a differentiable renderer to render the fine mesh onto the side and top planes respectively, compares it with the real segmented contour and depth map, and applies a geometric consistency loss consisting of a weighted sum of contour loss, depth loss and Chamfer distance loss.

9. The automatic measurement method for the body size of Muscovy ducks based on multimodal fusion and parametric mesh reconstruction according to claim 1, characterized in that, The body size parameters include seven items, which are directly measured from the standard attitude grid using the following geometric methods: Body oblique length: the Euclidean distance between the tip of the beak and the farthest point of the tail; Keel length and semi-submersible length: distance along the geodesic line on the grid surface; Chest width and chest depth: Cut a mesh section perpendicular to the spine at the chest joint and take the maximum transverse width and maximum longitudinal height of the section profile; Neck length: The sum of the Euclidean distances between adjacent joints on the cervical skeletal articular chain; Tibia length: the Euclidean distance between the knee and ankle joints of the skeleton.

Images