Method, system, device and medium for real-time weight estimation of unrestrained pigs in group housing

Abstract

The present invention relates to a method, system, device, and medium for real-time weight estimation of unconstrained group-raised pigs, comprising: synchronously collecting color images and depth images of group-raised pigs in a group-raised pig house scene; inputting the color images of the group-raised pigs into an instance segmentation model for multi-pig recognition and pixel-level mask segmentation to obtain color images of individual pigs; inputting the color images of individual pigs into a pig posture classification model to determine the posture category of the segmented individual pigs, and combining them with the posture category of the depth image to obtain multi-posture images of the individual pigs; inputting the multi-posture images of the individual pigs into a multi-posture dynamic fusion weight estimation model to perform cross-modal feature complementation and multi-posture dynamic weighted fusion to obtain real-time weight estimation results for the individual pigs. The present invention can be widely applied in the field of image processing technology.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a method, system, device and medium for real-time weight estimation of unrestrained pigs raised in groups. Background Art

[0002] Currently, non-contact pig weight estimation solutions primarily use images of pigs in a standard standing position, with their heads and backs straight. However, the natural behavior of pigs in group housing is complex, and capturing a standard standing position is difficult for unrestrained pigs. This significantly reduces the timeliness and coverage of current solutions that use images of pigs in a standard standing position as input.

[0003] Current visual algorithms lack adaptability to diverse postures: Models trained on a single standing posture struggle to effectively extract valid biometric features from the lying posture, such as spine length and trunk projection area. Existing improvements often rely on manual screening or rule-based filtering to eliminate non-standard posture images, but these suffer from two major drawbacks: First, posture filtering rules rely on empirically defined rules, are sensitive to lighting variations and animal species, and have poor generalization capabilities; second, they discard a large amount of non-standing posture data, preventing the model from learning multi-pose feature mappings, limiting algorithm robustness.

[0004] Existing technologies for utilizing multi-pose data are limited by static modeling. Most studies treat non-standard poses solely as noise and lack dynamic fusion mechanisms. For example, independent models are used to process standing and lying images, but the complementary relationship between pose features is not established. Alternatively, fixed weights are used to fuse multi-pose features, failing to dynamically adjust information contribution based on image quality and pose angle. In group-rearing scenarios, such methods are prone to estimation jumps due to pose changes or occlusion interference, resulting in high error fluctuations. Summary of the Invention

[0005] In response to the above-mentioned problems such as low data utilization caused by reliance on single-pose images, serious multi-target occlusion interference in group-raising scenarios, and insufficient cross-modal feature complementarity, the purpose of the present invention is to provide a method, system, equipment and medium for real-time estimation of the weight of unconstrained group-raised pigs, which, based on multi-pose RGB-D dynamic fusion technology, can achieve high-precision real-time estimation of the weight of pigs in complex poses.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for real-time weight estimation of unrestrained pigs in a group, comprising the following steps:

[0008] Synchronously collect color and depth images of group-housed pigs in a pig house scene;

[0009] The color images of group-raised pigs are input into the instance segmentation model for multi-pig recognition and pixel-level mask segmentation to obtain the color images of individual pigs.

[0010] The color image of the individual pig is input into the pig posture classification model, the posture category of the individual pig is judged after segmentation, and combined with the posture category of the depth image to obtain the multi-pose image of the individual pig;

[0011] The multi-pose images of individual pigs are input into the multi-pose dynamic fusion weight estimation model to perform cross-modal feature complementation and multi-pose dynamic weighted fusion to obtain the real-time weight estimation results of individual pigs.

[0012] Furthermore, the color image of the group-raised pigs is input into the instance segmentation model for multi-pig recognition and pixel-level mask segmentation to obtain the color image of the individual pigs, including:

[0013] The color images of group-raised pigs were input into a segmentation model based on Swin-Transformer to extract multi-scale feature maps.

[0014] The multi-scale feature maps are input into the region proposal network to generate a series of shared feature maps containing pig boundary candidate boxes, and redundant candidate boxes are eliminated through the non-maximum suppression algorithm;

[0015] The shared feature map is input into the model head for refined processing to obtain a color image that corresponds one-to-one to the individual pigs to be tested in the group pig house scene.

[0016] Furthermore, the multi-posture dynamic fusion weight estimation model includes a multimodal image fusion module, a multi-posture dynamic fusion module and a weight estimation module;

[0017] The multimodal image fusion module is used to fuse the color image and the depth image in the multi-pose image of the individual pig to obtain a spatial attention mask;

[0018] The multi-posture dynamic fusion module is used to dynamically fuse the multi-posture image data of the three postures of prone, side-lying and standing that are input simultaneously using the spatial attention mask as input to obtain the final multi-posture fusion feature;

[0019] The weight estimation module is used to obtain real-time weight estimation results of individual pigs using the final multi-posture fusion features as input.

[0020] Furthermore, the multimodal image fusion module includes:

[0021] A data preprocessing module is used to preprocess the received multi-pose images of individual pigs and uniformly adjust the color images and depth images of individual pigs to a fixed resolution;

[0022] The cross-modal feature mapping module takes color images of individual pigs in different postures as input and the corresponding depth images as supervision signals, and uses a pre-trained cross-modal feature mapping network to extract several intermediate feature maps containing cross-modal information;

[0023] The feature fusion and attention mechanism module is used to fuse the intermediate feature maps obtained to obtain a spatial attention mask as the feature representation after multimodal image fusion.

[0024] Furthermore, the multi-posture dynamic fusion module includes:

[0025] Input feature preprocessing module, used to extract feature maps of each posture separately;

[0026] The channel dimension fusion module is used to extract the features of each posture feature map and obtain the channel weighted coefficient of each posture through the soft maximum attention mechanism. It then combines the spatial attention mask generated in the multimodal image fusion stage to perform channel weighted fusion of the features of the prone, side-lying, and standing postures.

[0027] The spatial dimension fusion module is used to perform spatial weighted fusion of the features of each posture in the spatial dimension based on channel fusion;

[0028] The final fusion output module is used to integrate the channel-weighted and spatially weighted posture features through element-level addition to obtain the final multi-posture fusion feature output.

[0029] Furthermore, the channel dimension fusion module includes:

[0030] The global information extraction module is used to perform global average pooling and global maximum pooling on each posture feature map along the channel dimension to obtain two global description vectors;

[0031] The channel attention generation module is used to fuse the global average pooling and global maximum pooling vectors of each pose feature map in the channel dimension through element-wise addition to obtain a preliminary channel description;

[0032] The second fusion module is used to further fuse the preliminary channel descriptions of each posture feature map through element-level addition, and sequentially use 1×1 convolution, batch normalization and ReLU activation function for dimensionality reduction to obtain intermediate features;

[0033] The channel attention weight calculation module is used to map the intermediate features after dimensionality reduction to features of the specified dimension through a 1×1 convolution. The features corresponding to the three postures are respectively subjected to the soft maximum operation to obtain the channel attention weight of each posture;

[0034] The channel-weighted feature generation module uses the obtained channel attention weights of each posture and the spatial attention mask generated by the multimodal image fusion module to perform product operations on the channels of prone, side-lying and standing postures respectively to achieve channel-weighted fusion.

[0035] Furthermore, the spatial dimension fusion module includes:

[0036] The total information extraction module is used to perform element-wise addition on the channel-weighted feature maps of each posture to obtain the total information feature map;

[0037] A shared information extraction module is used to obtain a shared information feature map by performing element-wise multiplication on feature maps of any two postures;

[0038] A difference information extraction module is used to obtain a difference information feature map by performing element-wise subtraction on the feature maps of any two postures;

[0039] The feature fusion and dimensionality reduction module is used to fuse the total information feature map, the shared information feature map, and the difference information feature map through element-level addition to obtain the relationship-aware feature map. The relationship-aware feature map is processed using 1×1 convolution, batch normalization, and ReLU activation function, and then the processed feature map is mapped into spatial-level guidance features through 1×1 convolution;

[0040] The spatial attention generation and weighting module is used to perform a soft maximum operation on the spatial-level guidance features to obtain three masks, which are then point-multiplied with the channel-weighted posture features to achieve spatial weighted fusion.

[0041] In a second aspect, the present invention provides a system for real-time weight estimation of unrestrained pigs in group housing, comprising:

[0042] A multimodal image acquisition unit, used to synchronously acquire color images and depth images of group-raised pigs in a group-raised pig house scene;

[0043] The instance segmentation unit is used to input the color image of group-raised pigs into the instance segmentation model to perform multi-pig recognition and pixel-level mask segmentation to obtain color images of individual pigs;

[0044] A posture category judgment unit is used to input the color image of the individual pig into the pig posture classification model, judge the posture category of the individual pig after segmentation, and combine it with the posture category of the depth image to obtain a multi-posture image of the individual pig;

[0045] The dynamic fusion weight estimation unit is used to input the multi-pose images of individual pigs into the multi-pose dynamic fusion weight estimation model to perform cross-modal feature complementation and multi-pose dynamic weighted fusion to obtain real-time weight estimation results of individual pigs.

[0046] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions that, when executed by a computing device, enable the computing device to execute the method for real-time weight estimation of unrestrained pigs in groups.

[0047] In a fourth aspect, the present invention provides a computing device comprising: one or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, wherein the one or more programs include a method for executing the method for real-time weight estimation of unrestrained pigs in groups.

[0048] The present invention has the following advantages due to the adoption of the above technical solution:

[0049] 1. This invention innovatively integrates multimodal data and dynamic feature fusion technology to achieve accurate weight measurement in complex scenarios.

[0050] 2. The present invention synchronously collects color images and depth images of the pigs to be tested in a group-raising environment, and automatically identifies the pig's body occlusion area and posture category based on the instance segmentation algorithm of the improved Mask R-CNN architecture, effectively solving the occlusion and adhesion problem of group-raising pigs.

[0051] 3. The present invention aggregates the complementary information of side-lying, standing, and prone postures through a three-branch dynamic fusion network (F2Net) to achieve high-precision weight regression in occluded scenes.

[0052] This non-contact method overcomes the reliance of traditional visual weight estimation techniques on clear, unobstructed images. It maintains high estimation accuracy even in complex scenarios with pig occlusion, significantly reducing error compared to single-pose estimation models. Through an image restoration-pose fusion-weight regression process, it effectively addresses the feature extraction challenges associated with the dense interaction of pigs in intensive farming. Therefore, this method has broad application in the field of image processing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. Throughout the drawings, the same reference numerals are used to denote the same components. In the drawings:

[0054] Figure 1 1 is a flow chart of a method for real-time weight estimation of unrestrained pigs in group housing provided in an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of the process of multimodal image fusion provided in an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of the network structure of the cross-modal feature mapping module provided in an embodiment of the present invention;

[0057] Figure 4 is a schematic diagram of the process of multi-posture dynamic fusion provided in an embodiment of the present invention;

[0058] Figure 5 Schematic diagram of a multi-posture dynamic fusion network structure provided in an embodiment of the present invention;

[0059] Figure 6 It is a schematic diagram of the structure of the processing equipment provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0061] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0062] In some embodiments of the present invention, a real-time weight estimation method for unconstrained group-reared pigs is provided, comprising four core modules: image acquisition, individual pig instance segmentation mask generation, individual pig posture classification and judgment, and weight prediction using dynamic fusion of multi-posture features. Compared to traditional single-modality methods, this method significantly reduces weight estimation errors by compensating two-dimensional visual features with depth information and combining it with posture-adaptive dynamic weight allocation. This improves the model's generalization capabilities in complex farming scenarios and addresses the uneven contribution of different postures to weight estimation and the varying quality of postures across pigs at different growth stages.

[0063] Correspondingly, in other embodiments of the present invention, a system, device and medium for real-time weight estimation of unrestrained pigs raised in groups are provided.

[0064] Example 1

[0065] like Figure 1 As shown, this embodiment provides a method for real-time weight estimation of unrestrained pigs in a group, comprising the following steps:

[0066] 1) Synchronously collect color and depth images of group-housed pigs in a pig house scene.

[0067] In this embodiment, the color image can use the RGB color model, which is the most commonly used color model in the industry. It has three color channels and can be used to obtain the appearance characteristics of the pig, such as color, shape, and texture. The depth image is a single-channel image that captures and displays the relative or absolute distance of the object surface from the observation point. It can be used to obtain volumetric characteristics and pig posture characteristics.

[0068] In specific implementation, this embodiment uses the Microsoft DK depth camera to synchronously capture color images and depth images. In the resulting RGB-D image, the pixels of the color image and the depth image correspond one to one, and the shooting time is the same moment. The two types of images are consistent in time and space.

[0069] 2) The color images of group-raised pigs were input into an improved Mask R-CNN (Region-Based Convolutional Neural Networks) instance segmentation model for multi-pig recognition and pixel-level mask segmentation to eliminate interference from adjacent pigs and obtain pixel-level color images of individual pigs.

[0070] In group-rearing conditions, depending on the density of the pigs, the pigs may appear blocked or stuck together to varying degrees in the image when viewed from a bird's-eye view. This blocking or sticking of pigs may lead to incorrect identification of the same pig or omission of individual pigs, thus affecting the accuracy of subsequent pig weight estimation results.

[0071] Therefore, this embodiment uses an instance segmentation model based on the improved Mask R-CNN architecture to accurately segment individual pigs in a group piggery scene, distinguishing between different pigs with different weights to be estimated and their corresponding pixels. The improved Mask R-CNN instance segmentation model includes a segmentation model with Swin-Transformer as the backbone, a Region Proposal Network (RPN), and a model head. Specifically, it includes the following steps:

[0072] 2.1) Input the color images of group-raised pigs into a segmentation model based on Swin-Transformer to extract multi-scale feature maps.

[0073] In group piggery scenarios, color images often contain multiple pigs, and there may be dense occlusions between them. To effectively address this challenge, this example uses a segmentation model based on the Swin-Transformer architecture for instance segmentation. The Swin-Transformer is a model based on the Transformer architecture that, through a local window self-attention mechanism, can capture long-range dependencies in images and enhance feature expression under dense occlusion.

[0074] After the color image is fed into the segmentation model, it first performs feature extraction. Leveraging the powerful capabilities of the Swin-Transformer, multi-scale feature maps are extracted from the color image. These feature maps contain information about the color image at different scales, facilitating the subsequent accurate segmentation of individual pigs.

[0075] 2.2) The multi-scale feature maps are input into the region proposal network to generate a series of shared feature maps containing pig boundary candidate boxes. The non-maximum suppression (NMS) algorithm is then used to remove redundant candidate boxes.

[0076] In this embodiment, the RPN generates a series of candidate regions containing pig bounding boxes from the multi-scale feature map. Each candidate box corresponds to a pig in the image. To remove redundant candidate boxes, the present invention employs a non-maximum suppression (NMS) algorithm. This algorithm retains the locally optimal candidate box and eliminates other candidate boxes with high overlap, thereby reducing the computational complexity of subsequent processing.

[0077] 2.3) The shared feature map is input into the model head for refinement, obtaining an accurate mask that corresponds one-to-one to each individual pig in the group pig house scene.

[0078] In this embodiment, the model head includes two sub-modules: a classification branch and a mask branch.

[0079] The main task of the classification branch submodule is to perform binary classification on the objects within the candidate frame, that is, to determine whether the candidate frame contains individual pigs. To achieve this goal, the present invention uses a fully connected layer to process the features within the candidate frame and output a binary classification result. This binary classification result can indicate whether the candidate frame contains individual pigs, thus providing a reliable basis for subsequent mask generation.

[0080] The main task of the mask branch submodule is to generate an accurate mask that corresponds one-to-one to the individual pigs to be tested. In order to achieve this goal, the present invention first further processes the features within the candidate frame to generate a preliminary mask. Then, the preliminary mask is post-processed and optimized using morphological closing operations. Morphological closing operations can fill the holes in the mask and smooth the edges, thereby improving the accuracy and integrity of the mask. However, relying solely on morphological operations may still not be able to completely remove the noise areas in the mask. In order to solve this problem, the present invention further adopts a connected domain analysis algorithm to identify and remove smaller noise areas in the mask, and finally obtains an accurate mask that corresponds one-to-one to the individual pigs to be tested in the group pig house scene.

[0081] Furthermore, it is noteworthy that the improved Mask R-CNN architecture used in this embodiment is scalable and flexible. By adjusting model parameters and training strategies, the present invention can adapt to group-raising scenarios of different sizes and different types of individual pigs, thus achieving wide applicability.

[0082] The instance segmentation algorithm based on the improved Mask R-CNN architecture proposed in this embodiment achieves accurate segmentation of individual pigs in group-raising scenarios by introducing Swin-Transformer as the backbone network and combining the region candidate network and refined processing of the model head.

[0083] 3) The color image of the individual pig is input into the pig posture classification model trained based on the Mobilenetv3_small network. The posture category of the individual pig is judged after segmentation and combined with the posture category of the depth image to obtain the multi-pose image of the individual pig.

[0084] In this embodiment, a pre-trained pig posture classification model is used to determine posture categories. This process aims to automatically identify and analyze the different postures of individual pigs. This invention only considers the three most common postures: standing, side-lying, and prone. These provide key posture label input data for the subsequent multi-posture dynamic fusion weight estimation process.

[0085] Specifically, the pig posture classification model is built based on the Mobilenetv3_small network. Trained using a large number of pig posture samples, both manually and semi-automatically annotated, it can accurately identify the various posture characteristics of individual pigs. The model's input is a color image of an individual pig; its output is a posture label corresponding to the color image, i.e., the pig's posture category label, which includes three postures: "standing," "lying down," and "lying on its side." Because the color image and depth image are aligned, the resulting posture category labels also apply to the depth image, providing corresponding posture category labels. In this example, the obtained pig posture images are arranged and combined to form a 12-channel multi-pose image, which serves as the input for the subsequent F2Net.

[0086] In this embodiment, the pig posture classification model based on the color image as input obtains the posture category of the individual pig after segmentation, which can more realistically reflect the posture of the pig to be tested and provide prior information for subsequent weight estimation.

[0087] 4) The multi-pose images of individual pigs are input into the multi-pose dynamic fusion weight estimation model to perform cross-modal feature complementation and multi-pose dynamic weighted fusion to obtain the real-time weight estimation results of individual pigs.

[0088] In this example, the multi-pose images of individual pigs, obtained through preliminary processing, are fed into the F2Net model for dynamic multi-pose weight estimation. This model dynamically fuses information from RGB-D images of the same target pig in different poses to produce a final weight estimate. This process fully leverages the complementary information in multimodal images and multi-pose images, improving the accuracy and robustness of weight estimation. Through its unique network structure and algorithm, the F2Net model automatically adapts to the characteristics of pigs in different poses.

[0089] Specifically, the multi-posture dynamic fusion weight estimation model includes a multimodal image fusion module, a multi-posture dynamic fusion module, and a weight estimation module. The multimodal image fusion module is used to fuse the color image and depth image in the multi-posture image to obtain a spatial attention mask, which serves as the feature representation of the fused multimodal image. The multi-posture dynamic fusion module is used to dynamically fuse the multi-posture image data of the three postures of standing, lying on the side, and lying on the stomach, taking the spatial attention mask as input, to obtain the final multi-posture fusion feature. Dynamic fusion refers to setting learnable weights for the features of the channels of different postures. The weight estimation module is used to obtain the real-time weight estimation result of the target individual pig using the final multi-posture fusion feature as input.

[0090] Furthermore, in view of the actual application requirements in pig farm monitoring, where illumination changes dramatically and color images are greatly affected by illumination, while depth images are not affected by illumination changes and contain rich three-dimensional structural information, this embodiment constructs a multimodal image fusion mechanism based on self-supervised learning to fully utilize the complementary advantages of color images and depth images, thereby improving the perception of pig posture.

[0091] like Figure 2 As shown in Figure 2, the multimodal image fusion module includes:

[0092] The data preprocessing module is used to preprocess the received multi-pose images of individual pigs (a total of 12 channels) and uniformly adjust the color image and depth image of each pig to a fixed resolution.

[0093] The cross-modal feature mapping module is used to take color images of individual pigs in different postures as input, use the corresponding depth images as supervision signals, and adopt a pre-trained cross-modal feature mapping network to extract several intermediate feature maps containing cross-modal information.

[0094] The feature fusion and attention mechanism module is used to fuse the intermediate feature maps obtained to obtain a spatial attention mask, which serves as the feature representation after multimodal image fusion, and is further used by the subsequent cross-modal feature mapping module for feature extraction and pig posture discrimination.

[0095] Furthermore, if Figure 3 As shown in the figure, in the cross-modal feature mapping module, the cross-modal feature mapping network uses the UNet structure as the basic framework and Mobilenetv3_large as the backbone network to perform encoding and decoding operations on color images. Specifically, the cross-modal feature mapping network takes preprocessed color images as input and depth images as the target output. The network is trained through self-supervised learning, enabling it to learn the nonlinear mapping from color images to depth images and then extract cross-modal information. After network training is complete, the trained network is used to extract intermediate feature maps containing cross-modal information, achieving the fusion of color and depth images.

[0096] Furthermore, the feature fusion and attention mechanism module includes a pooling layer, a convolution layer, and a normalization layer. The pooling layer is used to perform global average pooling and global maximum pooling on each intermediate feature map output by the cross-modal feature mapping module, and then concatenate the two resulting feature vectors in the channel dimension; the convolution layer is used to convolve the concatenated features using a 3×3 convolution; and the normalization layer uses a sigmoid function to generate a spatial attention mask as the feature representation after multimodal image fusion.

[0097] In this embodiment, by mapping color images into depth images and using the spatial attention mechanism to extract cross-modal complementary information, the impact of lighting changes on image acquisition in pig farm monitoring is successfully solved, and the utilization effect of pig posture features is significantly improved.

[0098] Furthermore, if Figure 4 and Figure 5 As shown, the multi-posture dynamic fusion module includes:

[0099] Input feature preprocessing module is used to extract feature maps of each posture separately.

[0100] The channel dimension fusion module is used to extract the features of each posture feature map and obtain the channel weighted coefficient of each posture through the softmax attention mechanism. It is then combined with the spatial attention mask generated from the multimodal image fusion stage to perform channel-weighted fusion of the features of the prone, side-lying, and standing postures.

[0101] The spatial dimension fusion module is used to perform weighted fusion of the features of each posture in the spatial dimension based on channel fusion.

[0102] The final fusion output module is used to integrate the channel-weighted and spatially weighted posture features through element-level addition to obtain the final multi-posture fusion feature output. The multi-posture fusion feature can be used as the input for subsequent weight estimation or other downstream tasks.

[0103] Furthermore, the channel dimension fusion module includes:

[0104] The global information extraction module is used to perform global average pooling and global maximum pooling on each posture feature map along the channel dimension to obtain two global description vectors. Through the above pooling operation, more global information related to the posture can be retained, and the expression of posture-specific features can be enhanced;

[0105] The channel attention generation module is used to fuse the global average pooling and global maximum pooling vectors of each pose feature map in the channel dimension through element-wise addition to obtain a preliminary channel description;

[0106] The second fusion module is used to further fuse the preliminary channel descriptions of each posture feature map through element-level addition, and sequentially use 1×1 convolution, batch normalization and ReLU activation function for dimensionality reduction to obtain intermediate features;

[0107] The channel attention weight calculation module is used to map the intermediate features after dimensionality reduction to features of the specified dimension through a 1×1 convolution. The features corresponding to the three postures are respectively subjected to a softmax operation to obtain the channel attention weight of each posture;

[0108] The channel-weighted feature generation module uses the obtained channel attention weights and the spatial attention mask generated by the multimodal image fusion module to perform weighted operations on the feature maps of each posture, and performs product operations on the channels of the prone, side-lying and standing postures respectively. The product operation can ensure that the features of each posture can be adaptively adjusted according to the global channel information, and at the same time, the spatial attention information is combined to further enhance the features.

[0109] Furthermore, in the spatial dimension fusion module, by calculating the total information, shared information, and difference information between the feature maps of each posture, the complementary features between multiple postures are extracted, and the spatial weighting coefficient is obtained through the soft maximum attention mechanism to complete the final dynamic fusion. Specifically, this module includes:

[0110] The total information extraction module is used to perform element-wise addition on the channel-weighted feature maps of each posture to obtain the total information feature map;

[0111] A shared information extraction module is used to obtain a shared information feature map by performing element-wise multiplication on feature maps of any two postures;

[0112] A difference information extraction module is used to obtain a difference information feature map by performing element-wise subtraction on the feature maps of any two postures;

[0113] The feature fusion and dimensionality reduction module is used to fuse the total information feature map, the shared information feature map, and the difference information feature map through element-level addition to obtain the relationship perception feature map. The relationship perception feature map is then processed using 1×1 convolution, batch normalization, and ReLU activation function to achieve the purpose of dimensionality reduction. The processed feature map is then mapped into spatial-level guidance features through 1×1 convolution;

[0114] The spatial attention generation and weighting module performs a softmax operation on the spatial guidance features to obtain three masks. These masks are then dot-multiplied with the channel-weighted pose features to achieve spatial weighting, and the resulting spatially fused feature maps of the three poses are output. This spatial weighting operation enables the adaptive fusion of complementary spatially distributed information from different poses.

[0115] The multi-pose dynamic fusion method implemented through the above implementation has the following advantages: dynamic adaptive weighting, combining attention mechanisms at both the channel and spatial levels, can adaptively assign weights based on the feature quality and global information of different poses, giving the standard pose a higher weight and improving the overall feature expression effect; noise and uncertainty suppression, through differential information extraction and multi-level information fusion, fully utilizing the complementary relationship between poses, effectively reducing the impact of noise and uncertainty, and improving the robustness of subsequent tasks. Rich feature interaction, the dynamic fusion process simultaneously considers global information, shared information, and local differential information, achieving deep interaction of multi-dimensional features, providing a more accurate feature foundation for subsequent pig weight estimation or behavior recognition.

[0116] The multi-posture dynamic fusion technology proposed in this example first extracts global posture features and generates channel attention in the channel dimension. It then models the complementary relationships between the three postures in the spatial dimension. Using a soft-max attention mechanism, it obtains channel and spatial weighting coefficients, ultimately achieving adaptive fusion of posture features for stomach-recumbent, side-recumbent, and standing. This method effectively addresses the uneven contribution of different postures to weight estimation and the varying quality of postures at different growth stages.

[0117] Example 2

[0118] The above-mentioned embodiment 1 provides a method for real-time estimation of the weight of unrestrained pigs raised in groups. Correspondingly, this embodiment provides a system for real-time estimation of the weight of unrestrained pigs raised in groups. The system provided by this embodiment can implement the real-time estimation method for real-time estimation of the weight of unrestrained pigs raised in groups of embodiment 1. The system can be implemented by software, hardware, or a combination of software and hardware. For example, the system can include integrated or separate functional modules or functional units to execute the corresponding steps in each method of embodiment 1. Since the system of this embodiment is basically similar to the method embodiment, the process described in this embodiment is relatively simple. For relevant points, please refer to the partial description of embodiment 1. The embodiment of the system provided in this embodiment is merely illustrative.

[0119] The system for real-time weight estimation of unrestrained pigs in group housing provided in this embodiment includes:

[0120] A multimodal image acquisition unit, used to synchronously acquire color images and depth images of group-raised pigs in a group-raised pig house scene;

[0121] The instance segmentation unit is used to input the color image of group-raised pigs into the improved Mask R-CNN instance segmentation model for multi-pig recognition and pixel-level mask segmentation to eliminate interference from adjacent pigs and obtain pixel-level color images of individual pigs;

[0122] The posture category judgment unit is used to input the color image of the individual pig into the pig posture classification model trained based on the Mobilenetv3_small network, judge the posture category of the individual pig after segmentation, and combine it with the posture category of the depth image to obtain a multi-pose image of the individual pig;

[0123] The dynamic fusion weight estimation unit is used to input the multi-pose images of individual pigs into the multi-pose dynamic fusion weight estimation model to perform cross-modal feature complementation and multi-pose dynamic weighted fusion to obtain real-time weight estimation results of individual pigs.

[0124] Example 3

[0125] This embodiment provides a processing device corresponding to the real-time weight estimation method for group-raised, unrestrained pigs provided in this embodiment 1. The processing device can be a processing device for a client, such as a mobile phone, laptop computer, tablet computer, desktop computer, etc., to execute the method of embodiment 1.

[0126] like Figure 6 As shown, the processing device in this embodiment may include: a processor, a communications interface, a memory, and a communications bus. The processor, the communications interface, and the memory communicate with each other via the communications bus. The processor may call logic instructions in the memory to execute the method for estimating the weight of group-raised pigs.

[0127] In addition, the logical instructions in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] Example 4

[0129] The method for real-time estimation of the weight of unrestrained pigs in groups of this embodiment 1 can be specifically implemented as a computer program product, which may include a computer-readable storage medium carrying computer-readable program instructions for executing the method for real-time estimation of the weight of unrestrained pigs in groups of this embodiment 1.

[0130] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0131] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A method for real-time weight estimation of unrestrained pigs in group housing, characterized in that: The following steps are involved: Synchronously collect color and depth images of group-housed pigs in a pig house scene; The color images of group-raised pigs are input into the instance segmentation model for multi-pig recognition and pixel-level mask segmentation to obtain the color images of individual pigs. The color image of the individual pig is input into the pig posture classification model, the posture category of the individual pig is judged after segmentation, and combined with the posture category of the depth image to obtain the multi-pose image of the individual pig; The multi-pose images of individual pigs are input into the multi-pose dynamic fusion weight estimation model to perform cross-modal feature complementation and multi-pose dynamic weighted fusion to obtain the real-time weight estimation results of individual pigs. The multi-posture dynamic fusion weight estimation model includes a multi-modal image fusion module, a multi-posture dynamic fusion module and a weight estimation module; The multimodal image fusion module is used to fuse the color image and the depth image in the multi-pose image of the individual pig to obtain a spatial attention mask; The multi-posture dynamic fusion module is used to dynamically fuse the multi-posture image data of the three postures of prone, side-lying and standing that are input simultaneously, taking the spatial attention mask as input, to obtain the final multi-posture fusion feature; The weight estimation module is used to obtain a real-time weight estimation result of the individual pig to be tested by taking the final multi-posture fusion feature as input.

2. A method for real-time weight estimation of unrestrained pigs in group housing according to claim 1, characterized in that: The color image of group-raised pigs is input into the instance segmentation model for multi-pig recognition and pixel-level mask segmentation to obtain color images of individual pigs, including: The color images of group-raised pigs were input into a segmentation model based on Swin-Transformer to extract multi-scale feature maps. The multi-scale feature maps are input into the region proposal network to generate a series of shared feature maps containing pig boundary candidate boxes, and redundant candidate boxes are eliminated through the non-maximum suppression algorithm; The shared feature map is input into the model head for refined processing to obtain a color image that corresponds one-to-one to the individual pigs to be tested in the group pig house scene.

3. A method for real-time weight estimation of unrestrained pigs in group housing according to claim 1, characterized in that: The multimodal image fusion module includes: A data preprocessing module is used to preprocess the received multi-pose images of individual pigs and uniformly adjust the color images and depth images of individual pigs to a fixed resolution; The cross-modal feature mapping module takes color images of individual pigs in different postures as input and the corresponding depth images as supervision signals, and uses a pre-trained cross-modal feature mapping network to extract several intermediate feature maps containing cross-modal information; The feature fusion and attention mechanism module is used to fuse the intermediate feature maps obtained to obtain a spatial attention mask as the feature representation after multimodal image fusion.

4. A method for real-time weight estimation of unrestrained pigs in group housing according to claim 1, characterized in that: The multi-posture dynamic fusion module includes: Input feature preprocessing module, used to extract feature maps of each posture separately; The channel dimension fusion module is used to extract the features of each posture feature map and obtain the channel weighting coefficient of each posture through the softmax function. It then combines the spatial attention mask generated in the multimodal image fusion stage to perform channel weighted fusion of the features of the prone, side-lying, and standing postures; The spatial dimension fusion module is used to perform spatial weighted fusion of the features of each posture in the spatial dimension based on channel fusion; The final fusion output module is used to integrate the channel-weighted and spatially weighted posture features through element-level addition to obtain the final multi-posture fusion feature output.

5. A method for real-time weight estimation of unrestrained pigs in group housing according to claim 4, characterized in that: The channel dimension fusion module includes: The global information extraction module is used to perform global average pooling and global maximum pooling on each posture feature map along the channel dimension to obtain two global description vectors; The channel attention generation module is used to fuse the global average pooling and global maximum pooling vectors of each pose feature map in the channel dimension through element-wise addition to obtain a preliminary channel description; The second fusion module is used to further fuse the preliminary channel descriptions of each posture feature map through element-level addition, and sequentially use 1×1 convolution, batch normalization and ReLU activation function for dimensionality reduction to obtain intermediate features; The channel attention weight calculation module is used to map the intermediate features after dimensionality reduction to features of the specified dimension through a 1×1 convolution. The features corresponding to the three postures are respectively applied with the softmax function to obtain the channel attention weight of each posture; The channel-weighted feature generation module uses the obtained channel attention weights of each posture and the spatial attention mask generated by the multimodal image fusion module to perform product operations on the channels of prone, side-lying and standing postures respectively to achieve channel-weighted fusion.

6. A method for real-time weight estimation of unrestrained pigs in group housing as claimed in claim 4, characterized in that: The spatial dimension fusion module includes: The total information extraction module is used to perform element-wise addition on the channel-weighted feature maps of each posture to obtain the total information feature map; A shared information extraction module is used to obtain a shared information feature map by performing element-wise multiplication on feature maps of any two postures; A difference information extraction module is used to obtain a difference information feature map by performing element-wise subtraction on the feature maps of any two postures; The feature fusion and dimensionality reduction module is used to fuse the total information feature map, the shared information feature map, and the difference information feature map through element-level addition to obtain the relationship-aware feature map. The relationship-aware feature map is processed using 1×1 convolution, batch normalization, and ReLU activation function, and then the processed feature map is mapped into spatial-level guidance features through 1×1 convolution; The spatial attention generation and weighting module is used to apply the softmax function to the spatial-level guidance features to obtain three masks, which are then point-multiplied with the channel-weighted posture features to achieve spatial weighted fusion.

7. A real-time weight estimation system for unrestrained pigs raised in groups, characterized by: include: A multimodal image acquisition unit, used to synchronously acquire color images and depth images of group-raised pigs in a group-raised pig house scene; The instance segmentation unit is used to input the color image of group-raised pigs into the instance segmentation model to perform multi-pig recognition and pixel-level mask segmentation to obtain color images of individual pigs; A posture category judgment unit is used to input the color image of the individual pig into the pig posture classification model, judge the posture category of the individual pig after segmentation, and combine it with the posture category of the depth image to obtain a multi-posture image of the individual pig; The dynamic fusion weight estimation unit is used to input the multi-pose images of individual pigs into the multi-pose dynamic fusion weight estimation model to perform cross-modal feature complementation and multi-pose dynamic weighted fusion to obtain the real-time weight estimation results of individual pigs; The multi-posture dynamic fusion weight estimation model includes a multi-modal image fusion module, a multi-posture dynamic fusion module and a weight estimation module; The multimodal image fusion module is used to fuse the color image and the depth image in the multi-pose image of the individual pig to obtain a spatial attention mask; The multi-posture dynamic fusion module is used to dynamically fuse the multi-posture image data of the three postures of prone, side-lying and standing that are input simultaneously, taking the spatial attention mask as input, to obtain the final multi-posture fusion feature; The weight estimation module is used to obtain a real-time weight estimation result of the individual pig to be tested by taking the final multi-posture fusion feature as input.

8. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 6 .

9. A computing device, characterized in that include: One or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, wherein the one or more programs include instructions for executing any one of the methods according to claims 1 to 6.

Images