A captive animal weight prediction method based on environment and visual feature fusion

Abstract

The present application relates to the technical field of intelligent breeding, and particularly relates to a captive animal weight prediction method based on environment and visual feature fusion. The method comprises the following steps: synchronously collecting environment parameters, image data and weight labels, and constructing a triple data set; preprocessing the triple data set, and constructing a mapping relationship of environment parameters-single frame image-corresponding single weight of captive animals based on a timestamp and a preset spatial coordinate, and generating multi-modal training data; inputting the environment parameters in the multi-modal training data into an environment-image feature fusion module in a CNN-Transformer hybrid architecture which is pre-trained, and performing dimension matching with image features to obtain a fusion feature training model. Through constructing an environment and visual feature deep coupling, combining a time and space mapping, a multi-task constraint and a cross-scene adaptive mechanism, the present application realizes high precision, stability and generalization ability of captive animal weight prediction.

Description

Technical Field

[0001] This invention relates to the field of intelligent aquaculture technology, and in particular to a method for predicting the weight of captive animals based on the fusion of environmental and visual features. Background Technology

[0002] Traditional manual weighing methods rely on contact measurement and empirical estimation, approximating weight using empirical formulas. However, these methods are highly dependent on manual operation and suffer from poor data consistency, making them unsuitable for the real-time management needs of large-scale pig farms. Image regression methods based on monocular or binocular vision have gradually emerged, using convolutional neural networks to extract body contour and posture features from single-frame images, achieving non-contact weight estimation. However, these methods generally treat environmental factors as noise, failing to model key environmental parameters such as lighting, humidity, pen material, and ground reflection, leading to significantly increased errors when the model is deployed across different scenarios. The application of the Transformer structure in visual representation has significantly enhanced the ability to acquire multi-scale features, but environmental differences between different pig farms still result in inconsistent visual feature distributions, leading to insufficient model generalization ability. Some studies have attempted to introduce environmental labels as auxiliary variables, but most remain at the stage of simple splicing or fixed-weight fusion, failing to construct a dynamically adaptive environment-visual joint representation; at the same time, training data is mostly collected from single scenarios, lacking large-scale triplet data across pig farms and multiple time periods, and lacking guarantees of the reliability of training data. Summary of the Invention

[0003] Therefore, it is necessary for the present invention to provide a method for predicting the weight of captive animals based on the fusion of environmental and visual features, in order to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, a method for predicting the weight of captive animals based on the fusion of environmental and visual features includes the following steps:

[0005] Step S1: Synchronously collect environmental parameters, image data, and weight labels, and construct a triplet dataset;

[0006] Step S2: Preprocess the triplet dataset and construct a mapping relationship between environmental parameters, single-frame images, and the weight of the corresponding captive animal based on timestamps and preset spatial coordinates, and generate multimodal training data;

[0007] Step S3: Input the environmental parameters from the multimodal training data into the environment-image feature fusion module in the pre-trained CNN-Transformer hybrid architecture, perform dimensional matching with the image features, and obtain the fused feature training model;

[0008] Step S4: Design a multi-task loss function, pre-train the fusion feature training model, and adjust the parameters according to different scenarios to obtain a weight prediction model;

[0009] Step S5: Verify the environmental adaptability of the weight prediction model and perform error analysis to generate error optimization data; update the weight prediction model parameters based on the error optimization data to obtain the optimized weight prediction model.

[0010] This invention establishes a deep coupling mechanism between environmental parameters and visual features during weight prediction, enabling quantifiable representation of multi-source environmental factors such as light intensity, pen material, humidity fluctuations, and reflection interference, thus maintaining the stability of visual features in complex scenarios. A mapping relationship established based on timestamps and spatial coordinates ensures a strict correspondence between environmental records, keyframe images, and weight labels for the same captive animal, improving the consistency and reliability of training samples. The fusion of CNN and Transformer structures allows for the simultaneous acquisition of local body contours and global pose distribution, giving the model a more complete ability to express body structure. A multi-task loss structure introduces weight regression, body parameter constraints, and environmental consistency constraints, ensuring that the training process simultaneously considers visual features, environmental representation, and body semantics, improving prediction accuracy in complex scenarios. Cross-scenario parameter adjustment and error decomposition mechanisms enable the model to adaptively converge to differences in different farm environments, reducing cross-scenario prediction bias. Automatic parameter correction is achieved through error back-mapping and environmental modulation recalibration, giving the model continuous optimization capabilities during long-term deployment, ultimately significantly improving the stability, generalization ability, and environmental adaptability of captive animal weight prediction. Attached Figure Description

[0011] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0012] Figure 1 This is a schematic diagram of the steps of a method for predicting the weight of captive animals based on the fusion of environmental and visual features according to the present invention.

[0013] Figure 2 A schematic diagram of the algorithm framework for estimating the size of the model. Detailed Implementation

[0014] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0015] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0016] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0017] To achieve the above objectives, please refer to Figures 1 to 2 This invention provides a method for predicting the weight of captive animals based on the fusion of environmental and visual features. The method includes the following steps:

[0018] Step S1: Synchronously collect environmental parameters, image data, and weight labels, and construct a triplet dataset;

[0019] Step S2: Preprocess the triplet dataset and construct a mapping relationship between environmental parameters, single-frame images, and the weight of the corresponding captive animal based on timestamps and preset spatial coordinates, and generate multimodal training data;

[0020] Step S3: Input the environmental parameters from the multimodal training data into the environment-image feature fusion module in the pre-trained CNN-Transformer hybrid architecture, perform dimensional matching with the image features, and obtain the fused feature training model;

[0021] Step S4: Design a multi-task loss function, pre-train the fusion feature training model, and adjust the parameters according to different scenarios to obtain a weight prediction model;

[0022] Step S5: Verify the environmental adaptability of the weight prediction model and perform error analysis to generate error optimization data; update the weight prediction model parameters based on the error optimization data to obtain the optimized weight prediction model.

[0023] Preferably, step S1 includes:

[0024] Data such as light intensity, enclosure type, temperature and humidity, and camera installation parameters are collected by sensors and cameras and then uniformly quantified and encoded to form a structured set of environmental parameters.

[0025] The postures of captive animals and the area covered by the image are collected by the end-side camera device, and key frames are selected to form an image dataset;

[0026] The weight of captive animals was collected by manual weighing and weighing before slaughter, and weight tags were attached to obtain a set of weight tags;

[0027] The environmental parameters, image data, and weight labels are aligned in a three-way manner to obtain a triplet dataset.

[0028] In this embodiment of the invention, environmental data is collected using various types of sensors installed inside the enclosure. Light intensity is read by a lux meter fixed to the ceiling beam, with a sampling frequency of twice per second, and light intensity recorded in lux. The enclosure type is coded using a preset enclosure structure identification table, which includes three types of structures: enclosed, open, and semi-enclosed, each with a fixed structure code value. Temperature and humidity are collected by a temperature and humidity sensor composed of a thermistor and a humidity-sensitive capacitor; temperature is recorded in degrees Celsius and limited to -10 degrees Celsius. Up to 50 Humidity is recorded as a percentage and limited to 0% to 100%; camera installation parameters are quantified via a mounting bracket, including installation height (limited to 1.8m to 2.4m) and installation lateral angle (limited to 0°). Up to 30 ) and pitch angle (limited to 15) Up to 35 All the above environmental parameters are constructed into a fixed-length structured environmental parameter set according to a unified format. Each item in the environmental parameter set is numerically encoded and mapped to the range of 0 to 1 through a linear normalization method, so that subsequent steps can directly process the parameter set based on a unified scale.

[0029] Subsequently, footage of the captive animals was captured using end-view cameras positioned in the center of the enclosure. The cameras were fixed-resolution industrial cameras set at 1920 pixels. The resolution is 1080p, and the frame rate is set to 30 frames per second. To avoid redundant frames affecting data processing, a keyframe filtering process is performed based on the change in the image coverage area: the difference in the foreground area of ​​consecutive frames is compared with a preset threshold, which is set to 5% of the image coverage area. If the change exceeds the threshold, the frame is determined to be a keyframe; if the change is less than the threshold, the frame is skipped. The keyframe filtering logic ensures that the acquired image dataset includes images of significant changes in animal posture, such as turning around, standing up, and eating, thus ensuring sufficient coverage of posture diversity in the image dataset.

[0030] Weight tags are obtained in two ways: manual weighing inside the pens using electronic scales with a fixed capacity of 300 kg and an accuracy of 0.1 kg; and weighing before slaughter using an automatic weighing trough, performed once before each slaughter and recorded as the final reference value. To ensure the accuracy of the weight tags, the manual weighing and the slaughter weighing are compared. If the difference between the two weighing results is within 3%, the average value is used as the final weight tag; if the difference exceeds 3%, the slaughter weighing result is used. All weight tags are grouped into a set in kg.

[0031] After independently constructing the environmental parameter set, image dataset, and weight tag set, a three-way alignment process is performed. The three-way alignment uses a timestamp-and-spatial-location-based correspondence, with all sensor and camera acquisition times based on the same clock, and timestamp precision set to milliseconds. First, the timestamps of image frames are matched with the environmental parameter acquisition times. The environmental parameter values ​​at that moment are determined by finding the two closest time nodes and using linear interpolation. Then, the weight tags corresponding to the synchronization times are associated with the unique ear tag codes of individual animals. After these alignment steps, each record sequentially contains structured environmental parameters, keyframe images, and weight tags, forming a triplet dataset.

[0032] Preferably, after aligning the environmental parameters, image data, and weight tags, the process further includes:

[0033] The triplet dataset is verified by blockchain hash and anchored by linked timestamps to obtain a trusted triplet dataset.

[0034] The environmental text descriptions in the triplet trusted dataset are text-encoded, and the fence type, lighting level and camera installation parameters are converted into 128-dimensional text embedding vectors to obtain the environmental parameter embedding matrix.

[0035] Based on the ViT environment injection mechanism, a dual-channel gating structure is constructed by embedding the environment parameter matrix to obtain the text weight channel and the visual weight channel, and the sum of the text weight channel and the visual weight channel is 1.

[0036] When the light intensity is less than 500 lux, the text weight is automatically set to be greater than 0.7, resulting in environmental weight configuration data with low light intensity adaptability.

[0037] In this embodiment of the invention, after completing the three-way alignment of environmental parameters, image data, and weight tags, a complete data trust construction process is performed on the triplet dataset. The first stage of processing is blockchain hash verification. A fixed SHA256 hash function is used to operate on the byte stream of each triplet data serialization. The byte stream format strictly includes a fixed concatenation structure of 32 bytes of environmental parameters, 16 bytes of image frame index, and 8 bytes of weight tag, with a total length of 56 bytes. After performing a hash operation on this 56-byte input, a 32-byte hash value is generated and immediately written to consecutive blocks in the distributed ledger. The block contains the hash value field of the previous block and is connected in a chain structure to form a linked timestamp anchor. The timestamp is generated by a unified time synchronization module with a precision limited to milliseconds. Through the above fixed-order hash operation and chain linking, a trusted triplet dataset is constructed. This dataset maintains a strict correspondence with the original triplet data and can be verified for data modification behavior through block-by-block verification.

[0038] After the data credibility is established, the environmental text descriptions in the triplet credibility dataset are text-encoded. In this implementation, the enclosure type, illumination level, and camera installation parameters are all described using structured text templates. The enclosure type uses three fixed terms: closed enclosure, semi-closed enclosure, and open enclosure; the illumination level uses five terms after Lux partitioning: low illumination (0 to 300 lux), low to medium illumination (300 to 500 lux), medium illumination (500 to 800 lux), medium to high illumination (800 to 1200 lux), and high illumination (above 1200 lux); the camera installation parameters use "installation height". lateral angle The pitch angle is in "pitch angle" format, with a numerical range of 1.8m to 2.4m in height and 0m in lateral angle. Up to 30 Pitch angle 15 Up to 35 The above text description is input into a static word vector construction program, which converts the text description into a 128-dimensional vector using a fixed-dimensional linear transformation matrix. The linear transformation matrix has a dimension of M. 128, where M is the dimension of the sparse vector of the text description after dictionary indexing. This step generates an environment parameter embedding matrix, which is a fixed-length 128-dimensional row vector.

[0039] Next, a dual-channel gating structure is constructed based on the ViT environment injection mechanism. This implementation generates text weight channels and visual weight channels through two sets of independent weight generation functions. Both sets of functions employ a weighted summation structure and do not contain loops or conditional bifurcations. The text weight channel is generated by performing a dot product between the environment parameter embedding matrix and a fixed weight vector to obtain a scalar. Then, through linear normalization, Mapped to the 0-1 interval. The method for generating visual weight channels is exactly the same as the previous method, generating scalars. Text weights and visual weights within the same triple are processed during the normalization phase. and The operation ensures that the sum of the two is strictly equal to 1. The above dual-channel gating structure represents the relative intensity of the text channel and the visual channel through two mutually compensating channels.

[0040] Under low-light conditions, it is necessary to build low-light adaptive capabilities for environmental parameters. To this end, this implementation performs a threshold judgment on light intensity. When the light value recorded in the trusted triplet dataset is below 500 lux, the text weight is directly increased to greater than 0.7 through a linear overlay method. The specific steps are as follows: If the light intensity... If the value is 500 lux, then text weighting will be applied. illumination This expression ensures that the lower the illumination, the higher the text weight, ultimately limiting the text weight to no more than 1, and synchronously adjusting the visual weight according to the aforementioned rule that the sum of the channels is 1. This process generates environmental weight configuration data with low-light adaptability.

[0041] Preferably, step S1 is followed by:

[0042] Data sources from multiple farms are obtained, and the data is grouped by scenario. Federated learning node identifiers are configured for each scenario to obtain a cross-scenario participant index.

[0043] A preliminary analysis was conducted on the differences in data scale, distribution, and environmental characteristics among different farms. The gradient compression sparsity rate was set to 95%, and a pre-parameter table for federated training was obtained.

[0044] Statistical analysis of environmental differences in the triplet credible dataset is performed to calculate the theoretical lower bound of environmental generalization error, resulting in a generalization error benchmark table.

[0045] In this embodiment of the invention, after constructing the triplet trusted dataset, data sources from multiple farms are first acquired to establish a cross-farm data collaboration structure. In this embodiment, the number of farms is set to four, named Farm Area A, Farm Area B, Farm Area C, and Farm Area D. The environmental parameter acquisition equipment and end-side camera equipment for each farm area are uniformly calibrated to ensure consistent light sensor range (0 to 2000 lux) and consistent temperature and humidity sensor accuracy (…). 0.3 , (2%RH). To construct a cross-scenario participant index, the four farms were grouped based on geographical location, pen structure type, and camera installation differences. Geographical location was grouped by latitude: farms A and B were assigned to the Southern group, and farms C and D to the Northern group. Pen structure codes were statistically analyzed according to the three categories of closed, semi-closed, and open structures defined in step S1. Camera installation parameters were clustered based on the differences in height, lateral angle, and pitch angle; differences exceeding 10% were considered different scenarios. Using these three-dimensional grouping rules, a unique federated learning node identifier was generated for each farm, such as "FL-A01", "FL-B02", "FL-C03", and "FL-D04". These identifiers were written into the cross-scenario participant index table in a fixed format.

[0046] Subsequently, a pre-analysis was performed on the data size and distribution differences among different farms. Data size was determined by the number of reliable data points in triplet sets; for example, farm A had 120,000 data points, farm B had 150,000, farm C had 80,000, and farm D had 100,000. Distribution differences were assessed by comparing the statistical variances of light intensity, humidity, and camera installation angle, with the light intensity variance limited to [missing information]. The following values ​​represent the low-difference group; values ​​exceeding these values ​​represent the high-difference group. Humidity variance below 50% is considered low-difference; camera pitch angle differences exceeding 10° are also considered low-difference. The high-discrepancy group is defined as follows: Based on the three discrepancy indices mentioned above, region B is classified as a high-discrepancy region, while the remaining regions are classified as medium-discrepancy regions. In this implementation, the gradient compression sparsity rate is set to 95%, which is determined by a fixed sparse dimension rule, meaning that only the top 5% of the gradients uploaded by each node in each round, representing the largest numerical amplitude, are retained. The sparsity rate, along with the discrepancy evaluation results, is written into the federated training pre-parameter table.

[0047] After constructing the pre-training parameter table for federated training, the environmental differences in the triplet trusted dataset were statistically analyzed. The statistical process was based on the illumination, humidity, enclosure type encoding, and camera installation encoding for each record in the triplet trusted dataset. Illumination was statistically distributed in three segments: 0-500 lux, 500-1000 lux, and 1000-1500 lux; humidity was statistically distributed in three segments: 0-40%, 40-70%, and 70-100%; enclosure type was recorded in three frequency categories: enclosed, semi-enclosed, and open; and camera installation angle was recorded in three categories: horizontal angle 0°. Up to 15 With 15 Up to 30 Pitch angle 15 Up to 25 With 25 Up to 35 Regional statistics are performed. After the statistics are completed, the theoretical lower limit of the environmental generalization error is calculated using the theoretical lower limit of error. In this embodiment, the theoretical lower limit is calculated as the product of the environmental distribution difference and the empirical error factor, with the empirical error factor set to 0.02. The environmental distribution difference is represented by the KL distance values ​​of each parameter distribution. The theoretical lower limits of the four parameters—light intensity, humidity, enclosure type, and camera installation angle—are summed to generate the total theoretical error value. A generalization error benchmark table is generated through the above process.

[0048] Preferably, step S2 includes:

[0049] Based on timestamps and preset spatial coordinates, a mapping triplet dataset of environmental parameters, single-frame images, and weight labels is constructed using a triplet dataset.

[0050] The mapped triplet dataset is divided into multimodal structures, environmental parameters are encoded into structured environmental vectors, image data is formatted into visual input tensors, and weight labels are converted into supervision signals to obtain a structured multimodal dataset.

[0051] Based on the environmental parameter embedding matrix and environmental weight configuration data, the 128-dimensional text embedding vector is aligned with the visual input dimension to obtain the environment-visual alignment data before FiLM injection.

[0052] Based on a structured multimodal dataset, a dual-channel gating mechanism is invoked on the environment-visual alignment data. By dynamically and complementaryly configuring the text channel weights and visual channel weights, FiLM-injectable multimodal fusion pre-data is obtained.

[0053] Based on the preset input requirements, tensor quantization training batches are determined using the data before multimodal fusion, and multimodal training data is obtained.

[0054] In this embodiment of the invention, after constructing the trusted triplet dataset, the mapping and association of environmental parameters, image frames, and weight labels are completed based on the millisecond-level timestamps of the unified time synchronization system and the barracks coordinate system. The barracks coordinate system uses a fixed two-dimensional coordinate grid with a grid density of 0.1m. Each camera's shooting area has a unique spatial number, with a depth of 0.1m. Using the acquisition time of each image data point in the triplet trusted dataset as the primary key, the environmental parameter value at that time point is calculated using linear interpolation by finding the two closest environmental parameter records (with a time difference limited to within 200 milliseconds). Weight labels are matched using unique ear tag numbers, ensuring unbiased alignment with the spatial location of the image frame, thus constructing a complete mapping triplet dataset of environmental parameters, single-frame images, and weight labels.

[0055] Subsequently, a multimodal structure partitioning was performed on the mapped triplet dataset. Environmental parameters were encoded using structured fields, and each parameter underwent normalization: illumination value divided by 2000 yielded a value between 0 and 1; humidity was divided by 100 as a percentage, yielding a value between 0 and 1; enclosure type was mapped to a one-hot encoding of length 3 using a fixed dictionary; camera installation height was divided by 3, yielding a value between 0 and 1; and lateral and pitch angles were each divided by 45, yielding values ​​between 0 and 1. All fields were concatenated in a fixed order to form a structured environmental vector of length 12. Image data was acquired using industrial cameras at a fixed resolution and uniformly converted to 224. A three-channel tensor of size 224 is used, with each pixel value mapped from 0 to 255 by dividing by 255 to a range of 0 to 1, forming the visual input tensor. Weight labels in kg are mapped from 0 to 300 kg by dividing by 300 to a range of 0 to 1, forming the supervision signal. These three parts are organized in a consistent key-value pair manner to form a structured multimodal dataset.

[0056] After obtaining the structured multimodal dataset, an alignment operation is further performed based on the environment parameter embedding matrix and environment weight configuration data. The text embedding matrix is ​​a 128-dimensional environment embedding vector generated after step S1. The visual input tensor is expanded into several fixed-size sub-blocks, each with a channel dimension of 128, ensuring that the environment embedding is completely consistent with the visual input in dimension. The alignment operation is performed in a linear expansion manner: the environment embedding matrix is ​​copied to the same number of rows as the number of visual sub-blocks, forming N... A 128 matrix, where N is the number of visual sub-blocks, is used as the environment-visual alignment data before FiLM injection.

[0057] A dual-channel gating mechanism is applied to the environment-visual alignment data. In this implementation, the dual-channel gating mechanism consists of text channel weights and visual channel weights, which are kept additively equal to 1 through complementary constraints. The text weights are initialized by multiplying the environment parameter embedding matrix with a fixed weight vector, and then normalized to the range of 0 to 1. The visual weights are generated in the same way and constrained according to the formula: visual weight = 1 - text weight. When the illumination value recorded in the environment weight configuration data is below 500 lux, the text weights are adjusted accordingly. illumination The output is enhanced in form and limited to the range of 0.7 to 1, while the visual weights are automatically compressed according to complementary relationships. The dual-channel gated output is then stitched with the environment-visual aligned data to obtain the FiLM-injectable multimodal fusion pre-fusion data.

[0058] Finally, tensor training batches were constructed based on the pre-fusion data. The batch size was set to 32 records, with each record containing an environment vector, a visual input tensor, a text embedding matrix, and a supervision signal. The 32 consecutive pre-fusion data records were stacked in a fixed order to form multimodal training data, ensuring that the environmental conditions within each batch were in a natural order with the image acquisition time, thus avoiding distortion of environmental features within each batch.

[0059] Of particular importance is the construction of the ternary dataset mapping environmental parameters, single-frame images, and weight labels, specifically as follows:

[0060] Perform quality screening on the triplet trusted dataset and perform frame-by-frame timestamp alignment to obtain a time-synchronized dataset;

[0061] Spatial back projection analysis was performed on the time-synchronized dataset based on the camera installation parameters to obtain a matrix representing the spatial position of animals in their enclosures.

[0062] The spatial location matrix is ​​matched with the detection boxes of individual captive animals in the keyframe images to obtain a unique spatial identifier for each captive animal at each time point;

[0063] Based on the spatial identifier sequence, environmental parameters, keyframe images and corresponding weight labels are bound one-to-one to obtain a mapping ternary dataset of environmental parameters-single frame image-weight label.

[0064] In this embodiment of the invention, after completing the hash verification and timestamp anchoring of the trusted triplet dataset, a quality screening step is used to construct the basic data that can be used for mapping. The quality screening executes two fixed rules: First, images with a resolution lower than 1280 are removed. First, records of 720 were processed. Second, records with environmental parameter acquisition frequencies below 1Hz were removed. After screening, timestamp alignment was performed frame by frame. Timestamp alignment used a fixed 100-millisecond time window, with the acquisition time of each frame as the primary key. Synchronization data was constructed by searching for environmental parameter records acquired within the window range. For example, if the image timestamp was 15:22:10.200, only environmental parameter records within the interval of 15:22:10.150 to 15:22:10.250 were retained. If multiple records were included, the record with the smallest time difference was selected as the corresponding environmental parameter, thus constructing a time-synchronized dataset.

[0065] After time synchronization is complete, back-projection analysis is performed on the spatial relationships in the data. Camera installation parameters include installation height (limited to 2.2m to 2.8m) and tilt angle (limited to 15°). Up to 35 ), horizontal orientation angle (limited to −15) Up to 15 The image pixel coordinates are determined using pinhole imaging geometry, along with the focal length (fixed at 3.6mm). , ) and ground coordinates in the enclosure coordinate system ( , The correspondence between the pixels is determined by converting the pixel coordinates to normalized coordinates. ,in , As the imaging center, , The pixel scale corresponds to the focal length; then, the projected coordinates of the animal on the pen floor are obtained by solving the intersection of the vertical planes. Pixel-by-pixel backprojection is performed on all keyframes in the image to form a spatial position matrix, where each matrix element represents the effective occupancy value in the pen grid, with the grid resolution set to 0.1m. 0.1m.

[0066] The spatial location matrix is ​​then matched against the detection boxes for individual captive animals in the keyframe images. Each detection box is a rectangular region extracted from the image, containing a top-left corner (u1, v1) and a bottom-right corner (u2, v2). Back-projection is performed on the four corner points of the detection box to obtain the coordinates of four spatial vertices. The area of ​​the quadrilateral constructed from these four points is then used to determine the corresponding enclosure number. If the spatial region mapped by the detection box falls within the same enclosure coordinate range, the detection box is assigned the corresponding enclosure number. If the four points span multiple enclosures, the enclosure number with the largest area is used as the spatial identification number for the box. All detection boxes are sorted chronologically and combined with the enclosure numbers to form a spatial identifier sequence.

[0067] After obtaining the spatial identifier sequence, a one-to-one binding operation is performed between environmental parameters, keyframe images, and weight labels. The binding process uses the spatial identifier as the primary key and the timestamp-sorted sequence as the secondary key. For each spatial identifier, the environmental parameters corresponding to the time point are extracted from the time-synchronized dataset; the image frame corresponding to that spatial identifier is extracted from the keyframe sequence; and the weight label is associated with the spatial identifier through its ear tag number. For example, if the 3D spatial identifier of column A corresponds to ear tag number 1824 between 15:22:10 and 15:22:20, then the weight value with number 1824 is extracted from the weight label set and bound to all ternary data within the corresponding time interval. This ultimately constitutes a mapped ternary dataset of environmental parameters, single-frame images, and weight labels.

[0068] Preferably, step S3 includes:

[0069] Modality separation is performed on the multimodal training data, and the structured environment parameter input sequence and visual image input sequence are parsed separately to obtain the environment embedding input stream and the visual feature input stream;

[0070] The environment embedding input stream is dimensionally adjusted based on the environment parameter embedding matrix to match the visual feature dimensions of the CNN-Transformer hybrid architecture, resulting in a dimension-matched environment vector.

[0071] Based on the environmental weight configuration data, a dual-channel gating mechanism is invoked to match the dimensional environmental vector to obtain a weighted environmental vector;

[0072] The FiLM conditional modulation operation is performed on the weighted environment vector and injected into the feature extraction layer of the CNN-Transformer hybrid architecture to obtain joint environment-visual features.

[0073] The deep visual feature map is obtained by performing multi-head self-attention calculation on the visual feature input stream through the Transformer visual backbone network.

[0074] Cross-modal fusion is performed on deep visual feature maps and environment-visual joint features to obtain cross-modal fused features;

[0075] Serialization rearrangement and channel normalization are performed on the cross-modal fusion features to obtain the fusion feature sequence;

[0076] A fusion feature training model is constructed based on the fusion feature sequence, and the structural consistency of the model is verified.

[0077] In this embodiment of the invention, after completing the multimodal training data, a modality separation operation is performed on the data. The multimodal training data contains two types of structures: one is a structured environment parameter input sequence, and the other is a visual image input sequence. Modality separation is achieved through two fixed input pipelines: the environment pipeline reads the environment parameter vector (with a fixed dimension of 128-dimensional text embedding), and the visual pipeline reads the formatted image tensor (size 224). 224 3) The environment parameter vector enters the environment embedding input stream, and the image tensor enters the visual feature input stream. The environment pipeline uses a continuous read-in method, arranging each environment parameter in its original order in batches, so that the environment embedding input stream and the visual feature input stream maintain a one-to-one correspondence.

[0078] Based on the environment parameter embedding matrix, the dimensions of the environment embedding input stream are adjusted. Dimension adjustment is achieved using a fixed linear projection matrix with an input dimension of 128 and an output dimension of 768, to adapt to the visual feature dimensions of the CNN-Transformer hybrid architecture. This projection matrix is ​​applied to the environment embedding input stream through matrix multiplication, transforming each environment parameter vector into a 768-dimensional environment-matching vector. Dimension truncation is prohibited during the dimension adjustment process, and all input values ​​are linearly combined according to matrix weights.

[0079] Subsequently, a dual-channel gating call is performed on the dimension-matched environment vector based on the environment weight configuration data. The dual-channel gating structure includes a text weight channel and a visual weight channel, with both channel weight values ​​limited to the range of 0 to 1, and strictly satisfying the requirement that the weight sum is 1. When the illumination value is less than 500 lux, the text channel weight is automatically set to no less than 0.7. The weighting process is completed by linearly scaling each dimension according to the weight, resulting in a weighted environment vector.

[0080] After obtaining the weighted environment vector, FiLM conditional modulation is performed on it. FiLM modulation calculates the scaling and offset coefficients for each dimension of the weighted environment vector and feeds them into the feature extraction layer of the CNN-Transformer hybrid architecture. Inside the feature extraction layer, the visual activation features of each layer are adjusted according to a specific formula: Output value = Scaling coefficient. Original feature values ​​+ offset coefficients. All calculations are performed channel-by-channel, and a complete modulation process is performed once for each layer to obtain the environment-visual joint features.

[0081] After the visual feature input stream enters the Transformer visual backbone network, multi-head self-attention computation is performed. The number of multi-head structures is fixed at 12, and the dimension of each head is fixed at 64. The attention computation process is achieved by calculating the correlation value between the query vector, key vector, and value vector. This correlation value is normalized and used as a weighting coefficient on the value vector to form a deep visual feature map. The dimension of the deep visual feature map is fixed at 14. 14 768.

[0082] Subsequently, cross-modal fusion is performed on the deep visual feature map and the environment-visual joint feature. The fusion operation is completed by channel-by-channel addition, that is, each feature channel in the deep visual feature map is added element-by-element to the environment-visual joint feature of the same dimension. Before the addition operation, the environment-visual feature is spatially aligned with the visual feature map by repeated expansion, thereby constructing the cross-modal fused feature.

[0083] After cross-modal fusion feature generation, serialization rearrangement and channel normalization are performed. Serialization rearrangement arranges the 14 features in row-major order. 14 The 768-dimensional structure is unfolded into a sequence of length 196, with each sequence element being a 768-dimensional vector. Channel normalization uses a fixed mean of 0 and variance of 1, and linearly transforms the values ​​of each channel to obtain the fused feature sequence.

[0084] After the fused feature sequence is determined, a feature input structure framework for the fused feature training model is constructed based on its structure, and a consistency check is performed on this structure. The check process includes two fixed rules: first, checking whether the sequence length is fixed at 196; second, checking whether the number of channels is strictly equal to 768. If either rule is not met, it is marked as a structural anomaly and recorded in the check log.

[0085] Preferably, step S4 includes:

[0086] A multi-task loss function is constructed. After supervising the pre-training of the initial model, scene-differential adaptive training is performed to obtain a scene-specific adapted model.

[0087] A federated learning mechanism is introduced into the scenario-adaptive model. Based on the cross-scenario participant index, a multi-farm distributed training node is established to obtain the initial layout of the federated training.

[0088] Gradient compression-sparsening operation is performed on the model gradients uploaded by each participant in the cross-scene participant index. The gradient sparsity rate is set to 95%, and quantization compression is performed to obtain a sparse gradient set.

[0089] Global parameter aggregation is performed on the scenario-specific adaptation model based on sparse gradient sets to generate a cross-scenario generalization model;

[0090] The generalization error of the cross-scenario generalization model is evaluated on a unified validation set to obtain a generalization error quantification table;

[0091] The generalization error quantification table is compared with the index. When the percentage reduction of generalization error is greater than or equal to 25%, the online model update process is triggered to perform parameter adjustment on the cross-scenario generalization model and obtain the weight prediction model.

[0092] In this embodiment of the invention, a multi-task loss function is first constructed. This loss function consists of two parts: the first part is the weight regression error, which is calculated by averaging the squared difference between the predicted weight value and the actual weight label; the second part is the environmental consistency constraint error, which is calculated by averaging the absolute value of the difference between the weighted result of the environmental-visual joint features and the statistical mean of the environmental embedding input stream in the same batch. The final loss value is a weighted sum of the two errors with a weight ratio of 0.8 and 0.2 to form the multi-task loss result. The preliminary model is then subjected to supervised pre-training using this multi-task loss result to enable the model to form a preliminary weight prediction capability. Based on this, multiple rounds of adaptive training are performed to adapt to the environmental differences between different scenarios. The adaptive training method is as follows: in each training batch, the batch data is divided into multiple subgroups according to the scene label. The gradient direction of the loss of each group is recorded in different subgroups of the same batch. When the gradient directions of multiple subgroups deviate, these deviations are applied proportionally to the scaling factor of the feature channel, thereby forming a scene-specific adaptation model.

[0093] After obtaining the scenario-adaptive model, a federated learning mechanism is introduced. Based on the cross-scenario participant index, a unique node identifier is assigned to each farm, and a distributed training node layout table is constructed on the server side. The layout table records the identity of each node, data size, feature distribution deviation value, and scenario group number. This layout table constitutes the initial layout for federated training.

[0094] In each federated training round, each participating node calculates the model gradient based on its local data and then uploads the gradient to the central control unit. Gradient compression and sparsification are performed on the gradients uploaded by each participant, with a fixed sparsity of 95%. The sparsity operation is implemented by sorting each gradient vector by its absolute value, retaining only the top 5% of gradient elements, and setting the remaining 95% of gradient elements to zero. Subsequently, quantization compression is performed on the retained gradient elements. The quantization compression method involves dividing the gradient values ​​into 256 fixed levels (corresponding to 8-bit representations) and replacing each gradient value with the center value of its respective interval, thus obtaining a sparse gradient set.

[0095] Global parameter aggregation is performed on the scenario-specific adaptation model based on a sparse gradient set. The aggregation method is as follows: a weighted summation is performed on gradient elements at the same position, with the weights set according to the proportion of local data volume of each participant. In this embodiment, the data volume proportions are: 0.5 for large-scale farms, 0.3 for medium-sized farms, and 0.2 for small-scale farms. After weighted summation, the aggregation result is applied to the parameter update of the scenario-specific adaptation model, forming a cross-scenario generalization model.

[0096] The cross-scenario generalization model was evaluated for generalization error on a unified validation set. The unified validation set consisted of 10 days of data collected from three pig farms, where each data sample included a spatially aligned single-frame image, environmental parameters, and weight labels. The generalization error was calculated by averaging the absolute value of the prediction error for each sample to obtain a generalization error quantization table. The quantization table recorded the overall error value, the intra-scenario error value, and the inter-scenario error value.

[0097] The generalization error quantification table is compared with the error reduction percentage threshold. If the error reduction percentage is greater than or equal to 25%, the online model update process is triggered.

[0098] In the online model update process, the aggregated parameters are used as initial weights, and gradient direction calibration is performed on the key parameters of the cross-scenario generalization model (including feature scaling coefficients, channel offset parameters, and final regression weights). The calibration method is as follows: the average of the three most recent sparse gradient sets is stacked in chronological order, and the gradient vector after the average stacking is applied to the weight update formula, so that the parameters are adjusted in the long-term stable direction, and finally the weight prediction model is obtained.

[0099] Preferably, constructing a multi-task loss function and performing scene-difference adaptive training after supervised pre-training of the initial model includes:

[0100] A multi-task loss function consisting of weight regression loss, body shape parameter regression loss and environmental consistency loss is constructed for the fusion feature training model to obtain the multi-task loss set.

[0101] Based on the multi-task loss set, supervised pre-training is performed on the fusion feature training model to obtain a preliminary weight prediction model.

[0102] Based on environmental data from different farms, the preliminary weight prediction model was subjected to scenario-specific adaptive training to obtain a scenario-specific model.

[0103] In this embodiment of the invention, a multi-task loss function is first constructed for the fusion feature training model, consisting of weight regression loss, body shape parameter regression loss, and environmental consistency loss. The weight regression loss is expressed as a squared difference, calculated by averaging the squared difference between the predicted weight value and the weight label, with an error range limited to 0 to 500. The body shape parameter regression loss is constructed based on three body shape indicators: length, width, and area of ​​the pig's body contour. The error of each body shape parameter is expressed as an absolute difference, with each value limited to the intervals of 0 to 2, 0 to 1, and 0 to 5, respectively. The average of the three body shape errors forms the body shape parameter regression loss. The environmental consistency loss is calculated by averaging the difference between the 128-dimensional vector embedded in the environment input stream and the statistical mean of the activation features of each layer after FiLM modulation, with the error range limited to 0 to 3. The three losses are weighted in a ratio of 0.6:0.25:0.15 to form a multi-task loss set.

[0104] After establishing the multi-task loss set, supervised pre-training is performed on the fusion feature training model. During supervised pre-training, multimodal training data is input in batches, with each batch size fixed at 32. For each batch, the fusion feature sequence is input to the weight estimation output layer and the body shape parameter output layer. The output value is obtained through a complete forward computation, and then the multi-task loss is calculated with the true labels. Subsequently, the multi-task loss is applied to the weights of each layer according to a parameter adjustment rule with a fixed learning amplitude of 0.001, causing the parameters of the fusion feature training model to be updated in the direction of reducing loss. After 30 consecutive rounds of training, the loss curve tends to stabilize, at which point a preliminary weight prediction model is obtained.

[0105] After obtaining the preliminary weight prediction model, environmental data from different farms were grouped. Grouping was based on light intensity, pen material type, and humidity level. For example, data with light intensity less than 300 lux was classified as low light, 300-800 lux as medium light, and greater than 800 lux as high light. Pens were divided into metal, plastic, and mixed types. Humidity was divided into three groups: 30-50%, 50-70%, and greater than 70%. After grouping, data within the same group were treated as a sub-scene, forming sub-scene training batches.

[0106] The implementation method for scene-differentiated adaptive training is as follows: In each training iteration, the loss gradient direction of different sub-scene batches is recorded. When multiple sub-scenes exhibit opposite gradient directions on a certain feature channel, the scaling factor of that feature channel is adjusted by an increment of 1.02 or 0.98. The adjustment principle is: if the absolute gradient value of the feature is higher in the low-light group, its scaling factor is increased by 1.02; if the absolute gradient value is higher in the high-light group, it is decreased by 0.98. This adjustment is performed once after each round of training. After 20 consecutive rounds of training, the gradient direction differences between sub-scenes tend to stabilize, thus forming a scene-specific adaptation model.

[0107] Most importantly, the formula for calculating the percentage reduction in generalization error is as follows:

[0108] ;

[0109] in, To reduce the generalization error by a percentage, As a baseline generalization error, the numerical range is limited to 2 to 7 kg. This represents the generalization error of the current model.

[0110] Preferably, step S5, which verifies the environmental adaptability of the weight prediction model and performs error analysis, includes:

[0111] End-side inference tests were performed on the weight prediction model under different environmental conditions. An adaptability evaluation matrix based on multiple environmental indicators was constructed to obtain environmental adaptability verification data.

[0112] Scenario difference analysis was performed on the environmental adaptability verification data to identify the error shift of the weight prediction model in cross-scenario inference, and a high-error environment group and a low-error environment group were constructed to obtain a scenario error difference table.

[0113] Based on the mapping ternary dataset, the corresponding samples in the scene error difference table are labeled to verify consistency, and risky samples are removed to obtain a set of reliable error samples.

[0114] Error source classification is performed on the set of credible error samples to obtain the error decomposition matrix. The error source types include visual error, environmental weight error and body shape parameter regression error.

[0115] Based on the error decomposition matrix, the environmental parameter embedding matrix and the parameters of FiLM conditional modulation are back-mapped to construct the environmental modulation back spectrum and obtain the environmental modulation correction coefficients.

[0116] The text channel weights and visual channel weights are recalibrated based on the environmental modulation correction coefficient, and the text channel weights are automatically increased in low-light areas to obtain the environmental modulation recalibration parameters.

[0117] Feature reconstruction is performed on the environmental modulation recalibration parameters and error decomposition matrix to form an error optimization tensor, thus obtaining error optimization data.

[0118] In this embodiment of the invention, after completing step S4 and obtaining the weight prediction model, the model is deployed on an edge inference device to perform environmental adaptability verification. The edge device is a fixed-model embedded inference motherboard with a power consumption limited to 10W and an inference frequency of 4 times / second. During verification, the inference process is run under five environmental conditions: low light (0 to 300 lux), medium light (300 to 600 lux), high light (600 to 1200 lux), high humidity (80% to 95%), and strong reflectivity (ground reflectivity 0.7). For each environmental condition, 500 single-frame images are acquired and input into the weight prediction model. The model output value, environmental parameters embedded in the input stream, and the scaling and offset coefficients of the FiLM conditional modulation layer are recorded. All data are constructed into an environmental adaptability evaluation matrix according to the environmental dimension. This matrix is ​​divided into rows according to environmental conditions and columns according to output error. The error values ​​are calculated in the form of absolute difference, forming environmental adaptability verification data.

[0119] Based on the environmental adaptability verification data, scenario difference analysis was performed on the mean error under each environment. The difference analysis steps were as follows: the mean error of each environment was used as an independent indicator, all environmental errors were sorted from smallest to largest, the top 20% of environmental conditions were selected to form a low-error environment group, and the bottom 20% of environmental conditions were selected to form a high-error environment group. The two groups of environmental conditions and their corresponding errors were recorded in the scenario error difference table.

[0120] Subsequently, based on the mapped ternary dataset, label consistency verification was performed on all samples involved in the scene error difference table. The verification method was as follows: the timestamps of the weight label, environmental parameters, and single-frame image of each sample were compared, requiring the time deviation to be no more than 50 milliseconds; if the time deviation exceeded the limit, the sample was marked as a risk sample. After performing a removal operation on all risk samples, the remaining samples constituted a set of reliable error samples.

[0121] Error sources are classified within a credible error sample set. The classification method is as follows: First, visual error is calculated by comparing the offset of the target pig's outline bounding box with the activation peak position in the visual feature extraction layer; this offset is limited to the range of 0 to 15 pixels. Second, environmental weight error is calculated by comparing the absolute value of the difference between the corresponding dimension of the environmental parameter embedding matrix and the FiLM scaling factor; this difference is limited to the range of 0 to 3. Third, body shape parameter regression error is calculated by comparing the difference between the model's output body shape parameters and body shape labels to form the error category. The three types of errors are arranged into a matrix according to sample number to construct an error decomposition matrix.

[0122] Based on the error decomposition matrix, an error inverse mapping operation is performed on the environmental parameter embedding matrix and the FiLM conditional modulation process. The inverse mapping is as follows: the environmental weight error is used as input, and the scaling coefficients of the corresponding FiLM layer are adjusted inversely by a fixed ratio, which is 0.05 times the error value. Simultaneously, based on the positional offset of the visual error in the channel dimension, the dimension values ​​related to that channel in the environmental parameter embedding matrix are adjusted, with the adjustment range between 0.01 and 0.05. The two inverse adjustment processes are merged to form an environmental modulation inverse map, and the output is a set of environmental modulation correction coefficients.

[0123] The text channel weights and visual channel weights are recalibrated based on the environmental modulation correction coefficient. The calibration rule is as follows: if the environmental weight error in a low-light scene is greater than 1.0, the text channel weight is increased by 0.1, and the visual channel weight is decreased by the same amount; if the environmental weight error in a low-light scene is less than 1.0, the weights remain unchanged. The final environmental modulation recalibration parameters are then obtained.

[0124] A feature reconstruction operation is performed on the environmental modulation recalibration parameters and the error decomposition matrix. The reconstruction method involves applying each error type to the environmental modulation recalibration parameters according to their channel correspondence, ensuring that each error category produces different adjustment directions for different channels. All adjusted parameters are then superimposed channel by channel to form an error optimization tensor. This error optimization tensor is the error optimization data.

[0125] Preferably, updating the weight prediction model parameters based on error optimization data in step S5 includes:

[0126] The model performance corresponding to the error optimization data is quantitatively evaluated, and the online update process is triggered when the generalization error reduction percentage is less than 25%, thus obtaining the online update trigger signal.

[0127] The model parameters corresponding to the online update trigger signal are updated by gradient, and the gradient compression-sparseness mechanism in federated learning is executed to obtain the parameter update gradient set.

[0128] Global parameter aggregation is performed based on the parameter update gradient set, and joint parameter correction is performed on the Transformer backbone, CNN branches and FiLM modulation layer to obtain the weight prediction optimization model.

[0129] In this embodiment of the invention, the model performance corresponding to the error optimization data is first quantitatively evaluated. The quantitative evaluation method is as follows: the error optimization tensor is input into the validation interface of the weight prediction model, inference tests are performed through a fixed validation set, and the error value of each sample is recorded; the error value is calculated using the absolute difference, and the average value is calculated for each batch to form an error list. The error list is compared with the cross-scenario generalization model error benchmark table generated in step S4, and the percentage reduction in generalization error is calculated. If the percentage reduction in generalization error is less than 25%, an online update trigger signal is generated in the evaluation system. This signal is recorded in binary form, and a value of "1" indicates that an online update needs to be performed.

[0130] The online update trigger signal is bound to the parameter storage module of the weight prediction model. After triggering the online update process, all weights of the Transformer backbone layer, CNN branch layer, and FiLM modulation layer are extracted from the model parameters, and the gradients of each layer are calculated based on the error optimization tensor. The gradient calculation formula is: gradient value = 0.1 times the difference between the corresponding dimension value of the error optimization tensor and the current parameter value, where 0.1 is a fixed gradient scaling factor. All gradients are grouped by weight to form a complete gradient update. Subsequently, the gradient compression-sparseness mechanism in federated learning is applied to the updated gradient. The sparsity is fixed at 95%, that is, only the top 5% of the gradients with the largest absolute values ​​are retained, and the rest are set to zero. The retained gradient values ​​are quantized and compressed by mapping the gradient values ​​to 256 fixed levels according to the interval, with each level corresponding to the center value of a fixed numerical interval. The parameter update gradient set is formed in the above way.

[0131] Global parameter aggregation is performed based on the gradient set updated by parameters. The aggregation method is as follows: multiple gradient values ​​at the same parameter position are weighted and summed according to the data scale ratio of the participating scenarios; the data scale ratio is set as follows: large pig farms have a weight of 0.5, medium-sized pig farms have a weight of 0.3, and small pig farms have a weight of 0.2. The aggregation operation on the Transformer backbone layer is performed in hierarchical order; the aggregation operation on the CNN branch layers is performed element-wise according to the convolution kernel position; the aggregation operation on the FiLM modulation layer is performed linearly according to the correspondence between the scaling factor and the offset factor, and the linear fusion method is: fusion value = sum of the coefficients of each node according to their weights. The aggregated parameters are written into the original weight prediction model to form the weight prediction optimization model.

[0132] Please see Figure 2 The algorithm framework integrates environmental text parameters for enhanced training. The input layer consists of "environmental text parameters (camera height / angle, lighting, etc.) + video image data"; the intermediate layer consists of "text encoder + visual encoder". Sleep + Convolutional Neural Network Auxiliary Branch (Pig Contour / Pose Detection)"; Output Layer "Large Language Model Decoder (Re-estimation Result Output)".

[0133] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is not limited by the foregoing description. Thus, all changes falling within the meaning and scope of the equivalents of the application are intended to be included within the scope of the invention.

[0134] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for predicting the weight of captive animals based on the fusion of environmental and visual features, characterized in that, Includes the following steps: Step S1: Synchronously collect environmental parameters, image data, and weight labels, and construct a triplet dataset; Step S2: Preprocess the triplet dataset and construct a mapping relationship between environmental parameters, single-frame images, and the weight of the corresponding captive animal based on timestamps and preset spatial coordinates, and generate multimodal training data; Step S3: Input the environmental parameters from the multimodal training data into the environment-image feature fusion module in the pre-trained CNN-Transformer hybrid architecture, perform dimensional matching with the image features, and obtain the fused feature training model; Step S4: Design a multi-task loss function, pre-train the fusion feature training model, and adjust the parameters according to different scenarios to obtain the weight prediction model; wherein, step S4 includes: A multi-task loss function is constructed. After supervising the pre-training of the initial model, scene-differential adaptive training is performed to obtain a scene-specific adapted model. A federated learning mechanism is introduced into the scenario-adaptive model. Based on the cross-scenario participant index, a multi-farm distributed training node is established to obtain the initial layout of the federated training. Gradient compression-sparsening operation is performed on the model gradients uploaded by each participant in the cross-scene participant index. The gradient sparsity rate is set to 95%, and quantization compression is performed to obtain a sparse gradient set. Global parameter aggregation is performed on the scenario-specific adaptation model based on sparse gradient sets to generate a cross-scenario generalization model; The generalization error of the cross-scenario generalization model is evaluated on a unified validation set to obtain a generalization error quantification table; The generalization error quantification table is compared with the index. When the percentage reduction of generalization error is greater than or equal to 25%, the online model update process is triggered to perform parameter adjustment on the cross-scenario generalization model and obtain the weight prediction model. The process of constructing a multi-task loss function and performing scene-difference adaptive training after supervised pre-training of the initial model includes: A multi-task loss function consisting of weight regression loss, body shape parameter regression loss and environmental consistency loss is constructed for the fusion feature training model to obtain the multi-task loss set. Based on the multi-task loss set, supervised pre-training is performed on the fusion feature training model to obtain a preliminary weight prediction model. Based on environmental data from different farms, the preliminary weight prediction model was subjected to scenario-specific adaptive training to obtain a scenario-specific model. Step S5: Verify the environmental adaptability of the weight prediction model and perform error analysis to generate error optimization data; update the weight prediction model parameters based on the error optimization data to obtain the optimized weight prediction model; wherein, step S5, verifying the environmental adaptability of the weight prediction model and performing error analysis, includes: End-side inference tests were performed on the weight prediction model under different environmental conditions. An adaptability evaluation matrix based on multiple environmental indicators was constructed to obtain environmental adaptability verification data. Scenario difference analysis was performed on the environmental adaptability verification data to identify the error shift of the weight prediction model in cross-scenario inference, and a high-error environment group and a low-error environment group were constructed to obtain a scenario error difference table. Based on the mapping ternary dataset, the corresponding samples in the scene error difference table are labeled to verify consistency, and risky samples are removed to obtain a set of reliable error samples. Error source classification is performed on the set of credible error samples to obtain the error decomposition matrix. The error source types include visual error, environmental weight error and body shape parameter regression error. Based on the error decomposition matrix, the environmental parameter embedding matrix and the parameters of FiLM conditional modulation are back-mapped to construct the environmental modulation back spectrum and obtain the environmental modulation correction coefficients. The text channel weights and visual channel weights are recalibrated based on the environmental modulation correction coefficient, and the text channel weights are automatically increased in low-light areas to obtain the environmental modulation recalibration parameters. Feature reconstruction is performed on the environmental modulation recalibration parameters and error decomposition matrix to form an error optimization tensor, thus obtaining error optimization data.

2. The method for predicting the weight of captive animals based on the fusion of environmental and visual features according to claim 1, characterized in that, Step S1 includes: Data such as light intensity, enclosure type, temperature and humidity, and camera installation parameters are collected by sensors and cameras and then uniformly quantified and encoded to form a structured set of environmental parameters. The postures of captive animals and the area covered by the image are collected by the end-side camera device, and key frames are selected to form an image dataset; The weight of captive animals was collected by manual weighing and labeled with weight tags to obtain a set of weight tags; The environmental parameters, image data, and weight labels are aligned in a three-way manner to obtain a triplet dataset.

3. The method for predicting the weight of captive animals based on the fusion of environmental and visual features according to claim 2, characterized in that, After aligning the environmental parameters, image data, and weight tags, the following is also included: The triplet dataset is verified by blockchain hash and anchored by linked timestamps to obtain a trusted triplet dataset. The environmental text descriptions in the triplet trusted dataset are text-encoded, and the fence type, lighting level and camera installation parameters are converted into 128-dimensional text embedding vectors to obtain the environmental parameter embedding matrix. Based on the preset ViT environment injection mechanism, a dual-channel gating structure is constructed by embedding the environment parameter matrix to obtain the text weight channel and the visual weight channel, and the sum of the text weight channel and the visual weight channel is 1. When the light intensity is less than 500 lux, the text weight is automatically set to be greater than 0.7, resulting in environmental weight configuration data with low light intensity adaptability.

4. The method for predicting the weight of captive animals based on the fusion of environmental and visual features according to claim 1, characterized in that, Step S1 is followed by: Data sources from multiple farms are obtained, and the data is grouped by scenario. Federated learning node identifiers are configured for each scenario to obtain a cross-scenario participant index. A preliminary analysis was conducted on the differences in data scale, distribution, and environmental characteristics among different farms. The gradient compression sparsity rate was set to 95%, and a pre-parameter table for federated training was obtained. Statistical analysis of environmental differences in the triplet credible dataset is performed to calculate the theoretical lower bound of environmental generalization error, resulting in a generalization error benchmark table.

5. The method for predicting the weight of captive animals based on the fusion of environmental and visual features according to claim 1, characterized in that, Step S2 includes: Based on timestamps and preset spatial coordinates, a mapping triplet dataset of environmental parameters, single-frame images, and weight labels is constructed using a triplet dataset. The mapped triplet dataset is divided into multimodal structures, environmental parameters are encoded into structured environmental vectors, image data is formatted into visual input tensors, and weight labels are converted into supervision signals to obtain a structured multimodal dataset. Based on the environmental parameter embedding matrix and environmental weight configuration data, the 128-dimensional text embedding vector is aligned with the visual input dimension to obtain the environment-visual alignment data before FiLM injection. Based on a structured multimodal dataset, a dual-channel gating mechanism is invoked on the environment-visual alignment data. By dynamically and complementaryly configuring the text channel weights and visual channel weights, FiLM-injectable multimodal fusion pre-data is obtained. Based on the preset input requirements, tensor quantization training batches are determined using the data before multimodal fusion, and multimodal training data is obtained.

6. The method for predicting the weight of captive animals based on the fusion of environmental and visual features according to claim 1, characterized in that, Step S3 includes: Modality separation is performed on the multimodal training data, and the structured environment parameter input sequence and visual image input sequence are parsed separately to obtain the environment embedding input stream and the visual feature input stream; The environment embedding input stream is dimensionally adjusted based on the environment parameter embedding matrix to match the visual feature dimensions of the CNN-Transformer hybrid architecture, resulting in a dimension-matched environment vector. Based on the environmental weight configuration data, a dual-channel gating mechanism is invoked to match the dimensional environmental vector to obtain a weighted environmental vector; The FiLM conditional modulation operation is performed on the weighted environment vector and injected into the feature extraction layer of the CNN-Transformer hybrid architecture to obtain joint environment-visual features. The deep visual feature map is obtained by performing multi-head self-attention calculation on the visual feature input stream through the Transformer visual backbone network. Cross-modal fusion is performed on deep visual feature maps and environment-visual joint features to obtain cross-modal fused features; Serialization rearrangement and channel normalization are performed on the cross-modal fusion features to obtain the fusion feature sequence; A fusion feature training model is constructed based on the fusion feature sequence, and the structural consistency of the model is verified.

7. The method for predicting the weight of captive animals based on the fusion of environmental and visual features according to claim 1, characterized in that, Step S5, which involves updating the weight prediction model parameters based on error optimization data, includes: The model performance corresponding to the error optimization data is quantitatively evaluated, and the online update process is triggered when the generalization error reduction percentage is less than 25%, thus obtaining the online update trigger signal. The model parameters corresponding to the online update trigger signal are updated by gradient, and the gradient compression-sparseness mechanism in federated learning is executed to obtain the parameter update gradient set. Global parameter aggregation is performed based on the parameter update gradient set, and joint parameter correction is performed on the Transformer backbone, CNN branches and FiLM modulation layer to obtain the weight prediction optimization model.

Images