A piglet crush early warning method and system based on a two-stage cascade model

Abstract

The application provides a piglet crush early warning method and system based on a two-stage cascade model, relates to the cross technical field of computer vision and intelligent agriculture, and the method is to obtain a continuous video of sows and piglets from a top view, adopt a target detection and multi-target tracking model in a two-stage cascade model to process, and obtain pig target positioning data; according to this, a single pig local image sequence is generated by cutting, a time sequence behavior classification model is used for processing, and behavior category determination data is obtained; in combination with the above data, a preset piglet quantity checking mechanism and a behavior science prior rule are used for comparison and verification, and a final crush state analysis result is obtained; when crush or quantity loss exists, an early warning signal is generated. The application solves the problems of single-frame occlusion missing report, short-time sequence model motion capture deficiency and prediction violation of physiological common sense, and realizes high-precision, false alarm prevention and automatic crush early warning.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of computer vision and smart agriculture, and in particular to a method and system for early warning of piglet crushing based on a two-stage cascaded model. Background Technology

[0002] Pre-weaning mortality in piglets is one of the most prominent animal welfare and economic challenges in modern pig production. Studies have confirmed that crushing is the leading cause of pre-weaning mortality in piglets, occurring most frequently during the postpartum window when sows frequently change position. When sows perform key lethal actions such as shifting from standing to lying down or rolling over, piglets are highly susceptible to being crushed and killed because they cannot escape in time. Traditional production management relies mainly on manual inspections for intervention; this control model is time-consuming and labor-intensive, and has significant shortcomings in terms of comprehensiveness and sustainability, falling far short of the practical needs of modern precision management.

[0003] In recent years, computer vision models have been introduced into this field due to their non-contact core advantage. However, existing technologies still have significant drawbacks: First, single-frame image-based target detection schemes, such as those relying on the overlap between adult pigs and piglets, cannot distinguish between squeezed behavior and active approach for suckling, and fail immediately when piglets are completely occluded by the sow. Second, schemes based on skeletal joint extraction have extremely high image quality requirements, resulting in large detection errors and missed detections when occluded or under changing lighting conditions. Third, contour-tracking-based schemes are prone to contour merging and target loss when pigs are stuck together, and lack a warning mechanism for post-event alarms. Furthermore, pure visual black-box models are prone to misjudgments that conflict with the physiological patterns of pigs, and single-frame or short-time-series modeling struggles to capture the continuous evolution of squeezing behavior. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a piglet crushing early warning method and system based on a two-stage cascaded model. This solves the problems in existing piglet crushing monitoring technologies, such as missed reports due to single-frame image occlusion, the inability of short-time-series models to capture continuous motion evolution, and the fact that pure visual black-box prediction violates the physiological common sense of pigs. As a result, it is difficult to achieve high-precision, real-time, and false alarm-proof automated early warning of crushing events in large-scale farrowing houses.

[0005] To achieve the aforementioned objectives, the technical solution adopted by this invention is: a method for early warning of piglet crushing based on a two-stage cascaded model, comprising: S1: Acquire continuous overhead video image data of sows and piglets in the farrowing house, and use the target detection and multi-target tracking model in the two-stage cascaded model for identification and tracking processing to obtain pig target localization data including pig bounding boxes, classification labels and movement trajectories; S2: Based on the target localization data of pigs, local image sequence data of a single pig is generated by cropping from the continuous video image data from the top view, and the temporal behavior classification model in the two-stage cascaded model is used for feature extraction and temporal classification processing to obtain the behavior category determination data of each pig. S3: Based on behavioral category determination data and pig target positioning data, anomaly comparison and result verification are performed using a preset piglet number verification mechanism and behavioral prior rules to obtain the final crushing state analysis results. S4: Analyze the final crushing status analysis results. If either a crushing event or a missing number of piglets is detected, a piglet crushing early warning signal is generated, thus completing the piglet crushing early warning.

[0006] Further, S1 includes: We acquire continuous overhead video images of sows and piglets in the farrowing house, and use a target detector layer to perform feature extraction and position probability prediction to obtain initial detection feature data containing position coordinates and category information. Based on the initial detection feature data, a multi-target tracking algorithm layer is used to perform motion trajectory prediction and data association matching to obtain pig target localization data containing pig bounding boxes, classification labels, and motion trajectories; among them, the target detector layer and the multi-target tracking algorithm layer belong to the target detection and multi-target tracking model.

[0007] Furthermore, the target detection and multi-target tracking model includes: The target detector layer is used to perform feature extraction and classification regression processing based on continuous overhead video image data using an anchor-free decoupled detection method to obtain initial detection feature data. The multi-target tracking algorithm layer is used to update the trajectory based on the initial detection feature data using filtering prediction and Hungarian matching algorithm, so as to obtain pig target localization data containing pig bounding boxes, classification labels and movement trajectories.

[0008] Further, S2 includes: Based on the detection box coordinates in the pig target localization data, target region cropping and temporal slice reconstruction are performed in the top-view continuous video image data to obtain local image sequence data of a single pig. Based on local image sequence data, a temporal behavior classification model is used to extract short-term spatiotemporal features, focus channel attention, and model long-distance temporal dependencies to obtain behavioral category determination data for each pig.

[0009] Furthermore, the temporal behavior classification model includes: The two-dimensional convolutional backbone module is used to perform inter-frame displacement and short-time spatiotemporal feature extraction based on local image sequence data using a temporal shift mechanism to obtain multi-dimensional feature data. The channel attention module is used to perform channel attention weighting on multidimensional feature data using one-dimensional convolution and adaptive channel correlation modeling to obtain spatially enhanced feature data. The spatial pooling and dimensionality reduction projection module is used to perform spatial dimension compression and sequence reconstruction on spatially enhanced feature data using global average pooling and linear mapping to obtain low-dimensional temporal feature sequence data. The selective state space module is used to perform long-distance temporal dependency modeling and discretization state update processing on low-dimensional temporal feature sequence data through a selective mechanism of input dependency, so as to obtain long-term temporal context feature data. The linear classification module is used to output behavioral category determination data for each pig by using time-dimensional average fusion and fully connected mapping based on long-term temporal contextual feature data.

[0010] Furthermore, the expression for the spatial enhancement feature data is: ; in, Representation spatially enhanced feature data, Represents multidimensional feature data. This represents an adaptive global two-dimensional average pooling operation. Indicates the kernel size as One-dimensional local convolution operation, This represents the local convolutional kernel size calculated based on an adaptive function, used to adaptively determine the range of cross-channel interactions. This represents the Sigmoid activation function. This indicates a weighted operation of element-wise multiplication of channels.

[0011] Furthermore, the discretization update expression for the low-dimensional time-series feature sequence data is as follows: ; in, Indicates the first The temporal hidden state data at each time step is obtained by concatenating the sequences of all hidden state data along the time dimension to obtain long-term temporal context feature data. Indicates the first Temporal hidden state data at any given time. Represents the extraction of the first from low-dimensional time-series feature sequence data. Input feature data at time step This represents the adaptive time-series step size parameter of the model, dynamically generated on a time scale based on low-dimensional time-series feature sequence data. Represents the state transition matrix. This represents the input control matrix generated based on the discretization rules. This indicates the operation of the exponential function.

[0012] Further, S3 includes: Based on the sow position change recognition results in the behavior category judgment data, the piglet number verification mechanism is triggered to compare and statistically process the currently visible number of piglets in the pig target location data with the preset total number in the system, and obtain the number verification anomaly judgment data. Based on the behavior category determination data, the behavior prior rules are used to perform logical filtering on the physical categories of action jumps and detection box sizes in adjacent time periods, and confidence reduction correction is performed based on the spatial coverage rate of the target box to obtain the rule correction state data. Based on the anomaly detection data and rule correction status data, a comprehensive evaluation and processing is performed to obtain the final extrusion status analysis results.

[0013] Furthermore, the expressions for the spatial coverage rate and the rule correction state data are as follows: ; ; in, Indicates spatial coverage. This indicates the current target in the pig target location data. Bounding box data, This indicates a stable historical target extracted from pig target location data. Bounding box data, This function represents the operation of calculating the area based on the bounding box's width and height coordinates. This indicates the operation of finding the intersection of spaces. This represents the reduced target confidence level output in the rule-corrected state data. Indicates the current target in the behavior category determination data. The original classification confidence score, This represents the preset confidence level suppression penalty coefficient.

[0014] This invention provides a piglet crushing early warning system based on a two-stage cascaded model, comprising: The recognition module is used to acquire continuous video image data of sows and piglets from above in the farrowing house. It uses the target detection and multi-target tracking model in the two-stage cascaded model for recognition and tracking processing to obtain pig target localization data including pig bounding boxes, classification labels and movement trajectories. The classification module is used to crop local image sequence data of a single pig from the continuous overhead video image data based on the target positioning data of pigs, and to perform feature extraction and temporal classification processing using the temporal behavior classification model in the two-stage cascaded model to obtain the behavior category determination data of each pig. The comparison module is used to perform anomaly comparison and result verification based on behavioral category judgment data and pig target location data, using a preset piglet number verification mechanism and behavioral prior rules, to obtain the final crushing state analysis result. The early warning module is used to analyze the final crushing status analysis results. When it is determined that either a crushing event exists or the number of piglets is missing, a piglet crushing early warning signal is generated, thus completing the piglet crushing early warning.

[0015] The beneficial effects of this invention are as follows: This invention provides a piglet crushing early warning method based on a two-stage cascaded model, which has the following beneficial effects: First, by employing the target detection and multi-target tracking models in a two-stage cascaded model, target localization and behavior recognition are decoupled. This approach enables the parallel and lightweight generation of pig target localization data, including bounding boxes and motion trajectories, in complex overhead video footage, significantly improving the efficiency of concurrent detection of multiple pigs.

[0016] Secondly, a temporal behavior classification model is used to process local image sequence data. This method not only eliminates interference from complex backgrounds, but also extracts short-term motion cues and long-distance dependencies between consecutive frames through temporal classification, effectively overcoming the deficiency of traditional single-frame detection in parsing the semantics of continuous actions.

[0017] Finally, after obtaining the behavioral category determination data, a pre-set piglet count verification mechanism and behavioral prior rules were introduced for validation. The count verification mechanism, through anomaly comparison, fundamentally solves the problem of missed detection caused by the disappearance of visual features when piglets are completely obscured by the sow's body; while the prior rule validation filters out erroneous identifications that contradict physical size and movement continuity, ensuring the high reliability of the final crushing state analysis results, thereby achieving accurate crushing early warning. Attached Figure Description

[0018] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is a schematic diagram of a piglet crushing early warning system based on a two-stage cascaded model, as shown in some embodiments of this specification. Figure 2 This is an exemplary flowchart of a piglet crushing early warning method based on a two-stage cascaded model, as shown in some embodiments of this specification. Detailed Implementation

[0019] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0020] Example 1 Figure 1 This is a schematic diagram of a piglet crushing early warning system based on a two-stage cascaded model, as shown in some embodiments of this specification.

[0021] In some embodiments, the piglet crushing early warning system based on a two-stage cascaded model may include an identification module for acquiring continuous overhead video image data of sows and piglets in the farrowing house, and using the target detection and multi-target tracking model in the two-stage cascaded model for identification and tracking processing to obtain pig target localization data including pig bounding boxes, classification labels, and movement trajectories; a classification module for cropping local image sequence data of a single pig from the continuous overhead video image data based on the pig target localization data, and using the temporal behavior classification model in the two-stage cascaded model for feature extraction and temporal classification processing to obtain behavior category determination data for each pig; a comparison module for performing anomaly comparison and result verification processing based on the behavior category determination data and the pig target localization data using a preset piglet number verification mechanism and behavioral prior rules to obtain the final crushing state analysis result; and an early warning module for analyzing the final crushing state analysis result, and generating a piglet crushing early warning signal when either a crushing event or a missing piglet number is determined, thus completing the piglet crushing early warning.

[0022] In some embodiments, a piglet crushing early warning system based on a two-stage cascaded model can be used to execute a piglet crushing early warning method based on a two-stage cascaded model, including: S1: acquiring continuous overhead video image data of sows and piglets in the farrowing house, and using the target detection and multi-target tracking model in the two-stage cascaded model for identification and tracking processing to obtain pig target localization data including pig bounding boxes, classification labels, and movement trajectories; S2: based on the pig target localization data, cropping local image sequence data of a single pig from the continuous overhead video image data, and using the temporal behavior classification model in the two-stage cascaded model for feature extraction and temporal classification processing to obtain behavior category determination data for each pig; S3: based on the behavior category determination data and the pig target localization data, using a preset piglet number verification mechanism and behavioral prior rules for anomaly comparison and result verification processing to obtain the final crushing state analysis result; S4: analyzing the final crushing state analysis result, and generating a piglet crushing early warning signal when either a crushing event or a missing piglet number is determined, thus completing the piglet crushing early warning.

[0023] In some embodiments of this specification, the processor utilizes a piglet crushing early warning system based on a two-stage cascaded model to execute a piglet crushing early warning method based on a two-stage cascaded model. In this way, the aforementioned two-stage cascaded algorithm can be modularly integrated, achieving fully automated data flow from continuous video image acquisition to alarm signal generation. Through closed-loop collaborative operation of hardware and software, the system can perform 24 / 7 monitoring and real-time anomaly response of the farrowing house environment without human intervention, effectively replacing traditional time-consuming and labor-intensive overnight manual inspections, significantly reducing management costs for breeding enterprises and improving piglet survival rates.

[0024] Example 2 Figure 2 This is an exemplary flowchart illustrating a piglet crushing early warning method based on a two-stage cascaded model, according to some embodiments of this specification. Figure 2 As shown, the process includes the following steps. In some embodiments, the process may be executed by a processor.

[0025] S1: Acquire continuous overhead video images of sows and piglets in the farrowing house, and use the target detection and multi-target tracking model in the two-stage cascaded model for identification and tracking processing to obtain pig target localization data including pig bounding boxes, classification labels and movement trajectories.

[0026] Top-down continuous video image data is a collection of real-time images of sows and piglets in the farrowing stalls. For example, top-down continuous video image data can include stereoscopic image frames at a fixed frame rate, panoramic views of sows in prone or standing positions, images of the visible area of ​​piglet activity, and multi-frame continuous pixel data under constant lighting conditions.

[0027] In some embodiments, the processor can continuously capture images from a set top-down angle using a stereo camera mounted directly above the birthing barrier, convert the optical signals into a digital image matrix, sample them at equal intervals according to a predetermined frame rate and resolution, and finally perform uniform scaling to obtain continuous top-down video image data.

[0028] A two-stage cascaded model is a comprehensive prediction architecture consisting of two functionally independent but interconnected deep learning processing stages. For example, a two-stage cascaded model may include lightweight front-end network weight data for localization and tracking, back-end network structure parameters for temporal behavior parsing, and a cropped coordinate mapping table for data exchange between the front-end and back-end; that is, an object detection and multi-object tracking model and a temporal behavior classification model.

[0029] In some embodiments, the processor can pre-build a front-end target detection and tracking network architecture and a back-end temporal behavior classification network architecture, and train them separately using a labeled pig dataset containing multiple target locations and action categories. Finally, the two converged training models are connected and configured in the deployment environment through a data flow pipeline to obtain a two-stage cascaded model.

[0030] The object detection and multi-object tracking model is a combined algorithm model responsible for identifying the location of pigs and maintaining identity continuity in complex scenes. For example, the object detection and multi-object tracking model may include convolutional neural network weights for outputting bounding boxes and confidence scores, a Kalman filter state matrix for predicting the position of the next frame, and a Hungarian algorithm cost matrix for data association.

[0031] In some embodiments, the processor can load a pre-trained lightweight anchorless decoupled detection network architecture to extract multi-scale spatial features from the input image frame and regress the bounding box position and class probability. Then, the detection results are input into the tracking module containing state prediction and association matching logic for initialization to obtain the object detection and multi-object tracking model.

[0032] Pig target localization data is a collection describing the specific location and identity information of each identified pig in a particular image frame. For example, pig target localization data may include a sow bounding box composed of the diagonal coordinates of a rectangle, a unique ID code representing the specific identity of the pig, a basic classification label representing the sow or piglet, and a sequence of continuous motion trajectory coordinates spanning multiple frames.

[0033] In some embodiments, the processor can input top-down continuous video image data into the target detector to perform feature extraction and decoupled regression to generate initial detection boxes and class probabilities. Then, these initial data are input into the tracking algorithm layer, where the current state is predicted by filtering and the cost matrix is ​​calculated by matching with an existing trajectory database. If a match is successful, the identity identifier is updated to obtain the pig target location data.

[0034] In some embodiments, the processor can acquire continuous overhead video image data of sows and piglets in the farrowing house, perform feature extraction and position probability prediction processing using a target detector layer to obtain initial detection feature data containing position coordinates and category information; based on the initial detection feature data, perform motion trajectory prediction and data association matching processing using a multi-target tracking algorithm layer to obtain pig target localization data containing pig bounding boxes, classification labels, and motion trajectories; wherein, the target detector layer and the multi-target tracking algorithm layer belong to a target detection and multi-target tracking model.

[0035] Initial detection feature data consists of the raw recognition results output by the front-end detection network based on a single frame image, without cross-frame correlation. For example, initial detection feature data may include the two-dimensional spatial coordinates of the sow and multiple piglets in the current frame, the confidence scores predicted to each category, and the category probability vectors of basic postures such as standing or lying down.

[0036] In some embodiments, the processor can convert the original top-down video image into a tensor format acceptable to the network, feed it into the convolutional backbone network of the object detector for multi-scale feature map extraction, and then perform classification and regression calculations via the anchorless decoupled detection head, directly outputting the target attributes of the current image without relying on historical frame information, thus obtaining initial detection feature data.

[0037] In some embodiments, the target detection and multi-target tracking model includes: a target detector layer, used to perform feature extraction and classification regression processing based on top-down continuous video image data using an anchor-free decoupled detection method to obtain initial detection feature data; and a multi-target tracking algorithm layer, used to perform trajectory update processing based on the initial detection feature data using a filtering prediction and Hungarian matching algorithm to obtain pig target localization data containing pig bounding boxes, classification labels, and movement trajectories.

[0038] S2: Based on the target localization data of pigs, local image sequence data of a single pig is generated by cropping from the continuous video image data from the top view, and the temporal behavior classification model in the two-stage cascaded model is used for feature extraction and temporal classification processing to obtain the behavior category determination data of each pig.

[0039] Local image sequence data is a collection of short video clips that have been stripped of complex backgrounds and contain only continuous movements of a single, specified pig. For example, local image sequence data may include several consecutive frames of footage of a sow turning over, a close-up clip of a piglet's movement before it is obscured by the sow's body, or a local pixel matrix that has been normalized to a uniform size and arranged in chronological order.

[0040] In some embodiments, the processor can read the detection box coordinates in the pig target positioning data, perform pixel cropping and stripping on the corresponding coordinate area in the original top-down continuous video image frames, combine and assemble the cropped images for the same identity ID with a set time sliding window step, and perform interpolation and scaling to a uniform preset size to obtain local image sequence data.

[0041] Temporal behavior classification models are deep learning models specifically designed to parse continuous image sequences spanning a time span to determine fine-grained behavioral semantics. For example, a temporal behavior classification model may include parameters of a two-dimensional convolutional network responsible for short-term feature exchange, channel attention weight matrices responsible for enhancing target regions, and parameters of a selective state space layer responsible for capturing the evolution of actions over a second.

[0042] In some embodiments, the processor can embed time-shifting logic into a standard two-dimensional convolutional backbone network, connect it in parallel to a channel attention structure composed of one-dimensional convolutions, then reshape the dimension to connect a selective state space module with linear complexity, and finally connect a linear classification layer. The processor can then perform end-to-end optimization training using local video samples of pigs with temporal behavior labels to obtain a temporal behavior classification model.

[0043] Behavioral category determination data refers to the specific behavioral attributes output by the model after comprehensively analyzing an input sequence of actions. For example, behavioral category determination data may include discretized classification labels indicating whether the pig is classified as a sow turning from standing to lying down, a piglet nursing normally, or a piglet being crushed, as well as an array of prediction confidence probability distributions for each behavioral category.

[0044] In some embodiments, the processor can send long-term temporal context feature data into a fully connected layer for dimensional mapping transformation, perform average pooling fusion and aggregation on the feature vectors of all frames in the time dimension, and then process them through a normalized exponential function to generate the probability distribution of each candidate category, select the item with the highest probability, and obtain the behavior category determination data.

[0045] In some embodiments, the processor can perform target region cropping and temporal slice reconstruction processing on top-down continuous video image data based on the detection box coordinates in the pig target localization data to obtain local image sequence data of a single pig; based on the local image sequence data, a temporal behavior classification model is used to perform short-spatial feature extraction, channel attention focusing, and long-distance temporal dependency modeling to obtain behavior category determination data for each pig.

[0046] The bounding box coordinates are geometric values ​​that define the spatial extent of the pig within a two-dimensional image plane. For example, the bounding box coordinates may include the horizontal and vertical pixel coordinates of the top-left and bottom-right corners of the rectangle, the two-dimensional coordinates of the target's center point, and the relative width and height ratios of the bounding box.

[0047] In some embodiments, the processor can perform dense predictions on multi-scale feature maps using the decoupled detection head of the front-end detector, calculate the offset and scaling factor relative to the grid, and then map them back to the physical resolution coordinate system of the original video image for decoding and boundary constraints to obtain the detection box coordinates.

[0048] In some embodiments, the temporal behavior classification model includes: a two-dimensional convolutional backbone module, used to perform inter-frame displacement and short-term spatiotemporal feature extraction processing based on local image sequence data using a temporal shift mechanism to obtain multi-dimensional feature data; a channel attention module, used to perform channel attention weighting processing on the multi-dimensional feature data using one-dimensional convolution and adaptive channel correlation modeling to obtain spatially enhanced feature data; a spatial pooling and dimensionality reduction projection module, used to perform spatial dimension compression and sequence reconstruction processing on the spatially enhanced feature data using global average pooling and linear mapping to obtain low-dimensional temporal feature sequence data; a selective state space module, used to perform long-distance temporal dependency modeling and discretized state update processing on the low-dimensional temporal feature sequence data through an input-dependent selective mechanism to obtain long-term temporal context feature data; and a linear classification module, used to output behavioral category determination data for each pig based on the long-term temporal context feature data using temporal dimension average fusion and fully connected mapping processing.

[0049] Multidimensional feature data is a high-dimensional tensor extracted by the backbone network that simultaneously contains spatial texture appearance and short-term motion cues between adjacent frames. For example, multidimensional feature data may include local spatial activation response maps that preserve the image's dimensions, data blocks containing multi-level semantic channel information, and spatiotemporal hybrid feature maps that fuse partial channel information from adjacent frames.

[0050] In some embodiments, the processor can input local image sequence tensors into a two-dimensional convolutional backbone network embedded with temporal shift logic, shift and exchange feature channels along the channel dimension by a set proportion one frame forward or backward, and then perform conventional spatial two-dimensional convolution operations to extract local detail features and obtain multidimensional feature data.

[0051] In some embodiments, when the local image sequence data is input into a two-dimensional convolutional backbone module and a temporal shift mechanism is used for inter-frame displacement and short-time spatiotemporal feature extraction, the expression for the temporal shift feature tensor corresponding to the displacement operation is: ; in, This represents a temporal shift feature tensor, used to construct the multidimensional feature data after extraction via two-dimensional convolution. This represents the original five-dimensional feature tensor obtained after tensor reconstruction of the local image sequence data. This represents the index value of the current time frame. Indicates the index value of the feature channel. This represents a preset channel displacement scaling factor, used to control the number of feature channels participating in time-dimensional displacement. This represents the total number of channels in the original five-dimensional feature tensor.

[0052] Spatial augmentation feature data is a feature tensor that has been recalibrated using an attention mechanism to highlight key behavioral regions and suppress background interference. For example, spatial augmentation feature data can include adaptively weighted channel activation maps, feature matrices that amplify feature response values ​​for pressure points in piglets, and spatial semantic features that have removed static redundant information such as farrowing crate railings.

[0053] In some embodiments, the processor can generate channel descriptors by performing adaptive global two-dimensional average pooling on the input multidimensional feature data, analyze the interaction correlation between channels using adaptive-sized one-dimensional convolution to generate activation weights, and finally perform a channel-wise element-wise multiplication weighting operation on the weight vector and the original feature tensor to obtain spatially enhanced feature data.

[0054] In some embodiments, the expression for spatial enhancement feature data is: ; in, Representation spatially enhanced feature data, Represents multidimensional feature data. This represents an adaptive global two-dimensional average pooling operation. Indicates the kernel size as One-dimensional local convolution operation, This represents the local convolutional kernel size calculated based on an adaptive function, used to adaptively determine the range of cross-channel interactions. This represents the Sigmoid activation function. This indicates a weighted operation of element-wise multiplication of channels.

[0055] Low-dimensional temporal feature sequence data compresses the spatial two-dimensional structure, transforming each frame of an image into a sequence of one-dimensional feature vectors with high-density semantics and low dimensionality. For example, low-dimensional temporal feature sequence data can include a single-frame global descriptor array arranged in chronological order, a three-dimensional tensor with its spatial width and height dimensions removed, retaining only the time axis and channel axis, and a feature sequence that has undergone linear dimensionality reduction to reduce subsequent computational overhead.

[0056] In some embodiments, the processor can completely eliminate the length and width spatial dimensions by performing a global average pooling operation on the spatially enhanced feature data, transform it into a temporal sequence composed of independent frame vectors using a reshaping operation, and then perform channel number compression and dimensionality reduction processing through a linear projection mapping layer without activation functions to obtain low-dimensional temporal feature sequence data.

[0057] In some embodiments, the discretization update expression for low-dimensional time-series feature sequence data is: ; in, Indicates the first The temporal hidden state data at each time step is obtained by concatenating the sequences of all hidden state data along the time dimension to obtain long-term temporal context feature data. Indicates the first Temporal hidden state data at any given time. Represents the extraction of the first from low-dimensional time-series feature sequence data. Input feature data at time step This represents the adaptive time-series step size parameter of the model, dynamically generated on a time scale based on low-dimensional time-series feature sequence data. Represents the state transition matrix. This represents the input control matrix generated based on the discretization rules. This indicates the operation of the exponential function.

[0058] In some embodiments, when the spatial enhancement feature data is input into the spatial pooling and dimensionality reduction projection module, and spatial dimension compression and sequence reconstruction are performed using global average pooling and linear mapping, the expression for the low-dimensional temporal feature sequence data is: ; in, This represents the low-dimensional time-series feature sequence data. This represents the spatial enhancement feature data. This represents an adaptive global two-dimensional average pooling operation. This represents a tensor dimension reconstruction operation, used to convert a spatially compressed tensor into a three-dimensional format arranged in a time series. This represents the linearly reduced-dimensional projected weight matrix, obtained through end-to-end network training iterations. This represents the linear dimension-reduced projection bias vector, obtained through end-to-end network training iterations.

[0059] Long-term temporal contextual feature data is a feature representation that integrates global behavioral evolution information spanning a relatively long time window. For example, long-term temporal contextual feature data can include latent state variables of a sow from standing to finally lying down in a continuous process, key turning point feature sequences retained after filtering out useless action frames through a selective mechanism, and sequence context vectors that fuse historical and current input information.

[0060] In some embodiments, the processor can input low-dimensional temporal feature sequence data into a selective state space module, dynamically generate an adaptive temporal step size and control matrix based on the current frame features, use a discretized state update formula to combine the hidden state of the previous time step with the current input to update the current state, and concatenate the outputs of all time steps to obtain long-term temporal context feature data.

[0061] In some embodiments, when the long-term temporal contextual feature data is input into the linear classification module and processed using temporal averaging fusion and fully connected mapping, the expression for the initial classification probability matrix of the behavior category determination data is: ; ; in, This represents the initial classification probability matrix, which, after Softmax activation, is mapped to the probability distributions of each category in the behavior category determination data. This represents the video-level global feature vector obtained after averaging and fusing along the time dimension. This represents the total number of time segments in the sequence, i.e., the number of frames in the extracted local image sequence. This represents the long-term temporal context feature data. This represents the linear up-dimensional projection weight matrix. This represents the linear up-dimensional projection bias vector. This represents the weight matrix of the fully connected mapping in the final classification layer. This represents the bias vector of the final classification layer.

[0062] S3: Based on behavioral category determination data and pig target location data, anomaly comparison and result verification are performed using a preset piglet number verification mechanism and behavioral prior rules to obtain the final crushing state analysis results.

[0063] The final crushing status analysis result is a definitive conclusion that integrates AI visual recognition, pig count statistics, and biological correction. For example, the final crushing status analysis result may include a Boolean alarm flag that clearly indicates a sow lying down and crushing piglets, a record of the specific pen number that triggered the alarm, a valid abnormal behavior classification label that has been verified, and its corresponding corrected confidence value.

[0064] In some embodiments, the processor can receive image-based temporal classification results and quantity-based anomaly determination results in parallel, and simultaneously place the temporal classification results into a logical rule base containing mutually exclusive body shapes and spatial overlaps for matching and verification. It can also weaken the confidence of the violation results or force the correction of the category, and combine these effective information to determine whether the warning threshold is met, thus obtaining the final compression state analysis result.

[0065] In some embodiments, the processor can trigger a piglet count verification mechanism based on the sow position change recognition result in the behavior category determination data to compare and statistically process the currently visible number of piglets in the pig target location data with the preset total number in the system, and obtain count verification anomaly determination data; based on the behavior category determination data, logical filtering processing is performed on the physical categories of action jumps and detection box sizes in adjacent time periods using behavioral prior rules, and confidence reduction correction processing is performed based on the spatial coverage rate of the target box to obtain rule correction status data; based on the count verification anomaly determination data and rule correction status data, a comprehensive evaluation processing is performed to obtain the final crushing state analysis result.

[0066] The results of sow position change recognition are classification signals of significant changes in the body posture of adult sows detected by time-series models. For example, the results of sow position change recognition may include action trigger flags for the sow to change from standing to sitting, category labels for changing from sitting to lying on its side or prone, recognition timestamps for the sow rolling over, and probability scores of the corresponding actions.

[0067] In some embodiments, the processor can extract the output prediction for the sow's identity ID separately from the behavior category determination data in the backend, compare it with the category label state sequence in several consecutive time windows, and mark the state jump when a preset state transition path is detected and the confidence level is higher than the threshold, so as to obtain the sow's body position change recognition result.

[0068] The current visible number of piglets is the total number of live piglets whose outlines can be clearly identified by the front-end algorithm within the camera's field of view at the specific moment when the sow's posture changes. For example, the current visible number of piglets can include the number of normal piglets that are moving freely, the target number of piglets identified as being in a state of edge compression, and the effective number of piglets detected after excluding dead piglets and placental interference.

[0069] In some embodiments, the processor can immediately retrieve the bounding box list output by the front-end target detection and multi-target tracking model at the current timestamp after receiving the sow position change signal, traverse the list to filter out all bounding boxes whose classification labels belong to normal piglets or related survival status, deduplicate them, count and accumulate them to obtain the number of piglets currently visible.

[0070] The preset system total is the theoretical standard number of piglets actually surviving in a specific farrowing stall, as recorded in the pig farm management system backend. For example, the preset system total may include the total number of piglets in the litter manually entered by the farmer, the real-time inventory verification value after deducting the number of registered deaths and culles, and a fixed reference baseline value used for comparison with the visual roll call results.

[0071] In some embodiments, the processor can initiate a data query request to the central database or edge node server of the pig farm through a network communication interface, parse the corresponding farrowing limit field ID based on the physical number of the camera device to which the current video stream belongs, retrieve the latest storage record field of the field and cache it, and obtain the preset system total.

[0072] The anomaly detection data for quantity verification is a differential conclusion drawn from comparing the number of pigs detected in the current visual image with the total number registered in the background. For example, the anomaly detection data for quantity verification may include missing differences where the actual count is less than the theoretical count, Boolean alarm variables marked as indicating potential occlusion or crushing risks, and high-risk warning weight scores triggered when the quantity comparison is inconsistent.

[0073] In some embodiments, the processor can execute an arithmetic comparison logic unit to subtract or compare the integer value of the number of currently visible piglets extracted in real time with the integer value of the preset total number read from the system. If it is found that the number of visible piglets is less than the preset total number, an abnormal Boolean or enumerated variable structure containing missing cases and differences is generated to obtain the number verification anomaly judgment data.

[0074] Spatial coverage ratio is a geometric ratio that describes the degree to which two different target bounding boxes overlap and enclose each other on a two-dimensional image plane. For example, spatial coverage ratio can include a near-1 overlap ratio when a large sow bounding box completely encloses a small piglet bounding box, or a percentage of the total area of ​​the target whose body is partially occluded.

[0075] In some embodiments, the processor can extract the coordinate values ​​of the current target bounding box and all historically stable target bounding boxes from the pig target localization data, perform polygon intersection operation to calculate the absolute pixel area of ​​the intersection region of the two rectangles, and then divide the intersection area by the absolute total area of ​​the current target's own bounding box to obtain the spatial coverage rate.

[0076] Rule-corrected state data is the final behavioral judgment information that has been modified or weakened in confidence after being processed by a behavioral common sense filter. For example, rule-corrected state data may include the original confidence value that has been reduced due to spatial inconsistencies such as large boxes containing smaller boxes, a new classification label that has been forcibly corrected from sow to normal piglet due to the box area being too small, and the previous frame's safe state result that has been forcibly retained due to an illegal state transition.

[0077] In some embodiments, the processor can take the initial classification category and confidence level output by the time series model as input, and sequentially pass through the area size mutual exclusion check, the time series state transition machine consistency check, and the bounding box overlap rate calculation logic branch. When a judgment condition that violates physical common sense is hit, the processor can perform a multiplication operation to reduce the confidence level according to a preset penalty factor or directly overwrite the category field memory to obtain the rule correction state data.

[0078] In some embodiments, the expressions for spatial coverage rate and rule correction state data are as follows: ; ; in, Indicates spatial coverage. This indicates the current target in the pig target location data. Bounding box data, This indicates a stable historical target extracted from pig target location data. Bounding box data, This function represents the operation of calculating the area based on the bounding box's width and height coordinates. This indicates the operation of finding the intersection of spaces. This represents the reduced target confidence level output in the rule-corrected state data. Indicates the current target in the behavior category determination data. The original classification confidence score, This represents the preset confidence level suppression penalty coefficient; when the OverlapRatio is greater than 0.9, rule correction is triggered, and then the probability is multiplied by the penalty coefficient to obtain the final probability.

[0079] S4: Analyze the final crushing status analysis results. If either a crushing event or a missing number of piglets is detected, a piglet crushing early warning signal is generated, thus completing the piglet crushing early warning.

[0080] Piglet crushing warning signals are control commands or message messages ultimately used to drive external equipment or notify managers to take rescue actions. For example, piglet crushing warning signals may include alarm pop-up data packets containing time and pen number sent to the information panel of the inspection robot, level control signals that trigger the sound and light alarms in the pen, and alarm string messages with screenshots of abnormal scenes pushed to the farmer's mobile terminal via wireless communication network.

[0081] In some embodiments, the processor can monitor the memory block of the final squeezing status analysis results. When it captures a high-confidence tag that still confirms the squeezing event after rule verification, or captures an abnormal status code that is missing from the quantity verification, it can immediately assemble a standardized communication protocol data frame containing the device ID of the event, the timestamp of the event, and the type of the abnormality, and send it to the relevant receiving terminal through the network interface to obtain a piglet squeezing early warning signal.

[0082] In some embodiments, to ensure that the above formulas and logical judgment mechanisms can stably adapt to the physiological characteristics of pigs and on-site computing resources, the system presets several constant parameters, the specific value ranges and default settings of which are as follows: The proportional coefficient for controlling the displacement of the channel The value range is [0.1, 0.25]. In this embodiment, for a time sequence sample of 8 consecutive frames, The default value is 1 / 8; the feature dimension constant after dimensionality reduction. The value range is configured as [128, 512], and its default optimal value is 256 to balance calculation accuracy and processing latency; in the behavioral prior rules, the minimum sow frame area threshold used to distinguish between sows and piglets is set to 20,000 pixels (based on...). (Resolution baseline dynamic scaling); In the state correction stage, the default value of the confidence penalty coefficient for illegal state transitions is 0.7, the default value of the confidence penalty coefficient for mutually exclusive size and category cases is 0.8, and the confidence suppression penalty coefficient for spatial inconsistencies (such as large frames containing small frames) is... It is even more stringent, with a default value of 0.6.

[0083] In some embodiments, to verify the performance of the front-end detection model, performance evaluation experiments of mainstream target recognition models were conducted. Table 1 shows the performance evaluation table of mainstream target recognition models, which specifically illustrates the comparison of different convolutional neural network and visual Transformer architecture models in key indicators such as precision, recall, mean precision, inference speed, and model size.

[0084] Table 1 Performance Evaluation of Mainstream Target Recognition Models

[0085] In some embodiments, model ablation experiments were conducted to verify the effectiveness of each improved module in the time-series behavior classification model. Table 2 shows the results of the ablation experiment of the piglet crushing behavior classification model, specifically illustrating the changes in the Top-1 accuracy and Top-5 accuracy of the model when the Mamba module and the ECA module were introduced separately or simultaneously on the basic TSM model.

[0086] Table 2 Ablation Experiment Results of Piglet Crushing Behavior Classification Model

[0087] In some embodiments, comparative experiments were conducted to verify the compatibility of different attention mechanisms with the Mamba temporal module. Table 3 shows the results of the comparative experiments of different attention mechanisms, specifically illustrating the differences in Top-1 accuracy and Top-5 accuracy of the basic TSM model and the M-TSM model combined with the Mamba module, after adding Self Attention (SA), Squeezing and Activation (SE), Convolutional Block Attention (CBAM), and Efficient Channel Attention (ECA) respectively.

[0088] Table 3. Results of the comparative experiment on different attention mechanisms

[0089] In some embodiments, to verify the robustness of the model in capturing long temporal action features, comparative experiments with mainstream models were conducted at different temporal lengths. Table 4 shows the results of the comparative experiments with mainstream models at different temporal lengths, specifically illustrating the fluctuations and comparisons of model accuracy for TSN, TSM, VIVIT, VideoMAE, Slowfast, and the method proposed in this invention when inputting consecutive image sequences of 6, 8, 10, and 12 frames.

[0090] Table 4. Comparison of experimental results of mainstream models under different time series lengths.

[0091] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.

Claims

1. A method for early warning of piglet crushing based on a two-stage cascaded model, characterized in that, include: S1: Acquire continuous overhead video image data of sows and piglets in the farrowing house, and use the target detection and multi-target tracking model in the two-stage cascaded model for identification and tracking processing to obtain pig target localization data including pig bounding boxes, classification labels and movement trajectories; S2: Based on the target localization data of pigs, local image sequence data of a single pig is generated by cropping from the continuous video image data from the top view, and the temporal behavior classification model in the two-stage cascaded model is used for feature extraction and temporal classification processing to obtain the behavior category determination data of each pig. S3: Based on behavioral category determination data and pig target positioning data, anomaly comparison and result verification are performed using a preset piglet number verification mechanism and behavioral prior rules to obtain the final crushing state analysis results. S4: Analyze the final crushing status analysis results. If either a crushing event or a missing number of piglets is detected, a piglet crushing early warning signal is generated, thus completing the piglet crushing early warning.

2. The piglet crushing early warning method based on a two-stage cascaded model according to claim 1, characterized in that, S1 includes: We acquire continuous overhead video images of sows and piglets in the farrowing house, and use a target detector layer to perform feature extraction and position probability prediction to obtain initial detection feature data containing position coordinates and category information. Based on the initial detection feature data, a multi-target tracking algorithm layer is used to perform motion trajectory prediction and data association matching to obtain pig target localization data containing pig bounding boxes, classification labels, and motion trajectories; among them, the target detector layer and the multi-target tracking algorithm layer belong to the target detection and multi-target tracking model.

3. The piglet crushing early warning method based on a two-stage cascaded model according to claim 2, characterized in that, The target detection and multi-target tracking model includes: The target detector layer is used to perform feature extraction and classification regression processing based on continuous overhead video image data using an anchor-free decoupled detection method to obtain initial detection feature data. The multi-target tracking algorithm layer is used to update the trajectory based on the initial detection feature data using filtering prediction and Hungarian matching algorithm, so as to obtain pig target localization data containing pig bounding boxes, classification labels and movement trajectories.

4. The piglet crushing early warning method based on a two-stage cascaded model according to claim 1, characterized in that, S2 includes: Based on the detection box coordinates in the pig target localization data, target region cropping and temporal slice reconstruction are performed in the top-view continuous video image data to obtain local image sequence data of a single pig. Based on local image sequence data, a temporal behavior classification model is used to extract short-term spatiotemporal features, focus channel attention, and model long-distance temporal dependencies to obtain behavioral category determination data for each pig.

5. The piglet crushing early warning method based on a two-stage cascaded model according to claim 4, characterized in that, The temporal behavior classification model includes: The two-dimensional convolutional backbone module is used to perform inter-frame displacement and short-time spatiotemporal feature extraction based on local image sequence data using a temporal shift mechanism to obtain multi-dimensional feature data. The channel attention module is used to perform channel attention weighting on multidimensional feature data using one-dimensional convolution and adaptive channel correlation modeling to obtain spatially enhanced feature data. The spatial pooling and dimensionality reduction projection module is used to perform spatial dimension compression and sequence reconstruction on spatially enhanced feature data using global average pooling and linear mapping to obtain low-dimensional temporal feature sequence data. The selective state space module is used to perform long-distance temporal dependency modeling and discretization state update processing on low-dimensional temporal feature sequence data through a selective mechanism of input dependency, so as to obtain long-term temporal context feature data. The linear classification module is used to output behavioral category determination data for each pig by using time-dimensional average fusion and fully connected mapping based on long-term temporal contextual feature data.

6. The piglet crushing early warning method based on a two-stage cascaded model according to claim 5, characterized in that, The expression for the spatial enhancement feature data is: ； in, Representation spatially enhanced feature data, Represents multidimensional feature data. This represents an adaptive global two-dimensional average pooling operation. Indicates the kernel size as One-dimensional local convolution operation, This represents the local convolutional kernel size calculated based on an adaptive function, used to adaptively determine the range of cross-channel interactions. This represents the Sigmoid activation function. This indicates a weighted operation of element-wise multiplication of channels.

7. The piglet crushing early warning method based on a two-stage cascaded model according to claim 5, characterized in that, The discretization update expression for the low-dimensional time-series feature sequence data is: ； in, Indicates the first The temporal hidden state data at each time step is obtained by concatenating the sequences of all hidden state data along the time dimension to obtain long-term temporal context feature data. Indicates the first Temporal hidden state data at any given time. Represents the extraction of the first from low-dimensional time-series feature sequence data. Input feature data at time step This represents the adaptive time-series step size parameter of the model, dynamically generated on a time scale based on low-dimensional time-series feature sequence data. Represents the state transition matrix. This represents the input control matrix generated based on the discretization rules. This indicates the operation of the exponential function.

8. The piglet crushing early warning method based on a two-stage cascaded model according to claim 1, characterized in that, S3 includes: Based on the sow position change recognition results in the behavior category judgment data, the piglet number verification mechanism is triggered to compare and statistically process the currently visible number of piglets in the pig target location data with the preset total number in the system, and obtain the number verification anomaly judgment data. Based on the behavior category determination data, the behavior prior rules are used to perform logical filtering on the physical categories of action jumps and detection box sizes in adjacent time periods, and confidence reduction correction is performed based on the spatial coverage rate of the target box to obtain the rule correction state data. Based on the anomaly detection data and rule correction status data, a comprehensive evaluation and processing is performed to obtain the final extrusion status analysis results.

9. The piglet crushing early warning method based on a two-stage cascaded model according to claim 8, characterized in that, The expressions for the spatial coverage rate and the rule correction state data are as follows: ； ； in, Indicates spatial coverage. This indicates the current target in the pig target location data. Bounding box data, This indicates a stable historical target extracted from pig target location data. Bounding box data, This function represents the operation of calculating the area based on the bounding box's width and height coordinates. This indicates the operation of finding the intersection of spaces. This represents the reduced target confidence level output in the rule-corrected state data. Indicates the current target in the behavior category determination data. The original classification confidence score, This represents the preset confidence level suppression penalty coefficient.

10. A piglet crushing early warning system based on a two-stage cascaded model, used to execute the piglet crushing early warning method based on a two-stage cascaded model as described in any one of claims 1 to 9, characterized in that, include: The recognition module is used to acquire continuous video image data of sows and piglets from above in the farrowing house. It uses the target detection and multi-target tracking model in the two-stage cascaded model for recognition and tracking processing to obtain pig target localization data including pig bounding boxes, classification labels and movement trajectories. The classification module is used to crop local image sequence data of a single pig from the continuous overhead video image data based on the target positioning data of pigs, and to perform feature extraction and temporal classification processing using the temporal behavior classification model in the two-stage cascaded model to obtain the behavior category determination data of each pig. The comparison module is used to perform anomaly comparison and result verification based on behavioral category judgment data and pig target location data, using a preset piglet number verification mechanism and behavioral prior rules, to obtain the final crushing state analysis result. The early warning module is used to analyze the final crushing status analysis results. When it is determined that either a crushing event exists or the number of piglets is missing, a piglet crushing early warning signal is generated, thus completing the piglet crushing early warning.

Images