Pig farm abnormity monitoring system and method based on image recognition

Abstract

The invention relates to the technical field of image recognition and intelligent monitoring, and particularly discloses a pig farm abnormity monitoring system and method based on image recognition. According to the system, videos are synchronously acquired through a multi-view image acquisition array, target detection is performed by an edge computing node, cross-view fusion tracking and three-dimensional attitude reconstruction are performed by a cloud server, and continuous individual three-dimensional motion trail and attitude time sequence features are constructed. On the basis, a normal behavior mode is learned through a deep auto-encoder model, and automatic detection and early warning of abnormal behaviors of pigs are realized in combination with group statistical comparison. According to the method, shielding interference is overcome, and behavior monitoring continuity and analysis accuracy are improved.

Description

Technical Field

[0001] This invention belongs to the field of image recognition and intelligent monitoring technology, specifically relating to an image recognition-based abnormal monitoring system and method for pig farms. Background Technology

[0002] In modern livestock farming, especially large-scale pig farming, utilizing intelligent technologies to improve production management efficiency and ensure animal welfare has become an important development trend. Among these, computer vision-based automated monitoring systems provide key technical support for early disease warning, growth status assessment, and abnormal event detection by analyzing visual information such as pig behavior and posture.

[0003] Image recognition-based anomaly monitoring in pig farms aims to continuously collect video data using cameras deployed in pigsties, and automatically identify and track individual pigs using target detection and tracking algorithms, thereby analyzing their behavioral patterns to determine if any anomalies exist. The basic principle of this technology is to locate and continuously follow each target individual in complex scenes using a model.

[0004] Existing technologies typically employ a general object detection framework combined with multi-object tracking algorithms to identify and track pigs. However, in real-world scenarios involving intensive pig farming, frequent and severe occlusion between individuals can lead to broken detection frames and blurred target appearance features, making tracking and association extremely prone to failure, with individual tracking loss rates often exceeding 35%.

[0005] This tracking interruption directly leads to the inability to construct continuous and complete behavioral sequence data for the same pig, rendering subsequent anomaly analysis based on time-series behavior unreliable. Therefore, how to achieve robust and continuous tracking of individual pigs in complex farming environments with high density and strong occlusion is a key challenge restricting the practical application of vision-based anomaly monitoring technology in pig farms. Summary of the Invention

[0006] The purpose of this invention is to provide an image recognition-based system and method for monitoring anomalies in pig farms, in order to solve the technical contradiction that existing target tracking technologies suffer from high tracking interruption rates and the inability to construct continuous behavioral sequences due to frequent occlusion between individuals in high-density, heavily occluded pig housing environments, thus rendering subsequent anomaly analysis ineffective.

[0007] To achieve the above objectives, this invention provides an image recognition-based anomaly monitoring system for pig farms. The system includes a multi-view image acquisition array deployed within the pigsty, edge computing nodes, a cloud analytics server, and a management terminal.

[0008] The multi-view image acquisition array consists of multiple high-definition network cameras arranged in a preset stereo layout, used to synchronously acquire real-time video streams of the pigsty from different angles.

[0009] Edge computing nodes are directly connected to the image acquisition array to perform preliminary image preprocessing and target detection.

[0010] The cloud-based analytics server communicates with edge computing nodes via the network to perform core cross-view fusion tracking, 3D pose reconstruction, and abnormal behavior analysis. The management terminal receives and displays the analysis results and alerts.

[0011] Furthermore, the system employs a hierarchical and progressive architecture for individual identity maintenance and behavior analysis. The core of this architecture lies in overcoming the occlusion problem under a single viewpoint through cross-perspective data fusion and 3D modeling techniques, and providing richer spatial information for behavior analysis.

[0012] As one embodiment of the present invention, the target detection process performed by the edge computing node is as follows.

[0013] The edge computing nodes load a lightweight pig target detection model, which is trained on an improved YOLO architecture and specifically optimized for the morphological features of pigs.

[0014] The model receives video frames from a single camera and outputs the two-dimensional bounding boxes of all pigs in each frame and their confidence scores.

[0015] Edge computing nodes perform non-maximum suppression processing on the detection results to filter out redundant boxes with excessive overlap, and encapsulate the processed bounding box coordinates, timestamps, and corresponding camera numbers into data packets, which are then uploaded to the cloud analysis server in real time.

[0016] Furthermore, the cloud-based analysis server includes a cross-view fusion tracking module, a 3D pose reconstruction module, a temporal behavior modeling module, and an anomaly detection module.

[0017] The cross-view fusion tracking module is the core of the system, responsible for fusing two-dimensional detection data from different cameras, which may be severely occluded, into a continuous and reliable three-dimensional individual motion trajectory.

[0018] As one embodiment of the present invention, the workflow of the cross-view fusion tracking module is as follows.

[0019] This module maintains a global set of individual pigs, each with a unique identifier. For each set of 2D detection boxes synchronously uploaded from all cameras at any given time, the module first performs time alignment.

[0020] Next, the module calls a pre-calibrated multi-view geometric model, which is built based on the camera's intrinsic and extrinsic parameters.

[0021] The module attempts to back-project two-dimensional detection boxes from different perspectives and with similar spatial positions at the same time onto a three-dimensional space in a preset world coordinate system using a geometric model.

[0022] For multiple 2D detection frames that can successfully converge at a reasonable 3D spatial point through back projection, the module determines that they belong to the observation of the same pig individual from different perspectives.

[0023] For detection boxes that cannot be confirmed through multi-view intersection, such as a pig that is completely occluded in most views and only visible in one view, the module will activate an association algorithm based on appearance features and motion consistency.

[0024] The algorithm extracts the depth features of the isolated detection box and calculates the cosine similarity with the individual features of the most recent moment in the historical trajectory. At the same time, it predicts the motion state of the individual based on Kalman filtering and performs weighted fusion of feature similarity and distance to the motion prediction position to determine whether they belong to the same identity.

[0025] By employing a strategy that prioritizes multi-view geometric intersection and supplements it with feature motion correlation, the system assigns the most probable three-dimensional spatial coordinates to each pig at every moment and updates its identity trajectory.

[0026] As one embodiment of the present invention, the workflow of the three-dimensional pose reconstruction module is as follows.

[0027] This module receives individual data with confirmed identity and 3D location from the cross-view fusion tracking module.

[0028] For each individual, the module selects at least two original image regions from its multiple visible viewpoints that have the least occlusion.

[0029] The module loads a pre-trained 3D pig keypoint detection model. The model takes multi-view image patches as input and outputs the coordinates of 17 predefined anatomical keypoints on the pig's torso in 3D space.

[0030] These key points include the tip of the nose, eyes, base of the ears, shoulders, hips, and hooves. Using these 17 key points, the module can fit a simplified three-dimensional skeletal model, thereby quantifying an individual's standing, lying, and walking postures.

[0031] Furthermore, the workflow of the temporal behavior modeling module is as follows.

[0032] This module receives the three-dimensional position coordinates and three-dimensional attitude key point coordinates of each individual over a continuous time series.

[0033] The module first calculates a series of basic behavioral indicators, including but not limited to: instantaneous velocity and acceleration calculated based on three-dimensional position difference; Posture stability index calculated based on the height variance of trunk key points. Feeding proximity calculated based on the positional relationship between the key point at the tip of the nose and the preset area of ​​the feed trough; Social contact frequency and intensity based on three-dimensional distance calculation between individuals.

[0034] These basic indicators are sampled once per second to form a multidimensional time series.

[0035] As one embodiment of the present invention, the workflow of the anomaly detection module is as follows. This module includes an offline training phase and an online execution phase.

[0036] During the offline training phase, the module collects time-series data of multidimensional behavioral indicators of all individuals in the pig farm under normal and healthy conditions over a long period of time.

[0037] Using this data, the module trains deep autoencoder models for pigs at different growth stages, such as the nursery and fattening periods.

[0038] The encoder of this autoencoder compresses high-dimensional temporal behavioral features into a low-dimensional latent space, while the decoder attempts to reconstruct the input features from the latent space.

[0039] The training objective is to minimize the model's reconstruction error of normal behavior patterns.

[0040] During the online operation phase, for each individual's behavioral indicator sequence generated in real time, the module first inputs it into the corresponding growth stage autoencoder to calculate its reconstruction error.

[0041] At the same time, the module calculates the statistical deviation between the individual's current behavioral indicators and the distribution of the overall behavioral indicators of the pig herd during the same period, such as Mahalanobis distance.

[0042] The module sets two thresholds: Reconstruction error threshold and statistical deviation threshold. When an individual's reconstruction error exceeds its threshold for 60 consecutive seconds, and its statistical deviation also exceeds its threshold, the individual's behavior is deemed abnormal.

[0043] Abnormalities are initially categorized based on specific behavioral indicators that exceed the standard. For example, prolonged low movement speed and poor postural stability may indicate disease, while high-frequency and high-intensity close proximity between individuals may indicate fighting.

[0044] Furthermore, the system also includes a feedback learning and model optimization mechanism.

[0045] All events deemed abnormal by the system, along with at least 5 minutes of multi-view video footage before and after them and all analysis data, will be pushed to the management terminal for manual review.

[0046] Administrators confirm or correct annotations on the terminal.

[0047] These labeled data will be automatically imported into the training dataset for periodic incremental training of the object detection model, 3D keypoint model, and anomaly detection autoencoder, thereby achieving continuous iterative improvement of system performance.

[0048] As one embodiment of the present invention, the stereoscopic layout of the multi-view image acquisition array follows the following principles: The array should cover all areas of the pigsty that need to be monitored, with no blind spots; Any point in space must be covered by the field of view of at least two different cameras; The fields of view of adjacent cameras overlap; The camera's installation height and angle must ensure that it can capture the pig's back and side profile to facilitate 3D reconstruction.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention innovatively transforms the two-dimensional tracking problem under a single viewpoint into a three-dimensional spatial localization problem under multiple viewpoints by deploying a multi-view image acquisition array and constructing a cross-viewpoint fusion tracking module. Utilizing the principles of multi-view geometry, even if an individual is completely occluded from one or more viewpoints, as long as they are visible from other viewpoints, their identity and location can be determined by the intersection of three-dimensional spatial points. This fundamentally solves the problem of tracking loss caused by occlusion, elevates the continuity of individual tracking to a practical level, and lays an invaluable data foundation for subsequent accurate behavioral analysis.

[0050] 2. Building upon stable tracking, this invention further introduces a 3D pose reconstruction module to recover the 3D skeletal key points of the pig from images from multiple perspectives. This not only provides a richer pose description than 2D bounding boxes, making the quantification of behaviors such as lying down, standing, and lameness more accurate, but also makes the 3D information itself more robust to occlusion. The combination of 3D pose and 3D motion trajectory constitutes a comprehensive, high-precision spatiotemporal feature system describing individual behavior, enhancing the depth and reliability of behavioral analysis.

[0051] 3. The anomaly detection module of this invention adopts an unsupervised or semi-supervised learning paradigm based on a deep autoencoder, combined with a population statistical comparison method. The system does not need to predefine all possible abnormal behavior patterns; instead, it learns the data distribution of normal behavior through the model and automatically detects abnormal patterns that deviate from this distribution. This method can discover unknown and subtle early abnormal signs. Simultaneously, by reconstructing the error threshold and statistical deviation threshold through a dual-threshold mechanism, it reduces the false alarm rate caused by fluctuations in a single indicator, thereby improving the specificity and practicality of the early warning system.

[0052] 4. This invention constructs a complete technical closed loop from data acquisition, fusion tracking, 3D reconstruction to intelligent analysis and feedback optimization. The system distributes basic detection tasks through edge computing, ensuring real-time performance; performs complex calculations through cloud servers, guaranteeing in-depth analysis; and drives model iteration through human feedback, achieving the system's self-evolution capability. This system-level design makes this invention not only a monitoring tool but also a continuously learning and improving intelligent hub for pig farm management, capable of stably serving the core needs of healthy pig farming and refined management over the long term. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the overall technical architecture of the pig farm anomaly monitoring system based on image recognition proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the cross-view fusion tracking module in this invention; Figure 3 This is a logical flowchart of the three-dimensional pose reconstruction and behavior index extraction in this invention; Figure 4 This is a flowchart of the online detection and feedback learning process of the anomaly determination module in this invention; Figure 5 This is a schematic diagram of the data flow between the multi-view image acquisition array and the edge-cloud collaboration in this invention. Detailed Implementation

[0054] This invention provides a pig farm anomaly monitoring system based on image recognition. Please refer to the appendix. Figures 1 to 5 This system constructs a complete technological closed loop from physical perception to intelligent decision-making. Its core lies in overcoming the fundamental obstacle to continuous behavior monitoring caused by severe occlusion between individual pigs in high-density farming environments through multi-view collaboration and 3D modeling technology.

[0055] The system consists of four physical and logical components: a multi-view image acquisition array, edge computing nodes, a cloud analysis server, and a management terminal. The components interact with each other through wired or wireless networks to form a hierarchical and collaborative organic whole.

[0056] The multi-view image acquisition array is the sensing front end of the system deployed in the pigsty environment.

[0057] Please refer to the attached document. Figure 5 The array consists of multiple high-definition network cameras installed according to a preset three-dimensional geometric rule.

[0058] The array layout follows strict coverage and overlap principles to ensure the feasibility of 3D reconstruction and tracking.

[0059] Specifically, the array must completely cover all areas in the pigsty that need to be monitored, including the pigs' activity area, rest area, and feeding and drinking area, to achieve monitoring without blind spots.

[0060] At the spatial coverage level, the system requires that any potential location within the pigsty where pigs may be present must have its three-dimensional coordinates fall within the effective field of view of at least two different cameras simultaneously.

[0061] This requirement is a fundamental prerequisite for the subsequent cross-perspective geometric intersection algorithm to be executed.

[0062] In terms of field of view connectivity, the fields of view of adjacent cameras need to have some overlap. This helps to maintain at least dual-view coverage at the boundaries of the camera's field of view and avoid blind spots in monitoring.

[0063] In terms of installation parameters, the camera's installation height and tilt angle need to be specially calculated and adjusted. The core goal is to ensure that the camera can stably capture the pig's back and side profiles, rather than just looking down from above.

[0064] This is because the back and side profiles of pigs contain rich pose information, which is crucial for subsequent 3D keypoint detection and pose reconstruction.

[0065] Typical installation heights range from 2.5 meters to 4 meters above the ground, and pitch angles range from 15 degrees to 45 degrees downward. Specific values ​​need to be adjusted on-site according to the span of the pigsty and the height of the pens.

[0066] All cameras are connected to the pigsty's local area network via a network switch and receive time synchronization from a unified time synchronization protocol to ensure that all video streams are strictly synchronized in timestamps, with a synchronization accuracy requirement of better than 50 milliseconds.

[0067] The camera continuously captures color or grayscale video streams of the pigsty at a preset fixed frame rate, such as 15 frames per second or 25 frames per second, with a video resolution greater than 1920×1080 pixels.

[0068] Edge computing nodes are dedicated computing devices deployed on-site in pig farms or adjacent computer rooms, and they are directly connected to multi-view image acquisition arrays.

[0069] Please continue to refer to the appendix. Figure 5 The core responsibility of edge computing nodes is to perform real-time preprocessing and target detection in the first stage of the raw video stream to reduce network transmission bandwidth pressure and provide structured initial data for complex analysis in the cloud.

[0070] Each edge computing node is typically responsible for processing video streams from one or a group of logically associated cameras.

[0071] The node runs a lightweight pig target detection model, which is trained on an improved single-stage target detection architecture. The model structure has been deeply optimized for the morphological features of pigs as a specific target. For example, the distribution of receptive fields in each layer of the feature pyramid network has been adjusted to adapt to changes in pigs at different scales.

[0072] The model is loaded into the memory of the node's graphics processor or dedicated neural network accelerator, and receives video images from a designated camera in frames.

[0073] For each input image frame, the model performs forward inference computation and outputs two-dimensional rectangular bounding boxes for all images identified as pigs in that frame. The output information for each bounding box includes four parameters: the horizontal and vertical coordinates of the top-left corner of the bounding box in the image coordinate system, the width and height of the bounding box, and a confidence score representing the reliability of the detection result, which ranges from 0 to 1.

[0074] After obtaining the original detection results, the edge computing nodes immediately execute the nonmaximum suppression post-processing algorithm.

[0075] The algorithm iterates through all detection boxes and calculates the cross-union ratio (CUP) between any two boxes. If the CUP of two boxes is greater than a preset threshold of 0.5, and the confidence of one box is significantly lower than that of the other, the algorithm will suppress or delete redundant detection boxes with lower confidence.

[0076] The filtered set of valid detection boxes, along with the timestamp corresponding to the frame image with millisecond precision and the unique number of the source camera that generated the detection box, are encapsulated into a standard format data packet.

[0077] The data packets are encoded using serialization and uploaded in real time to the designated data receiving port of the cloud analysis server via a transmission control protocol.

[0078] Edge computing nodes maintain communication with cloud servers through a heartbeat mechanism to monitor their status and have local caching and breakpoint resume capabilities when the network is temporarily interrupted.

[0079] The cloud analytics server is the core computing and decision-making hub of the system, deployed on a cloud platform or high-performance server cluster with powerful graphics processing units and central processing units.

[0080] Please refer to the attached document. Figure 1 The cloud analytics server logically comprises a series of closely cooperating functional modules, mainly including a cross-view fusion tracking module, a 3D pose reconstruction module, a temporal behavior modeling module, and an anomaly detection module.

[0081] These modules exchange data with shared memory through internal message queues, forming a highly efficient data processing pipeline.

[0082] The cross-view fusion tracking module is a key technical component for solving the occlusion problem.

[0083] Please refer to the attached document. Figure 2 The core task of this module is to integrate the scattered and potentially severely occluded 2D detection data uploaded from all edge computing nodes, and through algorithm fusion, generate a continuous, stable, and uniquely identified movement trajectory for each pig in the 3D world coordinate system.

[0084] The module maintains a global collection of individual pigs in memory. Each individual is a dynamic data structure whose fields must include at least: A globally unique identifier that is generated when an individual is first identified by the system and remains unchanged throughout their lifespan; A time-series buffer used to store the historical three-dimensional location coordinates of this individual; A cache area used to store the depth feature vector of the individual's appearance; And a motion state estimator based on a Kalman filter, which internally maintains estimates of individual state variables such as position and velocity, as well as their covariance matrix.

[0085] For each newly arrived, time-aligned set of instantaneous data packets containing detection frames uploaded by all cameras at the same moment, the module initiates a multi-stage fusion tracking process. The first stage is the intersection of points in three-dimensional space.

[0086] The module calls a pre-calibrated multi-view geometric model. This model stores the internal parameter matrix, external parameter matrix, rotation matrix, and translation vector of each camera in the system. These parameters are obtained by performing stereo calibration on the pigsty environment beforehand.

[0087] The module iterates through all 2D bounding boxes at the current moment, attempting to pair bounding boxes that are spatially close and come from different cameras.

[0088] For a set of candidate matching boxes, the module calculates the equation of a straight line in three-dimensional space for each back-projection ray passing through the center of the detection box from the camera's optical center, using the image coordinates and camera parameters. Theoretically, back-projection rays from different perspectives belonging to the same real pig individual should converge at a single point in three-dimensional space.

[0089] The module calculates the optimal intersection coordinates of these rays using the least squares method and calculates the intersection residuals.

[0090] If the intersection point is located within a reasonable height range defined by the physical space of the pigsty, such as between 0 and 1 meter from the ground, and the intersection residual is less than a preset threshold, such as 0.1 meters, then it is determined that the group of detection boxes belongs to the same individual, and the individual is immediately assigned or confirmed as its identity identifier. At the same time, the three-dimensional coordinates of the intersection point are updated to the individual's current position in its trajectory.

[0091] Observations of individuals that are successfully linked through geometric intersection are considered high-confidence observations.

[0092] The second stage deals with detection boxes that cannot be confirmed by geometric intersection, such as a pig that is completely occluded by other pigs in most views and is only visible in one view.

[0093] At this point, the module initiates an association algorithm based on appearance features and motion consistency.

[0094] For each isolated two-dimensional detection box, the module first extracts a high-dimensional depth feature vector from its corresponding original image region. This vector is generated by a pre-trained convolutional neural network feature extractor and can characterize the pig's appearance information such as texture, color, and contour.

[0095] Meanwhile, the module filters out candidate individuals from the global set of individuals that have been updated in the most recent time, such as within the past 2 seconds, and whose predicted position is adjacent to the back-projection ray of the isolated detection box in three-dimensional space.

[0096] For each candidate, the algorithm performs two calculations. First, it calculates the cosine similarity between the feature vector of the current isolated detection box and the latest feature vector in the historical feature cache of that individual. The similarity value is between -1 and 1, with the closer to 1 indicating a greater similarity in appearance.

[0097] On the other hand, the Kalman filter of the candidate individual is invoked to predict its three-dimensional position at the current moment, and the shortest spatial distance from the predicted position to the back-projection ray of the current isolated detection box is calculated.

[0098] Subsequently, the algorithm normalizes the appearance similarity score and spatial distance score, and then performs weighted fusion according to preset weights, such as appearance weight 0.6 and motion weight 0.4, to obtain a comprehensive association score.

[0099] If the score is greater than the dynamically adjusted association threshold, the isolated detection box is associated with the candidate individual, and the Kalman filter update step of its 3D position estimate is updated using the observation from this single perspective and combined with the individual's historical motion model.

[0100] If the association scores of all candidate individuals are less than the threshold, the system initializes this isolated detection box as a new pig individual, assigns it a new globally unique identifier, and uses the preliminary three-dimensional estimate from this perspective as the starting point of its trajectory.

[0101] By employing a strategy that prioritizes multi-perspective geometric intersections and supplements them with feature motion correlations, the system can maximize the continuity of each individual's identity in the temporal dimension and output the most likely three-dimensional spatial coordinate sequence of each individual at each moment.

[0102] The 3D pose reconstruction module receives an identified individual data stream from the cross-view fusion tracking module.

[0103] Please refer to the attached document. Figure 3 The goal of this module is to recover the three-dimensional skeletal structure of pigs from two-dimensional images, providing a more refined pose description for behavior quantization than bounding boxes.

[0104] For each individual output by the cross-view fusion tracking module and its 3D position at a certain moment, the pose reconstruction module first needs to acquire the original multi-view image data of that individual.

[0105] Based on the individual's 3D position, the module performs a reverse query to determine which cameras' fields of view cover the location, and selects at least two of these viewpoints that have the lowest degree of occlusion on the individual at that moment.

[0106] The degree of occlusion is comprehensively evaluated by factors such as the texture complexity of the image region within the individual detection box, edge sharpness, and whether there are overlapping detection boxes.

[0107] The module extracts an image patch centered on the individual detection box and appropriately expanded from the corresponding original video frame, which is used as the input for 3D pose reconstruction.

[0108] The module loads a pre-trained 3D pig keypoint detection neural network model.

[0109] The model adopts a multi-view fusion architecture. Its input is image patches selected from multiple views. After feature extraction by their respective convolutional neural network branches, the features are fused in the intermediate layer. Finally, the output layer directly regresses the coordinates of 17 predefined anatomical key points on the pig's torso in the three-dimensional world coordinate system.

[0110] These 17 key points include the tip of the nose, left eye, right eye, left ear root, right ear root, neck, left shoulder, right shoulder, thoracic vertebrae, left hip, right hip, tail root, left foreleg, right foreleg, left hind leg, and right hind leg.

[0111] The model was trained using a large dataset of labeled ground truth 3D keypoints, which were obtained through motion capture systems or precise 3D scanning.

[0112] During the online inference phase, the module inputs the prepared multi-view image patches into the model. After forward propagation, the model outputs a 17×3 matrix, which contains the X, Y, and Z coordinates of 17 key points.

[0113] Subsequently, the module constructs a simplified three-dimensional line segment model, i.e. a three-dimensional skeletal model, based on the physiological connections between these key points, such as the nose tip connecting to the neck and the neck connecting to the shoulder.

[0114] By analyzing the angles, length ratios of each segment, and height of key points relative to the ground in this skeletal model, the specific posture of an individual can be precisely quantified.

[0115] For example, by calculating the angle between the line connecting the thoracic vertebrae to the midpoint of the buttocks and the horizontal plane, standing, leaning, and lying down can be distinguished; by analyzing the height and movement trajectory of key points of the limbs, gaits such as walking, jogging, and limping can be identified.

[0116] The temporal behavior modeling module is responsible for transforming discrete position and attitude observations into continuous, analyzable time series of behavioral features.

[0117] Please continue to refer to the appendix. Figure 3 The input to this module is the three-dimensional position coordinates and three-dimensional pose keypoint coordinates of each individual over a continuous time series. The sampling frequency is consistent with the video frame rate or the output frequency of the fusion tracking, for example, 15 samples per second.

[0118] The module internally calculates a series of basic behavioral indicators in real time, which characterize an individual's instantaneous state and actions from different dimensions. The first type of indicator is kinematic indicators.

[0119] Based on the individual's position sequence in three-dimensional space, the module obtains the instantaneous velocity vector by calculating the difference between the position vectors of adjacent time points. The magnitude of the instantaneous velocity is the instantaneous speed, and the unit is meters per second. Further difference of the velocity vector yields the instantaneous acceleration.

[0120] Simultaneously, the module calculates the total length of an individual's movement path and net displacement within a certain time window, such as the past 30 seconds, to assess its activity level. The second type of indicator is the posture stability indicator.

[0121] The module extracts key points representing the core of the torso, such as the midpoint of the shoulder and the midpoint of the hip, and calculates the vertical variance of these points.

[0122] The lower the variance value, the more stable the individual's posture, and the more likely it is to be in a state of stillness or uniform motion. The higher the variance, the more drastic the individual's posture changes, possibly indicating actions such as getting up, lying down, or struggling.

[0123] The third category of indicators is those related to food and water intake.

[0124] The system predefines the spatial range of the feed trough area and the water dispenser in a three-dimensional world coordinate system as a three-dimensional cubic region.

[0125] The module calculates in real time the three-dimensional coordinates of key points on the tip of an individual's nose and the shortest distance between these preset areas and the surface.

[0126] When the distance is less than a set threshold, such as 0.3 meters, the individual is determined to be in a state of near feeding or drinking, and the duration of this state is accumulated.

[0127] The fourth category of indicators is social behavior indicators. The module calculates the three-dimensional Euclidean distance between any two individuals in real time.

[0128] When the distance between two individuals is less than a preset social distance threshold, such as 0.5 meters, a contact event is determined to have occurred.

[0129] The module tracks each individual's contact frequency, average contact duration, and number of contacts within a time window to quantify their social activity and potential for conflict.

[0130] All of the above-mentioned basic behavioral indicators are sampled and output at a fixed frequency, such as once per second, to form a multi-dimensional behavioral feature vector. The feature vector of each individual is arranged in chronological order to form its unique multi-dimensional time series, providing a data foundation for subsequent anomaly analysis.

[0131] The anomaly detection module is the system's decision-making terminal, responsible for automatically identifying abnormal individuals that deviate from the normal pattern from massive behavioral time-series data.

[0132] Please refer to the attached document. Figure 4 This module employs a hybrid detection paradigm that combines unsupervised learning with population statistical comparison. Its workflow includes an offline model training phase and an online real-time detection phase.

[0133] During the offline training phase, the module needs to build a standard model of normal behavior.

[0134] The system collects time-series data of multidimensional behavioral indicators of all individual pigs in a pig farm under known healthy and stable conditions over a long period of time, such as two consecutive months.

[0135] These data constitute a sample set of normal behavior.

[0136] Considering that pig behavior patterns change significantly with growth stages, the system divides the data according to growth stages, such as the nursery period (30-70 days old), the early fattening period (70-120 days old), and the late fattening period (120 days old until slaughter).

[0137] For each growth stage, the module trains a deep autoencoder model using the corresponding normal behavior sample set.

[0138] This autoencoder is a special type of neural network, consisting of an encoder and a decoder.

[0139] The encoder receives a high-dimensional input vector, such as a long vector composed of all behavioral indicators from 60 time points over the past 60 seconds. It compresses this vector into a low-dimensional latent space vector through multiple nonlinear transformations. This vector captures the most prominent pattern features in the input data.

[0140] The decoder then attempts to reconstruct the original high-dimensional input vector from this low-dimensional latent space vector.

[0141] The training process uses the backpropagation algorithm to minimize the error between the reconstructed output and the original input. The commonly used error function is the mean squared error.

[0142] The goal of training is to enable the autoencoder to learn how to efficiently compress and reconstruct normal behavioral data.

[0143] Once training is complete, the autoencoder exhibits a small reconstruction error for normal data, but for anomalous data, since the patterns do not appear in the training set, the autoencoder struggles to accurately reconstruct them, resulting in a large reconstruction error.

[0144] During the online operation phase, the anomaly detection module continuously analyzes the real-time generated behavioral data stream.

[0145] For each individual to be evaluated, the module first slides a fixed-length time window of data from its latest behavioral time series data, such as data from the past 60 seconds, and organizes it into the same feature vector format as during training.

[0146] The feature vector is then input into a deep autoencoder corresponding to the current growth stage of the individual.

[0147] The autoencoder performs a forward propagation, where the encoder encodes the vector into a latent vector, and the decoder then decodes it into a reconstructed vector.

[0148] The module calculates the reconstruction error between the original input vector and the reconstructed vector, specifically using the mean squared error formula:

[0149] At the same time, the module performs group statistical comparisons.

[0150] The module obtains the same set of behavioral index values ​​for all other pigs in the same pig house and at the same growth stage at the current moment, calculates the mean vector and covariance matrix of these indicators in multidimensional space, and thus describes the normal behavioral distribution of the current pig herd.

[0151] Subsequently, the statistical deviation of the current behavioral feature vector of the individual to be evaluated relative to the distribution of this pig population is calculated, using Mahalanobis distance.

[0152] Mahalanobis distance takes into account the correlation between features and can more accurately measure the degree to which sample points deviate from the population center.

[0153] The larger the Mahalanobis distance value, the more the individual's behavior deviates from the group's norm.

[0154] The module sets two independent dynamic thresholds: the reconstruction error threshold and the statistical deviation threshold.

[0155] These two thresholds are not fixed values, but are set using statistical methods, such as taking the mean plus three times the standard deviation, based on the distribution of historical normal data on the corresponding indicators, and can be fine-tuned based on feedback from management.

[0156] Exception handling follows an AND logic: The module only determines that an individual's behavior is abnormal when the system detects that the reconstruction error value of an individual is greater than its corresponding dynamic threshold for 60 consecutive seconds, and its statistical deviation Mahalanobis distance is also greater than its corresponding dynamic threshold within the same time period.

[0157] This dual-threshold mechanism greatly reduces false alarms caused by transient noise, accidental behavior, or sensitivity to a single indicator.

[0158] Once an anomaly is detected, the module will perform preliminary classification and labeling of the anomaly type based on which specific behavioral indicators continue to exceed the limits.

[0159] For example, if the abnormally high values ​​are extremely low movement speed and extremely high postural stability, it may indicate that the individual is in a state of drowsiness or illness. If the frequency and intensity of social contact are consistently excessive, it may indicate that the individual is involved in conflict or has been bullied.

[0160] The judgment result, along with information such as confidence level, anomaly type, associated individual identity, time and location, is packaged into an early warning message.

[0161] The management terminal is the interface through which the system interacts with aquaculture managers, and it is usually provided in the form of a web application or a mobile application.

[0162] The terminal receives and visualizes various information from the cloud analysis server in real time, including real-time video footage of the pigsty, real-time location and trajectory heatmaps of the pigs, key behavioral indicator curves of individuals, and most importantly, a list of abnormal warnings.

[0163] When an abnormal warning is received, the terminal will provide a prompt through sound and light, and automatically associate and retrieve at least 5 minutes of original video footage from multiple perspectives before and after the abnormal event, the individual's complete 3D motion trajectory and posture reconstruction animation, and all relevant behavioral indicator charts for managers to review and diagnose.

[0164] Administrators operate on the terminal to confirm system alerts as "true positives", correct "errors in anomaly type labeling", or reject "false alarms".

[0165] The annotation information generated during this manual review process is automatically collected by the system and incorporated into the feedback dataset.

[0166] The system is designed with a closed-loop feedback learning and model optimization mechanism. Please refer to the appendix. Figure 4 All data that has been manually reviewed and labeled, whether confirmed abnormal samples or corrected normal samples, will be used as valuable incremental training data.

[0167] The system periodically, such as weekly or monthly, initiates model retraining tasks.

[0168] This task utilizes the existing training dataset plus the newly added feedback dataset to incrementally train or fine-tune the key algorithm models in the system.

[0169] These models include a lightweight pig target detection model on edge computing nodes, a 3D pig keypoint detection model in the cloud, and an anomaly detection autoencoder model for each growth stage.

[0170] By continuously injecting new, manually verified data, the system's target detection accuracy, posture estimation accuracy, and anomaly identification specificity and sensitivity are continuously improved, thus forming a self-evolving system that becomes smarter with use and adapts to changes in pig herd structure and new management needs in the long term.

[0171] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0172] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A pig farm anomaly monitoring system based on image recognition, characterized in that, include: A multi-view image acquisition array deployed inside the pigsty is used to synchronously acquire real-time video streams of the pigsty from different angles; The edge computing node connected to the multi-view image acquisition array is used to perform image preprocessing and target detection on the real-time video stream, and generate a data packet containing two-dimensional bounding box coordinates, timestamps and camera numbers; A cloud-based analytics server that communicates with the edge computing node is used to receive the data packets and perform cross-view fusion tracking, 3D pose reconstruction, and abnormal behavior analysis. The management terminal connected to the cloud-based analysis server is used to receive and display analysis results and early warning information.

2. The pig farm anomaly monitoring system based on image recognition according to claim 1, characterized in that, The cloud analytics server includes: The cross-view fusion tracking module is used to maintain a global set of individual pigs and to perform time alignment on the set of 2D detection boxes synchronously uploaded from all cameras at each moment; Call the pre-calibrated multi-view geometric model to back-project two-dimensional detection boxes from different viewpoints and with similar spatial positions at the same time onto a three-dimensional space in a preset world coordinate system; Multiple 2D detection frames that can successfully intersect at a reasonable 3D spatial point through back projection are determined to belong to the same individual pig. For detection boxes that cannot be confirmed through multi-view intersection, an association algorithm based on appearance features and motion consistency is initiated to extract the depth features of isolated detection boxes and calculate the cosine similarity with the individual features of the most recent moment in the historical trajectory. At the same time, the individual motion state is predicted based on Kalman filtering, and the feature similarity and the distance of the motion prediction position are weighted and fused to determine the identity. By employing a strategy that prioritizes multi-view geometric intersection and supplements it with feature motion correlation, we assign three-dimensional spatial coordinates to each pig at every moment and update its identity trajectory. A 3D pose reconstruction module is used to receive individual data with confirmed identity and 3D position output by the cross-view fusion tracking module; For each individual, select at least two original image regions from the multiple visible viewpoints that have the least occlusion. Load a pre-trained 3D pig key point detection model, take the multi-view image patch as input, output the coordinates of predefined anatomical key points on the pig's torso in 3D space, and fit a simplified 3D skeleton model based on the key points to quantify the pose. The temporal behavior modeling module is used to receive the three-dimensional position coordinates and three-dimensional attitude key point coordinates of each individual over a continuous time series. The study calculates basic behavioral indicators such as instantaneous motion velocity and acceleration based on three-dimensional position difference calculation, posture stability index based on the height variance of trunk key points, feeding proximity based on the positional relationship between the nose tip key point and the pre-set area of ​​the feed trough, and social contact frequency and intensity based on the three-dimensional distance between individuals. These indicators are then sampled at a fixed frequency to form a multi-dimensional time series. The anomaly detection module includes an offline training phase and an online execution phase. During the offline training phase, time series data of multidimensional behavioral indicators of all individuals in the pig farm under normal and healthy conditions over a long period of time are collected. Deep autoencoder models are trained for pigs at different growth stages. The encoder of the autoencoder compresses high-dimensional temporal behavioral features into a low-dimensional latent space, while the decoder reconstructs the input features from the latent space. The training objective is to minimize the reconstruction error of the model for normal behavioral patterns. During the online operation phase, for each individual's behavioral indicator sequence generated in real time, it is first input into the corresponding growth stage autoencoder to calculate the reconstruction error, and at the same time, the statistical deviation of the individual's current behavioral indicator from the overall behavioral indicator distribution of the pig herd during the same period is calculated; the reconstruction error threshold and the statistical deviation threshold are set. When an individual's reconstruction error exceeds its threshold for 60 consecutive seconds, and its statistical deviation also exceeds its threshold, the individual's behavior is judged to be abnormal.

3. The pig farm anomaly monitoring system based on image recognition according to claim 2, characterized in that, The target detection process performed by the edge computing node is as follows: A lightweight pig target detection model is loaded, which is trained based on an improved YOLO architecture and optimized for pig morphological features; Receive video frames from a single camera and output the 2D bounding boxes of all pigs in each frame and their confidence scores. Non-maximum suppression processing is applied to the detection results to filter out redundant boxes with excessive overlap; the processed bounding box coordinates, timestamps, and corresponding camera numbers are encapsulated into a data packet and uploaded to the cloud analysis server in real time.

4. The pig farm anomaly monitoring system based on image recognition according to claim 3, characterized in that, The specific process of the association algorithm based on appearance features and motion consistency is as follows: Extract the depth feature vector of isolated 2D detection boxes; Candidate individuals are selected from the global set of individuals that have been updated in the most recent time and whose predicted positions are adjacent to the back-projection ray of the detection box in three-dimensional space; For each candidate individual, calculate the cosine similarity between the current feature vector and the latest historical feature vector of that individual, and call the Kalman filter of that individual to predict its three-dimensional position at the current time, and calculate the shortest spatial distance from the predicted position to the back projection ray of the current detection box. The appearance similarity score and spatial distance score are normalized and then weighted and fused according to preset weights to obtain a comprehensive association score; If the overall association score is greater than the dynamically adjusted association threshold, then the detection box is associated with the candidate individual.

5. A pig farm anomaly monitoring system based on image recognition according to claim 4, characterized in that, The statistical deviation is calculated using Mahalanobis distance, and the calculation process for Mahalanobis distance is as follows: Obtain the same set of behavioral index values ​​for all other pigs at the same growth stage in the same pig house at the current moment, and calculate the mean vector and covariance matrix of these indicators in multidimensional space. Calculate the statistical deviation of the current behavioral feature vector of the individual to be evaluated relative to the distribution of the pig herd.

6. The pig farm anomaly monitoring system based on image recognition according to claim 5, characterized in that, The system also includes a feedback learning and model optimization mechanism; The specific process of the mechanism is as follows: All events judged as abnormal by the system, along with at least 5 minutes of multi-view video clips before and after them and all analysis data, are pushed to the management terminal for manual review and annotation. The labeled data is automatically imported into the training dataset, which is used to periodically perform incremental training on the target detection model in the edge computing node, the 3D key point model in the 3D pose reconstruction module, and the autoencoder model in the anomaly detection module.

7. The pig farm anomaly monitoring system based on image recognition according to claim 6, characterized in that, The stereoscopic layout of the multi-view image acquisition array follows the following principles: The array covers all areas of the pigsty that need to be monitored. Any point in the pigsty is covered by the field of view of multiple different cameras; The fields of view of adjacent cameras overlap; the installation height and angle of the cameras ensure that the back and side profiles of the pigs can be captured.

8. The image recognition-based pig farm anomaly monitoring system according to claim 7, characterized in that, The reconstruction error threshold and statistical deviation threshold are set as the mean plus three times the standard deviation based on the distribution of historical normal data on the corresponding indicators using statistical methods, and are fine-tuned based on feedback from management personnel.

9. A pig farm anomaly monitoring system based on image recognition according to claim 8, characterized in that, After determining that the behavior is abnormal, the anomaly detection module performs a preliminary classification of the anomaly type based on the specific behavioral indicators that continuously exceed the standard.

10. A method for monitoring anomalies in pig farms based on image recognition, characterized in that, The image recognition-based pig farm anomaly monitoring system described in any one of claims 1 to 9 is used to implement pig farm anomaly monitoring.

Images