A lactation sow and piglet behavior integrated detection system based on a patrol robot

Abstract

A kind of lactation sow and piglet behavior integrated detection system based on inspection robot, including inspection robot hardware system, sow and piglet behavior detection system;Inspection robot hardware system: utilize track type inspection robot to collect sow and piglet behavior image and video, storage, analysis and transmission are carried out;Sow and piglet behavior detection system: analyze sow static image and piglet dynamic short video, on the core computing unit of inspection robot, through YOLOv8, the posture of sow is detected, through lightweight TSM algorithm, piglet group behavior is detected, and the result is transported to database storage.Compared with traditional manual inspection, the present application based on inspection robot and computer vision avoids the intervention of artificial to lactation sow and piglet, maximally reduces the risk of zoonosis, improves inspection efficiency, reduces the cost of artificial input in farm.At the same time, the platform information management system is convenient to form effective production and breeding experience, with the characteristics of automation and intelligentization.

Description

Technical Field

[0001] This patent relates to the technical fields of computer vision, edge computing, and embedded systems. Specifically, it is an integrated detection system for the behavior of lactating sows and piglets based on an inspection robot, YOLOv8, and a lightweight TSM. Background Technology

[0002] As a vital component of agriculture, animal husbandry occupies a crucial position in the national economy. With the increasing demand for meat products and the continuous expansion of livestock and poultry farming, traditional farming methods can no longer meet market needs, necessitating a transformation towards modernization, automation, and intelligentization in animal husbandry. Pig farming, as an important branch of animal husbandry, is also facing new development opportunities and challenges.

[0003] The management of lactating sows and piglets is crucial for pig farming efficiency and risk mitigation. By monitoring the behavior of lactating pigs, farmers can directly understand their growth status, ensure the health and welfare of sows and piglets, and improve farm profitability. Furthermore, the lack of a platform-based information management system prevents inspectors from uniformly storing, managing, and analyzing individual pig behavior information, hindering the formation of effective production and farming experience and the development of smart farming models.

[0004] Early research focused on sow behavior detection using wearable sensors. However, this invasive method was prone to causing stress and harming animal welfare. Furthermore, due to the harsh environment of pig farms and the complexity of animal behavior, wearable devices often carried risks such as data distortion and device detachment. The emergence of machine vision technology made efficient, contactless pig behavior detection possible. Researchers developed pig behavior detection systems based on image processing and deep learning by setting up fixed cameras. However, in production applications, as pig farms expand, installing fixed cameras is not only costly but also susceptible to dust pollution in pig houses, leading to maintenance difficulties. Simultaneously, the transmission, storage, and analysis of massive amounts of video data increased the network load between cloud servers and data sources, consuming significant server storage and computing resources, and placing higher performance demands on cloud computing centers. Summary of the Invention

[0005] This invention addresses the problems existing in the background technology by proposing an integrated detection system for the behavior of lactating sows and piglets based on an inspection robot. Utilizing an edge computing chip mounted on the robot, a lightweight algorithm is developed to complete behavior detection near the data source, improving data processing and transmission efficiency. Based on Web technology, a software platform for detecting the behavior of lactating sows and piglets is developed, enabling real-time control of the inspection robot and visualization of behavioral data. Specifically, the YOLOv8 object detection algorithm is used to detect four static postures of sows in the gestation crate: standing, sitting, lying on their side, and chest recumbent. The TSM video understanding algorithm is used to extract features from the dynamic behavior of piglets in the gestation crate, recognizing three behaviors: resting, nursing, and standing. Considering both detection speed and accuracy, the algorithm model is lightweighted and deployed to the Jetson nano, the core computing unit of the inspection robot, using the TensorRT inference engine.

[0006] Technical solution:

[0007] An integrated detection system for the behavior of lactating sows and piglets based on an inspection robot includes: an inspection robot hardware system, a sow and piglet behavior detection system, an inspection robot dynamic control system, and a sow and piglet behavior detection web platform; specifically:

[0008] Inspection robot hardware system: Uses a track-mounted inspection robot to collect images and videos of sows and piglets' behavior, and then stores, analyzes, and transmits them;

[0009] Sow and piglet behavior detection system: Analyzes static images of sows and dynamic short videos of piglets. On the core computing unit of the inspection robot, the system uses YOLOv8 and lightweight TSM algorithms to detect the sow's posture and the piglet's group behavior, and sends the results to the database for storage.

[0010] The inspection robot dynamic control system performs Kappa consistency analysis based on the statistical results of sow and piglet behavior. When inconsistencies are found in the behavior of individual sows or groups of piglets in different limit stalls, the inspection strategy of the inspection robot is adjusted in a timely manner. The inspection frequency is increased for individualized sows and groups of piglets, and long-term fixed-point detection is carried out. Pig house staff are notified in a timely manner to facilitate the health management of lactating sows and piglets.

[0011] Sow and piglet behavior monitoring web platform: This involves database, middleware, and backend design, as well as the development of a front-end visual interface. The platform's main functions include user information management, robot inspection control, environmental data monitoring, and visualization of sow and piglet behavior data. It aims to provide a convenient operating terminal to assist farmers in automating and contactlessly managing lactating sows and piglets, thereby improving the digitalization level of pig farms.

[0012] Preferably, the inspection robot hardware system includes: the inspection robot body, a charging compartment, and a track. The track is laid in the middle of the upper part of the pigsty, and the charging compartment is set at the starting point of the track. The inspection robot uses a pulley system to closely follow the track for inspection. During the inspection task, it uses the core computing unit Jetson nano to perform real-time model inference and uploads the processing results to the web page front-end. When the inspection robot's battery level falls below a threshold, it will automatically return to the charging compartment for recharging.

[0013] Preferably, the inspection robot body includes: a core control module, a data processing module, a power supply module, a hardware peripheral module, a drive and transmission module, and a communication module. Specifically:

[0014] Core Control Module: A self-made robot PCB main control board is used, with an STM32F103C8T6 as the main control chip. It mainly includes 12-5V and 5-3.3V voltage regulator circuits, a serial communication interface, a sensor interface, four relays, and a motor interface. The voltage regulator circuit supplies power to the STM32, data processing module, and hardware peripheral modules. The serial communication interface connects the core control module and the data processing module. By controlling the armature engagement state of the four relays, the direction and magnitude of the motor input voltage are controlled, thereby controlling the robot's forward, backward, and stop movements.

[0015] Data processing module: The Jetson nano is used as the edge development board, equipped with a quad-core CPU and a 128-core GPU, as well as 4GB of memory, to perform sow and piglet behavior recognition tasks.

[0016] Power module: Uses a 12V rechargeable lithium battery to power the core control module. The charging module is located on top of the robot and charges by contacting the positive and negative terminals of the charging compartment.

[0017] Hardware peripherals: A USB camera is used to capture video in real time for data processing and uploading to the Jetson nano. Non-contact photoelectric sensors are used as sensing devices to ensure the robot's functions such as fixed-point monitoring, inspection return, and charging compartment location detection. Specifically, three photoelectric sensors are installed on the robot, located on both sides and in the forward direction. The photoelectric sensor in the forward direction is used to detect the L-shaped corner marker at the end of the track, while the left and right photoelectric sensors are used to detect the corner marker above the limit bar and the charging compartment position, respectively.

[0018] Drive transmission module: Includes a motor and pulley system. The motor is a DC geared motor. The pulley system consists of a drive wheel and auxiliary wheels. The drive wheel is powered by a gear set driven by the DC motor, and the auxiliary wheels are two rolling pulleys used to embed into the inspection track. During robot operation, with the help of the auxiliary wheels, the robot closely follows the inspection track, while the rolling friction between the drive wheel and the track enables the robot to move.

[0019] Communication module: A USB driverless network card is used as the wireless network card, which is inserted into the Jetson nano's card slot for network communication between the robot and front-end and back-end devices. The STM32 microcontroller inside the robot communicates with the Jetson nano via serial port.

[0020] Specifically, the model construction of the sow and piglet behavior detection system includes the following steps:

[0021] S1. Data Acquisition: Video data is acquired based on the inspection robot hardware system. The robot briefly stops above each limit bar and saves the video for 5 seconds to obtain dynamic behavior data of the piglet group. OpenCV is used to extract frames from the video data to obtain static images of the sow's posture.

[0022] S2. Definition of concepts: Definitions of four postures of sows: standing, sitting, lying on their side, and chest-lying; and definitions of three group behaviors of piglets: suckling, active, and resting.

[0023] S3. Data partitioning: Based on the concept definition, the sow posture dataset and the piglet group behavior dataset are partitioned into the sow posture dataset and the piglet group behavior dataset respectively according to the ratios of 8:1:1 and 6:2:2.

[0024] S4. Network Design: To balance detection speed and detection accuracy, YOLOv8 and TSM are used as the detection network models.

[0025] S5 and YOLOv8 model training: The YOLOv8 model was trained using a sow pose classification image dataset as a detector for sow pose under the restraint bar.

[0026] S6. TSM Model Training and Lightweighting: The TSM model was trained using a piglet behavior classification video dataset as a detector of piglet group behavior under the gestation bar, and MobileNetv2 was used to lightweight the network.

[0027] S7. Model Quantization and Deployment: Using TensorRT as the inference engine, the YOLOv8 and TSM trained models are converted into TensorRT models (.engine) using FP16 quantization and deployed to the Jetson nano, the core computing unit of the inspection robot.

[0028] S8. Behavior Statistics and Visualization: Transmit the results of various sow posture detection and piglet behavior recognition to the backend database, write SQL statements to automatically count the number of times various postures and behaviors occur each day, and publish them to the front-end visualization page.

[0029] Preferably, the sow's posture includes: standing, sitting, lying on its side, and chest recumbent; specifically defined as:

[0030] Standing: The sow's hooves touch the ground, her body remains upright, back up, belly down, and her belly does not touch the ground;

[0031] Sitting position: The sow's forelimbs are in near-vertical contact with the ground, while her hind limbs and tail are in complete contact with the ground. Her head is supported by her forelimbs and is significantly higher than her tail.

[0032] Side-lying position: The sow's forelegs and hind legs are folded under her body, her back is facing up, her abdomen is on the ground, and her udder is completely invisible;

[0033] Chest recumbent: The sow has her limbs outstretched, with one side of her body fully in contact with the ground, exposing her lactation area;

[0034] Preferably, the group behavior of piglets includes nursing, activity, and rest; considering that group behaviors of piglets may occur simultaneously, this invention sets the priority of piglet behaviors, and the specific behaviors are defined as follows:

[0035] Nursing: High priority, with more than one-third of the piglets in the group lying down or standing and suckling from the sow's udder;

[0036] Activity: In the priority category, more than one-third of the piglets in the group are in an upright position and moving around in the gestation stalls;

[0037] Rest: Low priority. Apart from the two behaviors mentioned above, the piglets are quiet and mostly lie on their sides.

[0038] Specifically, in the network design described in step S4, the loss function of the YOLOv8 detection network model uses Distributed Focus Loss (DFL Loss) and Complete Intersection Loss (CIOU Loss) for regression tasks, and Binary Cross-Entropy Loss (BCE Loss) for classification tasks. The number of model training iterations is determined based on the convergence of the loss function, and the specific calculation formula is as follows:

[0039]

[0040] In the formula, Indicates the label position. and Represent Rounding down to the left and right sides; and They represent and The probability of a point location appearing.

[0041]

[0042] Let be the intersection-union ratio (IoU) of the ground truth bounding boxes and the predicted bounding boxes. These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. This indicates the calculation of the Euclidean distance between two centers. It is a penalty factor used to fit the aspect ratio of the predicted bounding box to the aspect ratio of the ground truth bounding box.

[0043]

[0044] For the number of categories in the object detection task, in this study It is 5. The true value for the current category; This is the probability value that the model predicts the current category.

[0045] Preferably, in the YOLOv8 model training described in step S5, a warmup training method and momentum method are used for optimization. The warmup_epochs are set to 3, the warmup_momentum to 0.8, the initial momentum to 0.937, the batch_size to 16, the initial learning rate to 0.01, and the total number of training iterations to 300. Specifically, pre-trained weights from the COCO dataset are used as initial weights to improve the model's learning ability. A mixed-precision training strategy is adopted to reduce cache usage and improve training speed. Simultaneously, mosaic image enhancement is disabled in the last 10 epochs.

[0046] Specifically, in the training of the TSM algorithm described in step S6, 2D-CNN is used on the sampled frames to extract spatial features of the images. At the same time, the TSM module is used to fuse information from adjacent frames along the time dimension by moving the time channel, which enhances the network's feature extraction capability while effectively reducing the amount of computation.

[0047] Specifically, in the model quantization and deployment described in step S7, on the computationally limited Jetson nano platform, TSM_ResNet50, due to its huge number of parameters and computational load, struggles to process the massive amounts of data in large-scale pig farms in real-world applications. This invention lightweights TSM, replacing ResNet50 with MobileNetv2, effectively reducing the model's parameter count and computational load. Further, after FP16 precision quantization and TensorRT acceleration, the model's detection accuracy decreases slightly, but the detection speed increases by more than twenty times.

[0048] Specifically, the inspection robot dynamic control system, combined with data processed by the sow and piglet behavior detection system, performs Kappa consistency analysis, which includes the following steps:

[0049] S1. Inspection strategy for inspection robots:

[0050] During daily inspections, the inspection robot uses the OpenCV library to read data from a USB camera, and uses the core data processing module Jetson nano to infer the sow's posture and the piglet's group behavior in real time. The inference results are then transmitted to the backend database and the frontend web page.

[0051] S2. Inspection Result Analysis:

[0052] After the inspection robot finishes its inspection, Kappa consistency analysis is performed on the statistical data of sow posture and piglet group behavior in the gestation stalls for that day to check whether there are significant inconsistencies in the behavior of sows or piglets in each gestation stall.

[0053] S3. Adjust the inspection strategy of the inspection robot, increase the inspection frequency for sows and piglets with significant differences in behavior, conduct long-term fixed-point testing, and promptly notify pig farm staff to facilitate the health management of lactating sows and piglets.

[0054] Preferably, it also includes a web platform for detecting the behavior of sows and piglets, with front-end and back-end interaction to achieve real-time data updates and visualization.

[0055] Beneficial effects of the present invention

[0056] (1) Compared with traditional manual inspection, the present invention, based on inspection robots and computer vision, avoids human intervention in lactating sows and piglets, minimizes the risk of zoonotic diseases, improves inspection efficiency, and reduces labor input costs in farms. At the same time, the platform-based information management system facilitates the formation of effective production and breeding experience and has the characteristics of automation and intelligence.

[0057] (2) Compared with sensor-based pig behavior monitoring methods, the present invention has the significant feature of being non-contact. The data collection, transmission and processing process will not affect the daily life of lactating sows and piglets, thus maximizing animal welfare.

[0058] (3) Compared with the pig detection method based on fixed cameras, the present invention has the characteristics of efficient data processing and simple maintenance. It is equipped with an edge computing processing chip to replace the cloud computing mode, relieve the pressure on the cloud server, improve data processing efficiency, and ensure the security of data transmission.

[0059] (4) In view of the characteristics of piglet groups where multiple behaviors may occur simultaneously and the behavior expression is dynamic, this invention proposes a definition standard and classification method for piglet group behavior, and uses the setting of piglet group behavior priority as the classification standard when multiple behaviors occur at the same time. At the same time, a piglet group behavior recognition method based on video understanding network is proposed, which effectively captures the temporal relationship in piglet video sequences and improves recognition accuracy.

[0060] (5) The TSM_MobileNetv2 model proposed in this invention has significant advantages in terms of lightweight and efficiency. On the Jetson nano platform with limited computing power, the TSM_MobileNetv2 model, after FP16 precision quantization and TensorRT acceleration, has a detection speed that is more than 20 times faster.

[0061] (6) It realizes the automated real-time detection of the behavior of lactating sows and piglets in the gestation crate, analyzes the number of times various behaviors of sows and piglets occur during the inspection process in real time, dynamically adjusts the inspection strategy, and accurately locates individual sows and piglet groups with large differences in behavior. Attached Figure Description

[0062] Figure 1 This is a block diagram of the sow and piglet behavior detection system of the present invention.

[0063] Figure 2 This is a physical image of the inspection robot of the present invention.

[0064] Figure 3 This is a field test diagram of the inspection robot of the present invention.

[0065] Figure 4 This is a schematic diagram of sow posture detection in the example.

[0066] Figure 5 This is a schematic diagram of piglet group behavior detection in the example.

[0067] Figure 6 This is a flowchart of the YOLOv8 model training process in the embodiment.

[0068] Figure 7 This is a training effect diagram of the sow posture detector in the embodiment.

[0069] Figure 8 This is a flowchart of the TSM_MobileNetv2 model training process in the example.

[0070] Figure 9 This is a block diagram of TensorRT quantization and deployment in the embodiment.

[0071] Figure 10 This is a schematic diagram of the web-based inspection robot control page of the present invention.

[0072] Figure 11 This is a schematic diagram of the web-based data detection and inference result display page of the present invention. Detailed Implementation

[0073] The present invention will be further described below with reference to embodiments, but the scope of protection of the present invention is not limited thereto:

[0074] Combined with appendix Figure 1 The sow and piglet behavior detection system includes an inspection robot, tracks, a charging compartment, a cloud server, a client, and a host computer for training and implementing detection method models. The inspection robot, relying on lightweight tracks, monitors sows and piglets living in the pigsty in real time. The monitoring video data is continuously input into the robot's core data processing module, enabling real-time inference of the model. The system analyzes the sow and piglet behavior in the video, establishes a connection with the client, stores the detection results, and visualizes the data for different categories of behavior.

[0075] The following describes the complete steps of building the model using a specific example:

[0076] S1. Data Acquisition: Figure 2 and Figure 3 The paper presents both a physical image and a field test image of the inspection robot, which monitors sows and piglets living in the pigsty and obtains video and image data.

[0077] S2. Concept Definition: Combining Figure 4 The study defined four postures for sows: standing, sitting, chest-recumbent, and side-recumbent. Unlike sow behavior, piglet behavior is characterized by its group-oriented and dynamic nature, making it difficult to accurately infer individual piglet behavior from static images. Combined with... Figure 5 This paper proposes a definition standard and classification method for piglet group behavior, defining suckling, activity and rest behaviors. It also proposes the idea of ​​replacing the whole with multiple key indicators, using the sow's posture and the piglets' group behavior as important indicators to evaluate the health status of the sow and piglets. This can not only ensure that the key features learned are output, but also make the features more vivid and easier for subsequent analysis.

[0078] S3. Data partitioning: All data is partitioned into training, validation, and test sets to ensure model performance while preventing overfitting.

[0079] S4. Network Design: To balance detection speed and detection accuracy, YOLOv8 and TSM algorithms are used as the detection network model;

[0080] S5 and YOLOv8 model training: Combining Figure 6 A YOLOv8 model was trained using a dataset of sow pose images to serve as a sow pose detector. The training results are as follows: Figure 7 As shown. From Figure 7As can be seen, the model achieves a detection accuracy of over 94% for piglet targets and four sow postures: standing, sitting, chest-recumbent, and side-recumbent. It was optimized using a warmup training method and momentum optimization, with warmup_epochs set to 3, warmup_momentum to 0.8, initial momentum to 0.937, batch_size to 16, and initial learning rate to 0.01, for a total of 300 training iterations. Specifically, pre-trained weights from the COCO dataset were used as initial weights to improve the model's learning ability. A mixed-precision training strategy was employed to reduce cache usage and improve training speed, while mosaic image enhancement was disabled in the last 10 epochs.

[0081] Compared to traditional machine learning methods, deep learning-based convolutional neural networks can achieve self-driving through end-to-end training when sufficient data is available. The network model itself can discover target features without the need for manual feature design, thereby acquiring richer semantic information. This method can significantly improve recognition accuracy and speed.

[0082] S6 and TSM model training: combined with Figure 8 We trained a TSM video understanding network model using a piglet behavior video dataset. A 2D-CNN was applied to the sampled frames to extract spatial features. Simultaneously, the TSM module was used to fuse information from adjacent frames along the temporal dimension by shifting time channels. This approach avoids impacting the feature extraction capabilities of the 2D convolutional network and, by adding a temporal max-pooling layer, effectively reduces computational cost.

[0083] This paper proposes the use of the TSM algorithm to identify piglet group behavior. By using TSM to identify piglet behavior, it effectively avoids the difficulties in tracking individual piglets and identifying their behavior in actual pig farming environments where the number of suckling piglets is large. By identifying key behavioral changes in the pig herd through the group identification of individual piglets, the accuracy of piglet behavior identification is improved, which is beneficial for judging the growth status and health status of piglets.

[0084] S7. Model Quantization and Deployment: Specifically, for the sow posture and piglet target detection model, the optimal algorithm YOLOv8 was selected, combined with... Figure 9First, the model trained in the PyTorch framework (.pth structure) is exported and converted to a .wts structure. The next step is to choose an appropriate batch size and numerical precision; this paper sets the batch size to 1 and uses FP16 precision. Then, the model is converted from the .wts structure to a TensorRT engine, generating an .engine file. The application typically builds the engine once, serializing it to disk as a plan file for later use. To initialize the engine, the application first deserializes the plan file into memory, and finally performs inference by providing data to the engine.

[0085] S8. Behavior Statistics and Visualization: Transmit the results of various sow posture detection and piglet behavior recognition to the backend database, write SQL statements to automatically count the duration and frequency of various postures and behaviors each day, and publish them to the front-end visualization page.

[0086] Compared to traditional methods of manually identifying and statistically analyzing sow and piglet behavior, computer vision technology reduces errors in the detection process and is highly efficient. This paper proposes a lightweight sow pose detection model and a piglet group behavior recognition model, compared to other computer vision recognition methods. These models are deployed on the Jetsonnano edge computing device of an inspection robot, enabling high-accuracy and high-speed inference at near-data points, reducing the load on cloud servers.

[0087] This invention enables automated real-time detection of the behavior of lactating sows and piglets in pigsties. By comparing the duration and frequency of various characteristic behaviors of sows and piglets at different time periods, it is easy to observe the living habits of sows and piglets.

[0088] In other embodiments, the inspection robot dynamic control system combines data processed by the sow and piglet behavior detection system to perform Kappa consistency analysis. When anomalies occur, the inspection robot's inspection strategy is adjusted promptly and reported to the pig farm personnel. Specifically, this includes the following steps:

[0089] S1. Inspection strategy for inspection robots:

[0090] During daily inspections, the inspection robot uses the OpenCV library to read data from a USB camera. Through the core data processing module Jetson Nano, it infers the sow's posture and the piglet group's behavior in real time, and transmits the inference results to the backend database and the frontend web page. Figure 11 As shown.

[0091] S2. Inspection Result Analysis:

[0092] After the inspection robot finishes its inspection, Kappa consistency analysis is performed on the statistical data of sow posture and piglet group behavior in the gestation stalls for that day to check whether there are significant inconsistencies in the behavior of sows or piglets in each gestation stall.

[0093] S3. Adjust the inspection strategy of the inspection robot, increase the inspection frequency for sows and piglets with significant differences in behavior, conduct long-term fixed-point testing, and promptly notify pig farm staff to facilitate the health management of lactating sows and piglets.

[0094] Preferably, it also includes a web platform for detecting sow and piglet behavior, such as Figure 10 As shown, this page allows for remote control of the inspection robot. The detection web platform utilizes front-end and back-end interaction technology to achieve real-time data updates and visualization.

[0095] The method provided by this invention obtains various behavioral data and other information of lactating sows and piglets in a non-contact manner, so as to improve the welfare level of lactating sows and piglets through artificial intervention.

[0096] This application integrates computer vision, edge computing, and web development technologies to research an integrated detection system and method for the behavior of lactating sows and piglets. It develops an intelligent pig farm inspection robot and designs a platform for detecting the behavior of lactating sows and piglets based on this robot. The inspection robot performs inspection tasks on lactating sows and piglets along a preset track from a top-down perspective, effectively solving the problems of high difficulty and low efficiency associated with manual inspection. Edge computing, with its high bandwidth and low latency, allows for the integration of edge computing processing chips into the inspection robot, replacing cloud computing and alleviating the pressure on cloud servers, improving data processing efficiency, and ensuring the security of data transmission.

[0097] This system can improve the welfare of pig farming, assist farmers in managing the behavior information of sows and piglets in the farrowing house, lay the foundation for sow culling and piglet health assessment, and at the same time, it is expected to reduce production and breeding costs and promote the continuous improvement of the digitalization level of large-scale pig farming.

[0098] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. An integrated detection system for the behavior of lactating sows and piglets based on an inspection robot, characterized in that... It includes an inspection robot hardware system, a sow / piglet behavior detection system, an inspection robot dynamic control system, and a sow / piglet behavior detection web platform; specifically: Inspection robot hardware system: Uses a track-mounted inspection robot to collect images and videos of sows and piglets' behavior, and then stores, analyzes, and transmits them; Sow and piglet behavior detection system: This system analyzes static images of sows and short dynamic videos of piglets. On the core computing unit of the inspection robot, it uses YOLOv8 to detect sow posture and a lightweight TSM algorithm to detect piglet group behavior, then stores the results in a database. The model construction of the sow and piglet behavior detection system includes the following steps: S1. Data Acquisition: Video data is acquired based on the inspection robot hardware system. The robot briefly stops above each limit bar and saves the video for 5 seconds to obtain dynamic behavior data of the piglet group. OpenCV is used to extract frames from the video data to obtain static images of the sow's posture. S2. Definition of concepts: Definitions of four postures of sows: standing, sitting, lying on their side, and chest-lying; and definitions of three group behaviors of piglets: suckling, active, and resting. S3. Data partitioning: Based on the concept definition, the sow posture dataset and the piglet group behavior dataset are partitioned into the sow posture dataset and the piglet group behavior dataset respectively according to the ratios of 8:1:1 and 6:2:

2. S4. Network Design: To balance detection speed and detection accuracy, YOLOv8 and TSM are used as the detection network models. S5 and YOLOv8 model training: The YOLOv8 model was trained using a sow pose classification image dataset as a detector for sow pose under the restraint bar. S6. TSM Model Training and Lightweighting: The TSM model was trained using a piglet behavior classification video dataset as a detector of piglet group behavior under the gestation bar, and MobileNetv2 was used to lightweight the network. S7. Model Quantization and Deployment: Using TensorRT as the inference engine and employing FP16 quantization, the models trained with YOLOv8 and TSM are converted into TensorRT models and deployed to the Jetson nano, the core computing unit of the inspection robot. S8. Behavior Statistics and Visualization: Transmit the results of various sow posture detection and piglet behavior recognition to the backend database, write SQL statements to automatically count the number of times various postures and behaviors occur each day, and publish them to the front-end visualization page; The inspection robot dynamic control system performs Kappa consistency analysis based on the statistical results of sow and piglet behavior. When inconsistencies are found in the behavior of individual sows or groups of piglets in different limit stalls, the inspection strategy of the inspection robot is adjusted in a timely manner. The inspection frequency is increased for individualized sows and groups of piglets, and long-term fixed-point detection is carried out. Pig house staff are notified in a timely manner to facilitate the health management of lactating sows and piglets.

2. The system according to claim 1, characterized in that... The system also includes a web platform for detecting sow and piglet behavior, used for database, middleware and backend design, and development of front-end visual pages; The platform's functions include user information management, robot inspection control, environmental data monitoring, and visualization of sow and piglet behavior data. It aims to provide a convenient operating terminal to assist farmers in the automated and contactless management of lactating sows and piglets, thereby improving the digitalization level of pig farms.

3. The system according to claim 1, characterized in that... The inspection robot hardware system includes: the inspection robot body, the charging compartment, and the track; the track is laid in the middle of the upper part of the pigsty, the charging compartment is set at the starting point of the track, the inspection robot uses pulleys to closely follow the track to carry out inspections, and at the same time uses the core computing unit Jetson nano to perform real-time model inference during the inspection task and uploads the processing results to the web front end; when the inspection robot's battery is lower than the threshold, it will automatically return to the charging compartment to recharge; The inspection robot body includes: a core control module, a data processing module, a power supply module, a hardware peripheral module, a drive and transmission module, and a communication module, wherein: The core control module uses an STM32F103C8T6 as the main control chip and includes 12-5V and 5-3.3V voltage regulator circuits, a serial communication interface, a sensor interface, four relays, and a motor interface. The voltage regulator circuit supplies power to the STM32, data processing module, and hardware peripheral modules. The serial communication interface connects the core control module and the data processing module. By controlling the armature engagement state of the four relays, the direction and magnitude of the motor input voltage are controlled, thereby controlling the robot's forward, backward, and stop motion. Data processing module: The Jetson nano is used as the edge development board, equipped with a quad-core CPU and a 128-core GPU, as well as 4GB of memory, to perform sow and piglet behavior recognition tasks. Power module: Uses a 12V rechargeable lithium battery to supply power to the core control module; the charging module is located on top of the robot and achieves charging by contacting the positive and negative terminals of the charging compartment. Hardware peripheral modules: A USB camera is used to capture video in real time for Jetson nano to process and upload data; a non-contact photoelectric sensor is used as the sensing device to ensure the inspection robot's functions of fixed-point monitoring, inspection return, and charging compartment location detection; Drive transmission module: includes a motor and a pulley block. The motor is a DC geared motor. The pulley block consists of a drive wheel and an auxiliary wheel. The drive wheel is powered by a gear set driven by the DC motor. The auxiliary wheel consists of two rolling pulleys used to embed into the inspection track. During the operation of the robot, with the help of the auxiliary wheel, the robot closely follows the inspection track. At the same time, the rolling friction between the drive wheel and the track enables the robot to move. Communication module: A USB driverless network card is used as a wireless network card, which is inserted into the card slot of the Jetson nano for network communication between the robot and front-end and back-end devices. The STM32 inside the robot communicates with the Jetson nano via serial port.

4. The system according to claim 3, characterized in that... In the hardware peripheral module, the robot is equipped with three photoelectric sensors, located on both sides of the robot and in the forward direction. The photoelectric sensor in the forward direction is used to detect the L-shaped corner mark at the end of the track. The left and right photoelectric sensors are used to detect the corner mark above the limit bar and the position of the charging compartment, respectively.

5. The system according to claim 1, characterized in that... In the definition of the concept described in S2: Sow postures include: standing, sitting, lying on one's side, and chest recumbent; specific posture definitions are as follows: Standing: The sow's hooves touch the ground, her body remains upright, back up, belly down, and her belly does not touch the ground; Sitting position: The sow's forelimbs are in near-vertical contact with the ground, while her hind limbs and tail are in complete contact with the ground. Her head is supported by her forelimbs and is significantly higher than her tail. Side-lying position: The sow's forelegs and hind legs are folded under her body, her back is facing up, her abdomen is on the ground, and her udder is completely invisible; Chest recumbent: The sow has her limbs outstretched, with one side of her body fully in contact with the ground, exposing her lactation area; Piglet group behavior includes: nursing, activity, and rest; and piglet behavior priorities are set, with specific behavior definitions as follows: Nursing: High priority, with more than one-third of the piglets in the group lying down or standing and suckling from the sow's udder; Activity: In the priority category, more than one-third of the piglets in the group are in an upright position and moving around in the gestation stalls; Rest: Low priority. Apart from the two behaviors mentioned above, the piglets are quiet and mostly lie on their sides.

6. The system according to claim 1, characterized in that... In the network design described in step S4, the loss function of the YOLOv8 detection network model adopts Distributed Focus Loss (DFL Loss) and Complete Intersection Loss (CIOU Loss) for regression tasks, and Binary Cross-Entropy Loss (BCE Loss) for classification tasks. The number of model training iterations is determined based on the convergence of the loss function, and the specific calculation formula is as follows: In the formula, Indicates the label position. and Represent Rounding down to the left and right sides; and They represent and The probability of a point location appearing; Let be the intersection-union ratio (IoU) of the ground truth bounding boxes and the predicted bounding boxes. These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. This indicates the calculation of the Euclidean distance between two centers. It is a penalty factor used to fit the aspect ratio of the predicted bounding box to the aspect ratio of the ground truth bounding box; For the number of categories in the object detection task, The true value for the current category; This is the probability value that the model predicts the current category.

7. The system according to claim 1, characterized in that: In the YOLOv8 model training described in step S5, a warmup training method and momentum method are used for optimization. The warmup_epochs are set to 3, the warmup_momentum to 0.8, the initial momentum to 0.937, the batch_size to 16, the initial learning rate to 0.01, and the total number of training iterations to 300. During training, pre-trained weights on the COCO dataset are used as initial weights to improve the model's learning ability. A mixed precision training strategy is adopted to reduce cache usage and improve training speed. At the same time, mosaic image enhancement is turned off in the last 10 epochs. In the training of the TSM algorithm described in step S6, 2D-CNN is applied to the sampled frames to extract spatial features of the images. At the same time, the TSM module is used to fuse information from adjacent frames along the time dimension by moving the time channel, which enhances the network's feature extraction capability while effectively reducing the amount of computation.

8. The system according to claim 1, characterized in that... In the model quantization and deployment described in step S7, TSM is lightweighted, ResNet50 is replaced with MobileNetv2, and it undergoes FP16 precision quantization and TensorRT acceleration.

9. The system according to claim 2, characterized in that... The inspection robot dynamic control system, combined with data processed by the sow and piglet behavior detection system, performs Kappa consistency analysis, specifically including the following steps: S1. Inspection strategy for inspection robots: During daily inspections, the inspection robot uses the OpenCV library to read the USB camera, and uses the core data processing module Jetson nano to infer the sow's posture and the piglet's group behavior in real time. The inference results are then transmitted to the backend database and the frontend web page. S2. Inspection Result Analysis: After the inspection robot finishes its inspection, Kappa consistency analysis is performed on the statistical data of sow posture and piglet group behavior in the gestation stalls for that day to check whether there are significant inconsistencies in the behavior of sows or piglets in each gestation stall. S3. Adjust the inspection strategy of the inspection robot, increase the inspection frequency for sows and piglets with significant differences in behavior, conduct long-term fixed-point testing, and promptly notify pig farm staff to facilitate the health management of lactating sows and piglets.

Images