Livestock breeding management method and system applying reinforcement learning

Abstract

The application discloses a kind of livestock breeding management method and system applying reinforcement learning, it is related to the technical field of animal husbandry, the system is composed of several function modules, including: multi-source perception fusion module, obtains multi-source data by sensor, and the fusion point cloud generated by time-space alignment and point-by-point fusion to multi-source data;Instance segmentation and motion feature guided multi-level association tracking are executed on the fusion point cloud, and multi-dimensional individual features are extracted based on multi-level association tracking;According to the fusion point cloud, the semantic map of pasture gridding is synchronously constructed;Group driving decision module, with the multi-agent of the multiple mobile robots deployed, the multi-dimensional individual features and the semantic map are constructed into state space, the animal welfare risk index is designed based on the state space, and the reward function with navigation safety as weighted item, through multi-agent deep deterministic policy gradient algorithm, obtain the group driving strategy for driving livestock to the group area.

Description

Technical Field

[0001] This invention relates to the field of animal husbandry technology, specifically to a livestock breeding management method and system that applies reinforcement learning. Background Technology

[0002] As livestock farming develops towards large-scale and intelligent operations, the daily management of sheep flocks, especially herding and relocation, has become a key factor affecting breeding efficiency and animal welfare. Traditional sheep herding mainly relies on manual labor and sheepdogs, while relocation depends heavily on the herders identifying and manually separating each sheep into pens. This operational model faces prominent problems such as high labor intensity, low efficiency, and low standardization. Against the backdrop of increasing labor shortages, the use of drones, mobile robots, and artificial intelligence technologies to achieve unmanned sheep flock management has become an important research direction in the field of smart animal husbandry.

[0003] In existing technologies, sheepdog herding methods require extremely high levels of dog training and herder experience, resulting in poor operational consistency, long training cycles, and high costs. Furthermore, the herding effect is difficult to guarantee in complex terrain or large-scale sheep flock scenarios. To reduce reliance on human intervention and trained animals, some solutions introduce drones for sheep herding, utilizing drones to achieve simple waypoint flight and cruise functions, guiding the flock along preset paths. However, such solutions lack the ability to perceive the dynamic behavior of the flock in real time and to adjust adaptive strategies. When the flock disperses, becomes frightened, or exhibits abnormal behavior, the preset flight paths and simple approach and drive-away strategies often fail, making it impossible to achieve stable and reliable autonomous herding. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: A livestock farming management system that applies reinforcement learning includes: The multi-source perception fusion module acquires multi-source data through sensors and performs spatiotemporal alignment and point-by-point fusion to generate a fused point cloud. It then performs instance segmentation and motion feature-guided multi-level correlation tracking on the fused point cloud, extracts multi-dimensional individual features based on the multi-level correlation tracking, and simultaneously constructs a pasture-grid semantic map based on the fused point cloud. The group driving decision module uses multiple deployed mobile robots as multiple agents, constructs a state space by combining multi-dimensional individual features and semantic maps, designs an animal welfare risk index based on the state space, and a reward function with navigation safety as a weighted term. Through a multi-agent deep deterministic policy gradient algorithm, it obtains a group driving strategy for driving livestock to the grouping area, and outputs speed commands for each robot based on the group driving strategy. The individual grouping decision module collects the grouping channel occupancy status and waiting queue length after livestock enter the grouping waiting area. It integrates the body condition information estimated from visual recognition and the historical health records obtained from radio frequency identification to construct an individual profile for each livestock. Using the grouping channel occupancy status, the current individual profile, and the waiting queue length as the state, and the allocation to different grouping channels as the action, the module obtains the individual grouping strategy through dual deep Q-network optimization. In the collaborative execution and safety management module, each robot tracks its trajectory in real time through distributed model predictive control, based on speed commands and individual grouping strategies, while satisfying kinematic constraints and safety distance constraints between robots and between robots and livestock.

[0005] Furthermore, the process of extracting multidimensional individual features is as follows: For each tracked animal, the position is predicted using a Kalman filter and strongly correlated with the current detection. For strongly correlated but unmatched items, weak correlation matching is performed by calculating Mahalanobis distance based on the movement features of the tracked animals, thus maintaining the continuity of the tracked animals' identities.

[0006] Furthermore, the process of synchronously constructing a rasterized semantic map of the pasture based on the fused point cloud is as follows: Geometric feature point matching is used to estimate the pose of LiDAR frames and the resulting data is stitched together to generate a 3D point cloud map. The voxels in the 3D point cloud map are semantically labeled, with semantic labels including at least ground, fence, water source, and obstacle. The labeled 3D point cloud map is vertically projected into a 2D grid. The semantic label that appears most frequently in each grid is used as the dominant semantic label for that grid. The passability is calculated based on the proportion of ground points. Grids with a ground point proportion higher than a preset threshold are labeled as passable areas, forming a gridded semantic map.

[0007] Furthermore, the process of constructing a state space from multidimensional individual features and semantic maps is as follows: The robot's own state is defined by its relative distance and angle to the target area, as well as its current speed. The group state is defined by the distance and angle between the geometric center of the livestock herd and the robot, as well as the density distribution of the livestock herd in multiple sectors around the robot. The environmental perception state is defined by the distance to obstacles detected from the rasterized semantic map in multiple circumferential directions. The welfare representation state is defined by the animal welfare risk index calculated based on the degree of deviation of the body surface temperature from the baseline and the proportion of high-speed moving individuals in the total number of livestock. The historical values ​​of the welfare risk index at multiple consecutive decision moments are defined as the welfare risk trend state. The robot's own state, group state, environmental perception state, welfare representation state, and welfare risk trend state are spliced ​​together to form a complete state space.

[0008] Furthermore, animal welfare risks include: Animal welfare risk items are quantified based on the welfare risk index, which is a weighted average of the deviation of each animal’s body surface temperature from the dynamic baseline temperature of the individual, and the proportion of individuals with high-speed movement exceeding the preset warning threshold in the total population. The navigation safety items include: the navigation safety items construct graded penalties based on the circumferential obstacle distance vector, apply incremental penalties to the number and degree of directions entering the danger zone and warning zone, and superimpose collision penalties from the safety distance constraints between robots and the safety distance constraints between robots and livestock.

[0009] Furthermore, the process of outputting speed commands for each robot is as follows: Each robot acts as an independent intelligent agent, internally maintaining an actor network and a critic network, employing a centralized training and decentralized execution paradigm. During the training phase, it outputs continuous actions based on local observation states and superimposes exploration noise. After interacting with the environment, it stores joint observations, joint actions, and shared rewards in an experience replay buffer. It uses global joint states and joint actions to perform centralized value evaluation, and updates policy parameters with the obtained value gradient direction. The target network slowly approximates the online network parameters through soft updates. During the execution phase, it outputs a normalized velocity adjustment amount based on forward propagation of real-time local observation states. After inverse normalization, this is restored to a physical velocity command and superimposed on the current velocity. After smoothing and limiting, the command is sent to the robot's underlying motion controller.

[0010] Furthermore, the process of creating an individual file for each animal is as follows: Based on the fusion of point cloud and instance segmentation mask, a 3D point set of the livestock back is extracted. After filtering, body size parameters are extracted, and the weight is estimated by a pre-trained weight regression network. The body condition score is evaluated by a classification network based on the curvature features of the back point cloud and the body size ratio. At the same time, the identification code is read by RFID ear tag, and historical health data including feed intake, activity level, medical history and breeding records are retrieved from the breeding management database. The real-time body condition data estimated by vision is timestamped and weighted and fused with the historical records to generate a comprehensive health score. The tissue is labeled as an individual profile vector, and the data is synchronously written back to the database after the grouping is completed.

[0011] Furthermore, the process of obtaining the individual grouping strategy through dual deep Q-network optimization is as follows: The state is constructed by combining the occupancy status of the grouping channel, the individual files of the livestock to be grouped, and the length of the waiting queue. The discrete action is to be assigned to different grouping channels. A dual-path structure of online network and target network is constructed. The online network selects the action and the target network evaluates the value. The reward function is a weighted sum of grouping accuracy, queue smoothness, and channel load balancing. After training converges, only the online network is used to output the channel allocation action during the deployment phase.

[0012] Furthermore, the process of real-time trajectory tracking through distributed model predictive control is as follows: Each robot takes its current state, received speed commands, and individual grouping strategy as inputs, and constructs a finite-time domain optimization problem within the prediction time domain. The optimization objectives are speed tracking error and control increment smoothness, and the constraints are physical limit constraints, obstacle accessibility constraints based on semantic maps, inter-robot safety distance constraints based on the sharing of predicted trajectories of other robots, and safety distance constraints between robots and livestock. The optimization problem is solved online in a rolling manner, with only the first step of the control variable executed in each control cycle. During execution, the threshold of the animal and livestock safety distance constraints is applied based on the animal welfare risk index.

[0013] A livestock farming management method applying reinforcement learning includes the following steps: Step 1: Acquire multi-source data and perform spatiotemporal alignment and point-by-point fusion to generate a fused point cloud; perform instance segmentation and motion feature-guided multi-level correlation tracking on the fused point cloud, and extract multi-dimensional individual features based on the multi-level correlation tracking; simultaneously construct a pasture-rasterized semantic map based on the fused point cloud. Step 2: Using multiple deployed robots as multiple agents, construct a state space by combining multi-dimensional individual features and semantic maps. Based on the state space, design an animal welfare risk index and a reward function with navigation safety as a weighted term. Through the multi-agent deep deterministic policy gradient algorithm, obtain a group driving strategy for driving livestock to the grouping area. Based on the group driving strategy, output the speed command of each robot. Step 3: After the livestock enter the grouping waiting area, the occupancy status of the grouping channel and the length of the waiting queue are collected. The body condition information estimated from visual recognition and the historical health records obtained from radio frequency identification are fused to construct an individual profile for each animal. Using the occupancy status of the grouping channel, the current individual profile and the length of the waiting queue as the state, and the allocation to different grouping channels as the action, the individual grouping strategy is obtained through dual deep Q network optimization. Step 4: Based on speed commands and individual grouping strategies, each robot performs real-time trajectory tracking through distributed model predictive control, while satisfying kinematic constraints and safe distance constraints between robots and between robots and livestock.

[0014] The present invention provides a livestock breeding management method and system that applies reinforcement learning, which has the following beneficial effects: (1) This invention performs spatiotemporal alignment and point-by-point fusion of visible light, thermal infrared and lidar data through a multi-source perception fusion module, and performs multi-level correlation tracking guided by motion features. In the group driving decision module, a state space and reward function containing animal welfare risk index are constructed, so that the driving strategy can learn gentle driving behavior autonomously while pursuing efficiency. This solves the problem that existing livestock robot driving technology ignores animal stress response and is prone to livestock welfare loss.

[0015] (2) This invention uses a two-layer reinforcement learning architecture of group driving and individual grouping to decouple and serially connect group-level multi-agent collaborative driving and individual-level fine grouping. The driving strategy is obtained by multi-agent deep deterministic strategy gradient algorithm and the grouping strategy is obtained by dual deep Q network. This realizes the unmanned closed-loop management of the entire process from any location in the pasture to the specific target pen. It solves the problem that the driving and grouping links in the existing technology are separated and cannot take into account both group efficiency and individual accuracy.

[0016] (3) The present invention supports the state construction of the group driving decision module by the fused point cloud and semantic map output by the multi-source perception fusion module. The speed command output by the group driving decision module and the individual grouping strategy output by the individual grouping decision module jointly drive the distributed model prediction control of the collaborative execution and safety management module. At the same time, the animal welfare risk index is continuously monitored and fed back to the anomaly handling mechanism. This closed-loop collaborative mechanism makes the multi-source perception, two-layer decision and safety execution interdependent and mutually optimized, realizing the overall improvement of livestock driving and grouping operation efficiency, animal welfare and operational safety in complex pasture environments. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the system of the present invention; Figure 2 This is a schematic diagram of the overall method of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: Please see Figure 1 Embodiment 1 of this application provides a livestock breeding management system that applies reinforcement learning. The system includes: The multi-source perception fusion module acquires multi-source data through sensors and performs spatiotemporal alignment and point-by-point fusion to generate a fused point cloud. It then performs instance segmentation and motion feature-guided multi-level correlation tracking on the fused point cloud, extracts multi-dimensional individual features based on the multi-level correlation tracking, and simultaneously constructs a pasture-grid semantic map based on the fused point cloud. The process of generating a fused point cloud is as follows: The module simultaneously connects to three types of sensors: the first type is a visible light camera, which acquires color images of the pasture at a frequency of about 30 frames per second. Each pixel in the image contains values ​​for the red, green, and blue color channels, which can clearly record the coat color, texture, and outline of the livestock; the second type is a thermal infrared camera, which acquires thermal radiation images at a frequency of about 9 frames per second. The value of each pixel in the image reflects the surface temperature of the object at that location, which can non-contactly sense subtle changes in the surface temperature of the livestock; the third type is a lidar, which emits laser beams into the surrounding environment at a frequency of about 10 frames per second and receives reflected signals to generate point cloud data containing a large number of three-dimensional spatial points. Each point is determined by three coordinate values ​​to determine its spatial position. The overall point cloud depicts the precise geometric shape of objects such as the pasture ground, fences, and individual livestock.

[0020] These three types of sensors have different acquisition frequencies and use different coordinate systems. In their original state, the data are misaligned in both time and space dimensions, and must be aligned before they can be fused and used.

[0021] The system selects the timestamp of the lidar as a unified time reference; each point cloud frame of the lidar carries a precise timestamp, which is recorded as the reference time of that frame; at this reference time, the system searches for the frame with the closest timestamp in the frame sequence of the visible light camera, and also searches for the frame with the closest timestamp in the frame sequence of the thermal infrared camera; if the time difference between the found visible light frame and the reference time exceeds fifty milliseconds, the visible light frame is discarded; the same applies to the thermal infrared frame.

[0022] The three types of sensors each have their own independent coordinate systems: the visible light camera coordinate system, the thermal infrared camera coordinate system, and the lidar coordinate system. The system uses the lidar coordinate system as the world coordinate system and obtains the spatial transformation relationship between the sensors in advance through a joint calibration method. Specifically, a checkerboard calibration board is placed within the field of view of all sensors simultaneously, and multiple sets of synchronous data at different positions and attitudes are collected. By identifying the corresponding positions of the checkerboard corner points in the image and point cloud, the rotation and translation relationship from the visible light camera to the lidar, and the rotation and translation relationship from the thermal infrared camera to the lidar are calculated respectively. After obtaining these relationships, for any three-dimensional spatial point in the lidar point cloud, the system can calculate its corresponding pixel coordinates in the visible light image and the corresponding pixel coordinates in the thermal infrared image, thereby achieving accurate point-to-pixel correspondence with pixel-level precision.

[0023] After spatial alignment, the system proceeds to the point-to-point projection stage. For each 3D point in the lidar point cloud, the system performs the following operations: First, using the extrinsic and intrinsic parameters of the visible light camera, the 3D coordinates of the point are projected onto the visible light image plane to obtain its corresponding pixel position. If the projected point falls within the image area, the red, green, and blue color values ​​are read from that pixel position. Second, using the extrinsic and intrinsic parameters of the thermal infrared camera, the 3D coordinates of the point are projected onto the thermal infrared image plane to obtain its corresponding pixel position, and the thermal radiation intensity value is read from that position. This thermal radiation intensity value is related to the animal's body surface temperature. The first step is to analyze the local neighborhood point cloud around the point and select nearby points within a certain radius to estimate the surface normal vector of the point using a plane fitting method. The surface normal vector is a three-dimensional direction vector, and its three components represent the projection of the surface orientation onto the three coordinate axes, reflecting the geometric orientation and curvature of the object at that point. The second step is to calculate the Euclidean distance from the point to the nearest obstacle point in the entire point cloud scene. This distance value is used to measure the safety margin of the point's location; the smaller the distance, the closer it is to the obstacle.

[0024] After extracting the above information, each LiDAR point is no longer just a geometric point with three spatial coordinates, but a composite data unit carrying multi-dimensional information. The system constructs a fused feature vector for each point. This vector is composed of the following components: red, green, and blue color values ​​obtained from the visible light image, which together describe the visual appearance of the point; thermal radiation intensity value obtained from the thermal infrared image, which describes the surface temperature characteristics of the point; the three components of the surface normal vector calculated from the local neighborhood, which describe the geometric orientation of the point; and the Euclidean distance from the point to the nearest obstacle obtained from scene analysis, which describes the safe space information of the point. These eight values ​​are concatenated into an eight-dimensional feature vector. By combining the eight-dimensional vectors of all points in the point cloud, the so-called fused point cloud is formed.

[0025] This fused point cloud has the following significant advantages: it simultaneously contains color information, temperature information, 3D geometric information, and obstacle distance information, truly achieving spatial unification of multi-source information; based on this fused point cloud, subsequent instance segmentation can more accurately distinguish sheep from the background because it includes not only color and geometric cues but also body temperature cues; multi-target tracking can more reliably maintain individual identity because the spatial trajectory and geometric shape of each sheep in its local area are continuously recorded; the construction of a 3D semantic map can also directly perform operations such as ground extraction and obstacle annotation based on this fused point cloud; the entire process is interconnected, providing a solid perceptual data foundation for upper-level decision-making modules.

[0026] The process of extracting multidimensional individual features is as follows: The system has been specifically improved based on the widely used YOLOv8 object detection framework, adding a parallel mask prediction branch, which enables it to output detection boxes and instance masks simultaneously.

[0027] After the input image enters the backbone network, the backbone network extracts three layers of feature maps at different scales. These three feature maps are downsampled by 8 times, 16 times, and 32 times the original input image, respectively. That is, the width and height of each feature map are 1 / 8, 1 / 16, and 1 / 32 of the original image, respectively. The reason for this design is that shallow feature maps have high resolution and rich details, which are suitable for segmenting small livestock or fine contour edges; deep feature maps have large receptive fields and strong semantic information, which are suitable for recognizing large livestock and overall shape.

[0028] In the detection branch, each feature map layer predicts the regression parameters of the bounding box and the class probability at each location. At the same time, a specially added prototype mask generation network generates a set of prototype masks from the feature map, with the number of prototype masks set to 32. The spatial size of each prototype mask is 1 / 4 of the width and height of the input image, which is equivalent to medium resolution, preserving sufficient spatial details without bringing too much computational burden. These 32 prototype masks constitute a basic shape dictionary, in which some prototypes may tend to activate the head region of livestock, some tend to activate the torso region, and some tend to activate the leg region.

[0029] For each detected individual animal, the mask branch outputs an additional coefficient vector of length 32, where each value corresponds to a weight of a prototype mask. The 32 coefficients of the animal are then linearly weighted and summed with the 32 prototype masks. That is, each prototype mask is multiplied by its corresponding coefficient, and all results are added together. The final value is then compressed to the range of 0 to 1 using an activation function. Regions with values ​​greater than 0.5 are identified as the mask regions of that individual. The advantage of this decomposition method is that the model learns to break down the task of generating mask shapes into a prototype combination task. It has a stronger generalization ability for animals with different poses and body shapes, especially when the animals are close to each other and part of their bodies are occluded. This dictionary-based combination method is more robust than directly predicting the complete mask.

[0030] Once a mask has been generated for all detected livestock individuals, the system extracts the mask for each instance. The mask is a binary matrix of the same size as the input image, where pixels with a value of 1 represent the region belonging to that livestock, and pixels with a value of 0 represent the background or other livestock. By overlaying the mask onto the original image, the complete region of each livestock in the image can be accurately cropped.

[0031] For each cropped livestock region, the system uses a region of interest alignment operation to pool regions of different shapes and sizes into fixed-size 14x14 feature blocks. Each feature block retains 128 feature channels, thus obtaining a feature representation matrix with constant dimension regardless of the distance of the livestock in the image or how its posture changes. These feature blocks are fed into a residual network for further encoding, and finally output a 128-dimensional appearance embedding vector. This vector can be understood as the appearance fingerprint of this livestock in the current frame. For the same livestock, the value of this vector should be highly similar in consecutive frames.

[0032] In addition to physical features, the system also extracts features for each animal from multiple dimensions, summarizing them to form a multidimensional individual feature vector.

[0033] In terms of geometric features, the system extracts three quantization metrics from the binary region of the mask. The first is the principal axis length, which is determined by principal component analysis of the mask region to find the maximum extension direction of the pixel distribution. The span in this direction is the principal axis length, which roughly corresponds to the body length of the livestock from head to tail. The second is the vertical width, which is the width of the pixel distribution in the direction perpendicular to the principal axis, roughly corresponding to the body width of the livestock. The third is the mask area, which is the total number of pixels with a value of 1 in the mask, roughly corresponding to the projected area of ​​the livestock on the image, and is directly related to the size of the livestock.

[0034] In terms of thermal features, the point-to-pixel mapping relationship established in the previously fused point cloud was used to locate the area corresponding to the animal mask on the thermal infrared image. The thermal radiation intensity values ​​of all pixels in the area were counted, and their mean and standard deviation were calculated. The mean thermal radiation reflects the overall temperature level of the animal's current body surface, and the standard deviation reflects the uniformity of the body surface temperature. Abnormal increases in local temperature are often related to stress response or inflammation.

[0035] Regarding motion characteristics, the system records the pixel coordinates of the center point of the animal's mask in the current frame and compares them with the coordinates of the center point of the animal's mask in the previous frame. It calculates the displacement between the two points and divides it by the time interval between frames to obtain the instantaneous velocity of the animal at the current moment. This velocity is represented by a two-dimensional vector, with two components corresponding to the speed of movement in the horizontal and vertical directions, respectively, and the unit is pixels per second. The magnitude and direction of the instantaneous velocity are important bases for judging the animal's behavioral state. When the velocity exceeds the threshold for normal walking, it may mean that the animal is running or has been frightened.

[0036] In summary, each detected animal is encoded as a multidimensional individual feature vector in each frame. This vector consists of: 128-dimensional appearance features, 3-dimensional geometric features (principal axis length, vertical width, and mask area), 2-dimensional thermal features (mean temperature and standard deviation of temperature), and 2-dimensional motion features (horizontal velocity and vertical velocity), totaling 135 dimensions. These 135 values ​​comprehensively describe the individual state of an animal at the current moment from four dimensions: visual appearance, geometric size, body temperature, and movement pattern.

[0037] In densely populated livestock farming environments, frequent mutual occlusion and similar physical features among livestock pose a significant challenge to maintaining individual identity. Traditional methods rely solely on matching appearance features, which can easily lead to confusion in appearance vectors when livestock turn around, squat down, or intersect with other individuals, resulting in incorrect identity switching. To address this issue, this solution proposes a three-level cascaded correlation tracking strategy that integrates motion features.

[0038] The system maintains an independent Kalman filter for each tracked animal. The state vector maintained by this filter contains six components, namely: the horizontal and vertical pixel coordinates of the animal's mask center on the image, the horizontal velocity, the vertical velocity, the horizontal acceleration, and the vertical acceleration. At each time step, the Kalman filter first predicts the current state based on the state of the previous time step and the control model. When a new detection result arrives, the predicted value is fused with the measured value to continuously correct the estimation of the animal's position.

[0039] Once a new frame of image has been segmented to obtain the detection results of all livestock in the current frame, the tracking algorithm enters the three-level association matching process.

[0040] The first level of association is strong association matching; the system calculates the comprehensive association cost between the detection result of each current frame and each existing tracking trajectory; this cost is obtained by weighted summation of three indicators; the first is the appearance feature distance, that is, the Euclidean distance between the appearance embedding vector of the detection result and the appearance embedding vector maintained by the trajectory, the smaller the distance, the more likely they are to be the same animal; the second is the spatial location distance, that is, the Euclidean distance between the trajectory center position predicted by the Kalman filter and the actual center position of the detection result, the closer the predicted value is to the measured value, the smaller the cost; the third... The term is the mask intersection-union ratio complement, which is the intersection-union ratio between the mask of the detection result and the mask when the trajectory last appeared. The higher the intersection-union ratio, the better the overlap between the two regions. After taking the complement, it is converted into a cost index. The weights of the three terms are set to 0.4, 0.3 and 0.3 respectively, emphasizing both appearance features and motion space information. After calculating the cost matrix between all detections and all trajectories, the Hungarian algorithm is used to solve for the globally optimal matching scheme. For each pair of matching combinations, if the comprehensive cost is lower than the preset strong correlation threshold, the match is considered successful, and the identity of the current detection is assigned to the corresponding trajectory.

[0041] The second level of association is weak association matching, specifically handling detections and trajectories that failed to match in the first level. In this level, the system only uses motion features for association judgment, no longer relying on appearance information. This is because some livestock may undergo drastic changes in posture, such as turning or squatting, leading to abrupt changes in their appearance embedding vectors. The appearance cost in the first level is too high, resulting in a failed match, but their movement path remains continuous. Specifically, the Mahalanobis distance between the predicted position of the existing trajectory by the Kalman filter and the actual position of the unmatched detection is calculated. Mahalanobis distance is a standardized distance that considers the covariance of the data distribution and reflects the degree of deviation of the data in a statistical sense better than Euclidean distance. If the Mahalanobis distance is less than the corresponding threshold of the chi-square distribution at 95% confidence, the detection and trajectory are considered to meet the motion consistency constraint, and the association is established. This level of matching can effectively rescue livestock individuals whose appearance features temporarily change but whose movement trajectories are obvious, preventing identity interruption.

[0042] The third level of association processing handles new and lost trajectories. For detection results that still do not match any trajectory after two levels of matching, the system assumes that a new animal may have entered the field of vision and initializes it as a new trajectory to be confirmed. This new trajectory is in the unconfirmed state for the first 3 frames. If it can be stably matched for 3 consecutive frames, it is upgraded to a formal trajectory. For existing trajectories that have not been matched by any detection for 3 consecutive frames, they are marked as lost, their identity information is temporarily retained but they do not participate in matching. If they are still not matched for 10 consecutive frames, it is determined that the animal has left the perception range or has been in a state of complete occlusion for a long time, and its trajectory is permanently deleted.

[0043] The process of constructing a rasterized semantic map of the pasture is as follows: A lidar odometry method based on geometric feature points is adopted. For each frame of input point cloud, two types of points with significant geometric features are first screened from the massive 3D points. The first type is edge points, which are located on the surface of objects where the curvature changes drastically, such as the edge of a fence post, the corner of a trough, and the ridgeline of livestock. The method to determine whether a point is an edge point is to calculate the roughness of the local point set formed by the point and several of its nearest neighbors in a certain direction. Points with larger roughness values ​​are considered to be on the geometric edge. The second type is planar points, which are located on relatively flat surfaces, such as pasture ground, wall facades, and the side of stationary vehicles. The method to determine planar points is that the roughness value of the local point set is small, and these points can be well fitted by a local plane.

[0044] Between two consecutive point cloud frames, the system uses a variant algorithm of iterative nearest point to find the correspondence between edge points and planar points in the previous frame and in the current frame. For each edge point in the previous frame, it finds the edge line formed by the nearest edge point of the same type in the current frame and calculates the vertical distance from the point to the corresponding edge line. For each planar point in the previous frame, it finds the local plane formed by the nearest planar point of the same type in the current frame and calculates the vertical distance from the point to the corresponding plane. Using all these distances from points to lines and points to planes as constraints, an optimization problem is constructed regarding the rotation and translation parameters between the two frames. By iteratively solving the problem, the sum of all distances is minimized, thereby obtaining an accurate estimate of the relative pose transformation between frames.

[0045] By accumulating and multiplying the inter-frame transformation matrices obtained in each frame in chronological order, the global pose of the lidar relative to its starting position at any given time can be obtained. Using these global poses, the local point cloud of each frame is transformed to a unified world coordinate system. By stitching the point cloud frame by frame, a dense three-dimensional point cloud model covering the entire pasture area is gradually constructed, which is the three-dimensional geometric map of the pasture.

[0046] After obtaining the 3D geometric map of the ranch, it is necessary to assign semantic meanings to different areas in the map so that they can be effectively utilized by the decision-making module. The system uses a variety of visual and geometric cues to perform semantic classification on each voxel in the 3D point cloud map.

[0047] Ground recognition is the first and most crucial step in semantic annotation. The system uses a random sampling consensus algorithm in conjunction with a planar model to extract ground points from the entire point cloud. The algorithm randomly selects three non-collinear points to determine a candidate plane and counts the number of internal points in the entire point cloud that are less than a certain tolerance away from the candidate plane. After multiple random iterations, the plane with the most internal points is identified as the pastureland. All points within 0.05 meters of this ground plane are marked as grassland or passable ground.

[0048] The identification of fences and barriers utilizes their geometric features; these objects are vertical linear structures that extend along a certain direction, have a certain height in the direction perpendicular to the ground, and have a periodically repeating cross-section. The system searches for clusters of points that are continuously distributed along a certain horizontal direction, with a height exceeding the ground by 0.5 meters to 2.0 meters, and with approximately uniform spacing between adjacent points in non-ground points, and marks these clusters as fences. For barriers composed of posts and beams, identification is also performed by detecting clusters of points arranged in long straight lines.

[0049] The identification of water sources combines geometric depression features and laser reflection intensity features. In a pasture environment, the point cloud at water troughs or natural water sources usually presents a local depression shape on the ground. Due to the specular reflection effect of the water surface, the lidar receives a very weak echo signal, and the point cloud appears sparse or even hollow in this area. The system detects areas that meet both the conditions of local depression and sparse reflection points and marks them as water sources.

[0050] Building identification is relatively simple; any continuous structure above the ground with a regular facade and flat roof, a height exceeding 2.5 meters, and a high point cloud density is labeled as a building; various scattered obstacles that do not belong to any of the above categories are uniformly labeled as obstacles.

[0051] Each voxel in the 3D point cloud map is given a semantic label, which may include: grass, fence, water source, building, and obstacle.

[0052] While 3D semantic maps are rich in information, their massive data volume makes them unsuitable for real-time querying and path planning by the decision-making module. Therefore, it is necessary to reduce their dimensionality to 2D raster maps while retaining key semantic and navigability information.

[0053] The 3D point cloud map is vertically projected onto a horizontal plane, and uniform square grids are divided on the horizontal plane, with each grid having a side length of 0.5 meters; the entire pasture area is divided into a two-dimensional grid array composed of these 0.5-meter by 0.5-meter grids.

[0054] For each grid, the system performs the following information aggregation operation: First, it statistically analyzes the semantic label distribution of all LiDAR points within the area corresponding to the grid, and selects the semantic label that appears most frequently as the dominant semantic type of the grid; for example, if more than 80% of the points in a grid are marked as grassland, then the semantic type of the grid is grassland.

[0055] Calculate the accessibility index of the grid; accessibility is defined as a value between 0 and 1, with a higher value indicating that the grid is more suitable for robot passage; for grids marked as grassland and with flat terrain, accessibility is directly determined by the area ratio of grassland points within the grid: divide the number of points marked as grassland in the grid by the total number of points in the grid, and if the ratio is greater than 0.7, the grid is considered accessible; if the dominant semantic of the grid is a building, fence, or obstacle, its accessibility is directly set to 0, indicating that it is completely impassable; for water source areas, the grid accessibility is set to 0, but an additional water source marker is reserved for the decision module to refer to when planning livestock drinking paths.

[0056] The median elevation of all points within the grid is used as the average elevation of the grid; the elevation difference between adjacent grids is used for subsequent robot slope assessment and anti-tipping constraints.

[0057] The group driving decision module uses multiple deployed mobile robots as multiple agents, constructs a state space by combining multi-dimensional individual features and semantic maps, designs an animal welfare risk index based on the state space, and a reward function with navigation safety as a weighted term. Through a multi-agent deep deterministic policy gradient algorithm, it obtains a group driving strategy for driving livestock to the grouping area, and outputs speed commands for each robot based on the group driving strategy. The process of constructing the state space is as follows: Each mobile robot is equipped with a real-time dynamic differential global positioning system and an inertial measurement unit, which can acquire its own position and motion information with centimeter-level accuracy. When constructing the state space, the robot first needs to clarify the spatial relationship between itself and the task target.

[0058] The system pre-marks the target area for the herding task on the map. This target area is usually the entrance to the group waiting area. For the k-th robot, the system calculates the relative position between the robot's current position and the center of the target area and represents this relationship in polar coordinates. The polar coordinates contain two components: one is the straight-line distance from the robot to the target area, and the other is the angle between the robot's forward direction and the target direction. The distance value is normalized by dividing it by the maximum diagonal length of the pasture so that it falls within the range of 0 to 1. The angle value is expressed in radians and divided by pi to normalize it to between -1 and 1. This polar coordinate representation is more intuitive for robot navigation than using rectangular coordinates directly. The angle information directly tells the robot which direction to adjust its course, while the distance information tells the robot how far it still needs to travel.

[0059] In addition to location information, the robot's current velocity is also an important component of the state space. The velocity is represented by a two-dimensional vector, with two components representing the robot's velocity along the horizontal and vertical axes of the world coordinate system on the ground plane, respectively, in meters per second. The velocity vector is also normalized by dividing each component by the robot's maximum operating speed, limiting its value range to between -1 and 1.

[0060] The target of the herding task is a flock of sheep consisting of several animals. Therefore, the group state of the flock is the core basis for the agent's decision-making. The system extracts statistical information at the group level from multi-dimensional individual features to construct a compact representation of the flock's group state.

[0061] First, the system determines the position of the geometric center of the sheep flock relative to the robot. The system obtains the mask center positions of all tracked sheep in the current frame from the perception module, averages the three-dimensional coordinates of these positions, and obtains the geometric center point of the flock. Then, it calculates the polar coordinates of this geometric center relative to the current robot, represented by two components: normalized distance and normalized deflection angle. This information allows the robot to know the overall location of the flock and its distance from it, serving as the most basic spatial reference in the herding strategy.

[0062] Secondly, there's the sheep density distribution. Knowing only the center of the flock isn't enough; the dispersion and distribution pattern directly influence the choice of herding strategy. The system divides the horizontal plane into 25 equal-angled fan-shaped regions (5 x 5) centered on the robot's current position. Each fan-shaped region covers a radial distance from 0 meters to the maximum effective distance of the sensing range. For each of these 25 fan-shaped regions, the proportion of currently detected sheep in that region relative to the total number of sheep in the flock is calculated, resulting in a 25-dimensional density distribution vector. Each element of this vector is a value between 0 and 1. When the flock is concentrated in one direction, the values ​​in a few corresponding fan-shaped regions are higher than the others. When the flock is evenly dispersed, the values ​​in each fan-shaped region tend to be average. The density distribution vector provides the agent with spatial layout information about the flock, helping the robot determine whether it needs to move around to the side or rear of the flock to herd it.

[0063] During the process of driving away obstacles, the robot must avoid collisions with fences, buildings, and other obstacles, while planning a reasonable route using passable areas; environmental obstacle information comes from a gridded semantic map constructed by the perception module.

[0064] The system emits 16 virtual detection rays evenly in the surrounding planar space, centered on the robot's current position. The angular interval between adjacent rays is 22.5 degrees, covering the entire 360-degree circumference. For each ray, the system searches outward grid by grid along the ray direction from the rasterized semantic map, recording the Euclidean distance from the first impassable grid encountered in that direction to the robot. If no impassable area is encountered within the maximum detection distance of the ray, the distance value in that direction is recorded as 30 meters. Finally, a 16-dimensional obstacle distance vector is obtained, with each dimension ranging from 0 to 30 meters. After dividing by 30 meters, the value is normalized to 0 to 1. These 16 distance values ​​collectively characterize the 360-degree passability space around the robot. The larger the distance value, the more open and safe the direction is; the smaller the distance value, the closer the direction is to the obstacle.

[0065] In addition, the system also extracts the accessibility information of the area where the sheep are currently located from the semantic map; by querying the accessibility values ​​of the grid where the geometric center of the sheep is located and its 8 neighboring grids, the average value is taken as the overall accessibility of the terrain where the sheep are located; this value is also in the range of 0 to 1, which is used to help the agent determine whether the current terrain is suitable for continuing to drive the sheep. If the accessibility is too low, it is necessary to consider guiding the sheep to change routes.

[0066] This scheme introduces a parameter in the state space that directly reflects the stress level of livestock, enabling the agent to perceive the impact of its driving behavior on animal welfare when making decisions, thereby learning a more gentle and efficient strategy. The design of this parameter is based on two core indicators from the multidimensional individual characteristics output by the multi-source perception fusion module: body surface temperature characteristics and instantaneous movement speed characteristics.

[0067] The system calculates a real-time updated welfare risk index for the sheep flock, which consists of two parts. The first part is the body temperature deviation. For each tracked sheep, the system continuously maintains its resting body surface temperature baseline. The baseline is calculated through a slowly updated sliding window, taking the average of all body surface temperatures of that sheep over the last 60 seconds. At the current decision point, the absolute value of the difference between each sheep's current average body surface temperature and its own baseline is calculated, and then the absolute value of this temperature difference of all sheep in the flock is averaged to obtain the group average body temperature deviation. The higher the body temperature deviation, the stronger the overall stress of the sheep flock.

[0068] The second part is the proportion of high-speed moving individuals. The system sets a speed warning threshold as the standard for judging whether livestock are in a state of panic running. This threshold is set to 2 meters per second based on the normal walking speed of sheep. For each sheep, the magnitude of its instantaneous movement speed vector is obtained. If the magnitude exceeds the warning line of 2 meters per second, the sheep is marked as a high-speed moving individual. The number of high-speed moving individuals in the entire flock is counted and divided by the total number of sheep to obtain a proportion value between 0 and 1. The higher the proportion, the more individuals in the flock are in a state of panic.

[0069] The welfare risk index is obtained by summing the above two items, dividing by 2, and taking the average value, which ranges from 0 to 1. This index is directly incorporated into the local observation state of each robot as a separate dimension of the state vector. When the welfare risk index is high, the animal welfare risk term in the reward function will give a larger negative reward. During the policy optimization process, the agent will learn the association between high welfare risk and adverse consequences, and thus actively converge to a more gentle driving behavior mode.

[0070] Meanwhile, the system also maintains a short-term welfare risk memory for each robot. Specifically, the welfare risk index of the three most recent consecutive decision moments is concatenated into a time-series vector of length 3. In this way, the agent not only knows the current stress level, but also can perceive whether the stress level is rising, falling, or remaining stable, thereby making more forward-looking policy adjustments.

[0071] All components are concatenated into a complete state vector in a fixed order. For the k-th robot, its state vector consists of the following components: normalized target distance and deflection angle (2 dimensions); normalized current robot speed (2 dimensions); normalized relative distance and deflection angle between the sheep flock center (2 dimensions); 25-dimensional sheep flock density distribution vector; 16-dimensional circumferential obstacle distance vector; 1-dimensional terrain accessibility of the sheep flock location; 1-dimensional real-time welfare risk index; and 3-dimensional short-term welfare risk memory. The total number of these components is 52.

[0072] The process of designing an animal welfare risk index is as follows: When livestock are exposed to external stressors, they exhibit a series of physiological and behavioral stress responses, the most typical of which are changes in body surface temperature and abnormal movement behavior.

[0073] At the physiological level, when livestock are subjected to stress stimuli such as fright, chasing, or rough driving, their sympathetic nervous system is rapidly activated, resulting in a large secretion of adrenaline and noradrenaline. This triggers peripheral vasoconstriction and increased blood flow to skeletal muscles, leading to transient but measurable abnormal fluctuations in body surface temperature. These fluctuations may manifest as a sharp rise in body temperature or a drop in temperature at the extremities. By continuously monitoring livestock body surface temperature non-contactly using thermal infrared imaging and comparing it with the individual's historical baseline temperature in a resting state, signals of stress response in the physiological dimension can be captured.

[0074] At the behavioral level, livestock exhibit abnormal escape behavior when in a state of fear or pain. The most obvious manifestation is that the running speed suddenly exceeds the activity intensity range of normal walking. Under normal circumstances, the spontaneous movement speed of sheep is usually between 0.5 meters per second and 1.5 meters per second. When the speed significantly exceeds 2 meters per second, it can be almost certain that the individual is in a state of escape after being frightened. In addition, the higher the proportion of individuals running at high speed in the entire flock at the same time, the greater the intensity and scope of the stressor.

[0075] Based on the above principles of livestock behavioral physiology, this scheme combines two indicators—body surface temperature deviation and the proportion of individuals moving at high speed—to construct a comprehensive animal welfare risk index.

[0076] The system maintains an independent historical sequence of body surface temperature for each continuously tracked animal. This sequence is updated by the thermal features extracted in each frame by the multi-source sensing fusion module. The thermal features include two elements: the mean body surface temperature, which is the arithmetic mean of the temperature values ​​corresponding to the radiation intensity of all thermal infrared pixels in the masked area of ​​the animal; and the standard deviation of body surface temperature, which reflects the uniformity of the body surface temperature of the individual.

[0077] To determine whether the current temperature is abnormal, the system needs to establish a normal body temperature baseline for each animal under resting conditions. The baseline establishment method adopts a sliding window averaging strategy: the system retains all valid temperature measurements for the animal within the last 60 seconds, removes obviously abnormal outliers, and then calculates the arithmetic mean of the remaining values, which is the animal's current normal body temperature baseline value. This baseline is updated once per second, using a slow update method, so that the baseline can adapt to normal body temperature fluctuations caused by diurnal rhythms and changes in ambient temperature, while not being contaminated by short-term stress-induced body temperature fluctuations.

[0078] At each decision point, the system calculates the absolute value of the difference between the current average body surface temperature of each animal in the flock and its own normal body temperature baseline, which is the temperature deviation of that individual. If an animal's normal body temperature baseline is 39.0 degrees and the current measurement is 40.2 degrees, then its temperature deviation is 1.2 degrees. The arithmetic mean of the temperature deviations of all individuals in the flock is taken to obtain the average temperature deviation of the flock. The larger the value of the average temperature deviation, the stronger the physiological stress response of the flock as a whole.

[0079] In parallel with the body temperature deviation, the proportion of high-speed moving individuals is calculated based on motion characteristics; the multi-source perception fusion module provides an instantaneous motion velocity vector for each tracked animal; this vector consists of two components, representing the pixel movement speed of the animal in the horizontal and vertical directions on the image plane, respectively. After conversion by camera calibration parameters, the motion speed in the real physical world can be obtained, with the unit being meters per second.

[0080] The system sets a speed warning threshold of 2 meters per second. This threshold is based on sheep behavior research data: sheep usually move at a speed of no more than 1.5 meters per second when grazing freely, and about 1 meter per second when grazing. However, when startled, they can run at a speed of more than 3 meters per second. Therefore, setting the warning line at 2 meters per second can effectively distinguish between normal activity and frightened running. It can sensitively detect stress escape behavior without generating a large number of false alarms due to fluctuations in normal activity.

[0081] At each decision point, the system iterates through all tracked livestock in the flock. For each animal, it calculates the magnitude of its current instantaneous velocity vector, which is the square root of the sum of the squares of the horizontal and vertical velocity components. If the magnitude exceeds 2 meters per second, the animal is marked as a high-speed moving individual. The number of individuals marked as high-speed moving individuals in the entire flock is counted and divided by the total number of individuals in the flock to obtain a decimal between 0 and 1, which is the proportion of high-speed moving individuals. When the proportion is 0, it indicates that the flock is in a completely normal activity state. When the proportion is close to 1, it indicates that almost the entire flock is running in panic, and the stress is severe.

[0082] After obtaining two sub-indicators, the group's average body temperature deviation and the proportion of individuals moving at high speeds, the system synthesizes them into a unified animal welfare risk index. Since the two sub-indicators have different dimensions and value ranges, they need to be normalized separately.

[0083] The normalization method for the average body temperature deviation of the group is as follows: divide it by the preset maximum tolerance value for body temperature deviation, which is set to 2.0 degrees; when the average body temperature deviation reaches or exceeds 2.0 degrees, the normalization value is 1; when the average body temperature deviation is 0, the normalization value is 0; a linear mapping is performed in between; the proportion of individuals in high-speed movement is already a value in the range of 0 to 1, and no additional processing is required.

[0084] The welfare risk index is synthesized by weighted average of normalized body temperature deviation and the proportion of individuals with high-speed movement. The weight of each sub-indicator is 0.5, indicating that physiological stress and behavioral stress are equally important in the comprehensive assessment. The final value of the welfare risk index ranges from 0 to 1, with a larger value indicating a more severe welfare risk to the flock.

[0085] The position of this index in the state space is reflected on two levels. First, the real-time welfare risk index, as an independent dimension of the state vector, allows the agent to directly perceive the immediate stress level caused to livestock by the current driving behavior at each decision moment. Second, in order to endow the agent with the ability to perceive stress trends, the system also maintains a short-term welfare risk memory vector. This vector is composed of the welfare risk indices of the three most recent consecutive decision moments stored in a first-in-first-out queue. That is, the index values ​​of the current moment, the previous 0.5 seconds, and the previous 1.0 seconds are arranged into a temporal micro-vector of length 3. Through these three consecutive values, the agent can determine whether the welfare risk is increasing, decreasing, or remaining stable, thereby making forward-looking policy adjustments.

[0086] The ultimate goal of designing the animal welfare risk index is to establish a feedback pathway between driving behavior and animal welfare consequences in the reward mechanism of reinforcement learning; in the reward function of the group driving decision module, the animal welfare risk term is directly calculated by the welfare risk index.

[0087] Specifically, at each decision step, the system assigns a negative reward (penalty) to the agent based on the calculated welfare risk index of the current state. The severity of the penalty is positively correlated with the value of the welfare risk index: the higher the index, the heavier the penalty; when the index is close to 0, almost no penalty is imposed. This mechanism allows the agent to gradually discover, through trial and error and learning to optimize the long-term cumulative reward, that action sequences that cause a sharp increase in the welfare risk index will bring significant penalty costs, while behaviors such as maintaining an appropriate distance and guiding the flock from the side and rear at a gentle speed can simultaneously achieve both driving efficiency and low welfare risk. Ultimately, the agent autonomously learns driving strategies, accurately driving the flock to the target area while minimizing livestock stress losses associated with the driving operation.

[0088] It is worth noting that the normal baseline body temperature of livestock is not constant; the diurnal rhythm causes body temperature to fluctuate regularly from 0.5 degrees to 1.0 degrees throughout the day, and the daily variation of ambient temperature also affects the rate of heat dissipation from the body surface; in addition, there are differences in the basal body temperature level of livestock of different breeds, ages, and physiological stages.

[0089] To enable welfare risk assessment to adapt to these dynamic changes, the system incorporates an adaptive update mechanism in maintaining the body temperature baseline. The baseline sliding window design ensures that older measurements naturally slide out of the window over time, while new measurements continuously slide in, allowing the baseline to slowly track normal changes in livestock body surface temperature over long-term scales. Simultaneously, an outlier removal mechanism prevents occasional stress-induced temperature fluctuations from contaminating the baseline, ensuring its purity. Furthermore, the system can integrate circadian rhythm information to maintain independent baseline values ​​for different time periods, further reducing the interference of diurnal rhythms on welfare assessment. The process of designing a reward function with navigation safety as a weighted term is as follows: The system uses each mobile robot as the center and divides the area around the robot into three concentric safety zones based on the distance between the robot and various obstacles.

[0090] The innermost layer is the danger zone, defined as a circular area with a radius of 0.5 meters centered on the robot's geometric center. This 0.5-meter threshold is determined based on the robot's own dimensions and safe braking distance. The omnidirectional mobile robot used in this solution has an external dimension of approximately 0.8 meters in length and 0.6 meters in width, and its outer envelope radius is approximately 0.5 meters. When any obstacle, other robot, or animal enters this area, it means that actual contact has occurred or a collision is about to occur, which is an emergency situation that must be absolutely avoided.

[0091] The middle layer is the warning zone, defined as a ring-shaped area with a radius of 0.5 meters to 1.5 meters. This area is a safety buffer zone. When an obstacle or other moving target enters this area, although a collision has not yet occurred, the room left for the robot to react and adjust its trajectory is quite limited. Within this area, the system needs to send a relatively strong warning signal to the agent to prompt it to take evasive actions such as deceleration and turning.

[0092] The outermost layer is the safety zone, defined as the entire space beyond a radius of 1.5 meters. Within this range, the robot has ample time to react to obstacle avoidance and can perform its driving task normally, with zero or minimal navigation safety penalties.

[0093] This three-tiered safety zone division allows navigation safety items to apply differentiated penalties based on the urgency of the danger, avoiding a one-size-fits-all approach to safety assessments that could lead to overly conservative agent behavior and reduced driving efficiency.

[0094] During the state space construction process, the system has generated a 16-dimensional circumferential obstacle distance vector for each robot; this vector records the normalized distance values ​​of the nearest impassable obstacles in 16 evenly distributed directions around the robot's circumference; the design of the navigation safety item directly utilizes this ready-made state representation for collision risk assessment.

[0095] For each of the 16 directions, the system maps the normalized distance of that direction to the corresponding security status level. The normalized distance is the ratio of the actual physical distance to the maximum effective detection distance of 30 meters. Therefore, when the normalized distance is 1, the corresponding actual distance is greater than or equal to 30 meters, and when the normalized distance is close to 0, the corresponding actual distance is close to 0 meters.

[0096] The judgment logic is as follows: if the actual distance corresponding to the normalized distance in a certain direction is less than 0.5 meters, it means that the obstacle in that direction has entered the danger zone, triggering a danger status marker; if the actual distance is between 0.5 meters and 1.5 meters, it means that the obstacle is in the warning zone, triggering a warning status marker; if the actual distance is greater than 1.5 meters, it is considered safe in that direction, and no marker is generated.

[0097] To aggregate information from 16 directions into a single scalar indicator reflecting the overall level of danger, the system employs a combination of extreme value statistics and average statistics. First, it takes the minimum normalized distance among the 16 directions, with this minimum distance representing the most pressing collision threat currently facing the robot. Second, it counts how many of the 16 directions are in the danger zone and how many are in the warning zone. These two statistics together characterize the spatial distribution and severity of collision risks around the robot.

[0098] Based on the above quantification results, the navigation safety item calculates a penalty value for each robot at each decision time. This penalty value will be added to the total reward signal as a negative reward.

[0099] The penalty function is designed according to the principle of tiered penalty, with heavier penalties for closer objects. It first handles the most urgent cases of intrusion into dangerous areas. If the minimum distance in any of the 16 directions is below the danger threshold of 0.5 meters, the system applies a larger baseline penalty value. Moreover, the penalty intensity increases with the number of directions that intrude into the dangerous area: for each additional direction that enters the dangerous area, the penalty value is increased proportionally. This means that when the robot approaches an obstacle simultaneously from multiple directions, it will be subject to more severe penalties, thus prompting it to prioritize moving in open directions.

[0100] Secondly, the system handles the situation in the warning area. If the minimum distance is between 0.5 meters and 1.5 meters and no direction enters the danger zone, the system applies a medium-intensity penalty value. Similarly, the more directions that enter the warning area, the heavier the penalty. The penalty value in the warning area is about one-quarter of the penalty value in a single direction in the danger zone, reflecting the gradient design of the more sufficient the safety margin, the lighter the penalty.

[0101] If the actual distances in all 16 directions are greater than 1.5 meters, meaning the robot is completely within a safe zone, then the navigation safety item will not incur any penalty and will contribute zero, allowing the agent to focus on optimizing other objectives such as driving efficiency and group cohesion.

[0102] In addition, the penalty function introduces a dynamic modulation factor related to the robot's current speed. When the robot moves towards an obstacle at a high speed, even if the distance in that direction is still within the warning zone, the actual collision risk is amplified because the braking distance requirement increases with speed. Therefore, the penalty value is multiplied by a speed modulation coefficient, which is determined by the ratio of the robot's speed component in the direction towards the nearest obstacle to its maximum speed. The higher the ratio, the larger the modulation coefficient. This design enables the robot to automatically reduce its speed in narrow or obstacle-dense areas, achieving a more robust safety strategy.

[0103] In scenarios where multiple robots work together to herd sheep, navigation safety includes not only the relationship between the robots and static environmental obstacles, but also the dynamic safety constraints between robots and between the robots and the sheep; these constraints are also included in the penalty framework of navigation safety items.

[0104] For robot-to-robot safety, the system monitors the distance between each robot and all other robots in real time. The safety distance threshold between any two robots is set at 1.0 meter, which is approximately twice the radius of the robot's outer envelope circle, leaving sufficient collision avoidance margin. When the distance between any two robots is less than 1.0 meter, an additional collision penalty is added to the navigation safety items of the closer robot. The intensity of this penalty is inversely proportional to the distance between the robots: the smaller the distance, the greater the penalty, which encourages the robots to actively disperse and avoid overcrowding in the formation.

[0105] Regarding safety between the robot and the sheep, the system extracts the distance between the sheep center and the robot from the sheep density distribution vector in the state space, and monitors whether individuals at the edge of the sheep are too close to the robot. The safe distance threshold between sheep and the robot is set at 1.5 meters, slightly larger than the safe distance between robots. This is because livestock, as living animals, have more unpredictable movement trajectories than robots, and getting too close to the robot will directly induce stress in the livestock, complementing the animal welfare risk item. When any sheep is less than 1.5 meters away from the robot, the navigation safety item activates additional penalties to prompt the robot to maintain an appropriate following distance when herding, avoiding close herding.

[0106] The aforementioned environmental obstacle avoidance penalty, robot collision penalty, and robot-sheep spacing penalty together constitute the complete navigation safety item; each penalty value is calculated and multiplied by its respective sub-weight coefficient, and then summed to obtain the total penalty value of the navigation safety item.

[0107] The total penalty value of the navigation safety item is used as the fourth weighted component in the reward function of the group driving decision module, and is combined with the driving efficiency item, group cohesion item, and animal welfare risk item in a weighted summation manner.

[0108] In the weighting of the four weighted items, the navigation safety item has a weight of 0.15. This weight value is a result of careful consideration: if the weight is too high, the agent will become overly conservative, preferring to remain stationary rather than approach any obstacles, leading to a sharp decline in driving efficiency; if the weight is too low, the agent may ignore collision risks in pursuit of efficiency, frequently causing scrapes or even crashing into fences. The weight of 0.15 makes the navigation safety item play a soft constraint role in the total reward. It will not completely dominate the agent's behavior, but it will apply a sufficiently strong penalty signal to correct dangerous actions when a collision risk does exist.

[0109] The process of outputting the speed commands for each robot is as follows: The reason for choosing the multi-agent deep deterministic policy gradient algorithm in this scheme is that the driving task has dual requirements for the continuity of actions and the cooperation of multiple agents. Unlike algorithms such as deep Q-networks that only support discrete action spaces, the deep deterministic policy gradient algorithm naturally supports the output of continuous action spaces. The robot's speed command can be freely selected in a two-dimensional continuous range, which can achieve smooth and precise motion control. Its multi-agent extended version further solves the problem of environmental non-steadiness when multiple agents learn at the same time. That is, even when the policies of other agents are constantly changing, each agent can still learn a stable cooperative policy.

[0110] Each mobile robot, acting as the kth agent, maintains two sets of deep neural networks: an online network for actual decision-making and training updates, and a target network for value estimation during stable training. Each network contains two sub-networks: the actor network, which outputs actions directly based on the current local observations, i.e., a continuous action vector; and the critic network, which evaluates the expected value of the long-term cumulative reward that can be obtained by performing joint actions in the current global state.

[0111] The actor network is structured as a four-layer fully connected feedforward neural network. The input layer receives the local observation state vector of the agent, with a dimension of 52 as defined earlier. The two hidden layers each contain 256 neurons and use a linear rectified function as the activation function to extract nonlinear high-order features in the state. The output layer contains two neurons, corresponding to the two components of the action vector, namely the longitudinal velocity adjustment and the lateral velocity adjustment. The activation function of the output layer is a hyperbolic tangent function, which naturally limits the output value to between -1 and 1, which corresponds exactly to the normalized range of the velocity adjustment.

[0112] The Critics Network also employs a four-layer fully connected structure, but its input dimension is much larger than that of the Actors Network. During the intensive training phase, the Critics Network can obtain the joint state and joint actions of all agents. Therefore, its input consists of two concatenated parts: the first part is the concatenated vector of the local observation states of all K agents, with a total dimension of 52 multiplied by K; the second part is the concatenated vector of the action vectors of all K agents, with a total dimension of 2 multiplied by K. This global information is only visible during the training phase and is not needed during the execution phase. The hidden layer structure of the Critics Network is the same as that of the Actors Network, and the output layer has only one neuron, outputting a scalar value representing the expected long-term cumulative reward, i.e., the Q-value, under the current global state and joint actions.

[0113] The online network and the target network have the same structure. The parameters of the target network are slowly approximated by the parameters of the online network through soft updates. The soft update coefficient is usually set to about 0.005, that is, in each round of training, the parameters of the target network are updated only by a tiny step of five-thousandths in the direction of the online network parameters, thereby ensuring the stability of the target value and the convergence of the training process.

[0114] Before the actual training begins, all robots’ online network parameters are first randomly initialized, and the target network parameters are copied from the online network. In order to enable the agent to fully explore various possible combinations of states and actions in the early stage of training, an exploration noise that follows a normal distribution is superimposed on the action output. The standard deviation of the noise is large in the early stage of training, gradually decays as training progresses, and eventually stabilizes at a very small value, achieving a smooth transition from extensive exploration to refined utilization.

[0115] At each decision-making moment, with a time interval of 0.5 seconds, the system executes the following interactive loop: First, each robot uses the output of its own sensors and perception modules to acquire the current local observation state vector; then, each robot's actor network calculates a deterministic motion vector based on the current local observation state. Since it is in the training phase, random exploration noise is superimposed on the motion vector, and this noise is sampled independently on each component of the motion vector; this noisy motion vector is then clipped to an effective range of -1 to 1; third, all robots use their respective motion vectors as velocity commands to act on their respective actuators, while simultaneously... The system moves and interacts with the environment; fourth, after 0.5 seconds, the environment enters a new state, and the system recalculates the values ​​of each weighted term in the reward function to obtain the environmental feedback reward for this step; fifth, the system stores the experience data generated by this interaction in the experience replay buffer; each piece of experience data contains the following: the joint local observation state of all agents at the current moment, the joint action vector executed by all agents, the total environmental feedback reward shared by all agents, and the joint local observation state of all agents at the next moment; the capacity of the experience replay buffer is set to 1 million experience data, and when the buffer is full, the oldest record will be eliminated one by one.

[0116] After a certain amount of empirical data has accumulated in the experience replay buffer, the system begins to train the network parameters of the agent in a concentrated manner. In each round of training, a small batch of empirical data is randomly sampled from the buffer. The batch size is usually set to 64 or 128 data points to ensure the stability of gradient estimation and computational efficiency.

[0117] For each sampled experience, the core operation of centralized training is to calculate a target value for the critic network of each agent. The steps for calculating the target value are as follows: First, input the local observation state of each agent in the next time step into their respective target actor networks to obtain the deterministic potential optimal action of each agent in the next time step without exploration noise. Second, input the joint state and joint potential optimal action of all agents in the next time step into the target critic network of the k-th agent to obtain a target Q value. Third, add the immediate reward of this step to the target Q value and multiply it by a discount factor to obtain the expected target value of the agent. The choice of the discount factor reflects the trade-off between long-term planning and short-term interests. A higher discount factor makes the agent more concerned about the final result of the driving task rather than the small gains and losses in the short term.

[0118] The loss function of the online critic network is defined as the square of the error between the predicted Q-value of the online critic network for the current joint state and joint action output and the target value calculated above. The gradient of this loss function with respect to the parameters of the online critic network is calculated using the backpropagation algorithm, and the parameters of the online critic network are updated using the gradient descent optimizer so that its predicted Q-value gradually approaches the target value.

[0119] The online actor network is then updated; the optimization goal of the actor network is to maximize the critic network's evaluation of its output action, that is, to make the critic network believe that the actor's strategy can bring high long-term returns; the policy gradient is calculated using the chain rule: first, the gradient of the critic network's output Q value with respect to the joint action vector is calculated, then the part of this gradient corresponding to the k-th agent's action is multiplied by the gradient of the k-th agent's output action with respect to its own parameters; this gradient is used to update the parameters of the online actor network, so that the actor network's output action moves in the direction that can obtain a higher Q value.

[0120] After all agents' online networks have been updated, a soft update operation is then performed on the target networks of all agents. Each parameter of the target actor network is slightly shifted in the direction of the corresponding parameter in the online actor network according to the soft update coefficient, and each parameter of the target critic network is also slightly shifted in the direction of the corresponding parameter in the online critic network according to the soft update coefficient.

[0121] The above training steps are repeated in each round. Actual training is usually carried out in a simulation environment first because it is too costly and time-consuming to learn from scratch using real robots and sheep in a real pasture. The simulation environment constructs a virtual pasture scene based on a gridded semantic map of the pasture. The sheep behavior model adopts a multi-agent simulation method based on group movement rules, which can realistically simulate the group behavior of sheep when they are driven, such as gathering, dispersing, turning and escaping. After the strategy converges to a satisfactory level in the simulation environment, it is fine-tuned and transferred using a small amount of real scene data before finally being deployed to a real pasture.

[0122] After training convergence, the agent enters a distributed execution mode. In this mode, the critic network of each robot no longer participates in the computation, and only the actor network is retained for real-time decision-making. The advantage of this design is that the robot only needs its own local observation state during execution, without needing to obtain information from other robots or rely on a central control node. The system has strong robustness and scalability.

[0123] In each decision cycle, each robot independently performs the following operations: First, it obtains the latest local observation state vector from the perception module and sends the 52-dimensional state data into the local actor network for a forward propagation. The forward propagation calculation of the actor network is completed on the embedded artificial intelligence computing unit, taking less than milliseconds, which fully meets the real-time requirement of a 0.5-second decision cycle. The actor network outputs a 2-dimensional motion vector, with two components representing the normalized longitudinal velocity adjustment and lateral velocity adjustment, respectively. The system performs inverse normalization on the motion vector, that is, multiplies each component by the robot's maximum speed of 3 meters per second to obtain the velocity adjustment in the true physical sense, in meters per second.

[0124] Next, the system superimposes the speed adjustment amount with the robot's current actual speed to obtain a new desired speed command. If the current robot speed is 1.0 m / s horizontally and 0.5 m / s vertically, and the actor network output adjustment amount is +0.1 horizontally and -0.05 vertically, then after inverse normalization, an increment of +0.3 m / s horizontally and -0.15 m / s vertically is obtained, and the new speed command is 1.3 m / s horizontally and 0.35 m / s vertically. To avoid the speed command changing too drastically and causing the robot's movement to be uneven, the system performs smoothing and amplitude limiting processing on the rate of change of the speed command, and the change amplitude between two adjacent commands does not exceed the product of the maximum acceleration and the decision interval.

[0125] The smoothed speed command is sent to the robot's underlying motion controller via a wireless communication network. The underlying motion controller runs on the robot's embedded system. After receiving the speed command, it converts it into the target speed of each drive wheel and performs closed-loop speed regulation through a proportional-integral-derivative control algorithm, ultimately enabling the robot to move smoothly on the pasture ground.

[0126] A significant advantage of the multi-agent deep deterministic policy gradient algorithm is that, although the policy of each robot is updated independently during training, the agents will spontaneously form a tacit cooperative behavior pattern during training by sharing the reward function and the global perspective of the critic network. This cooperation is not achieved by artificial rule programming, but emerges naturally from the optimization process of maximizing the common reward.

[0127] Typical collaborative patterns include: when a flock of sheep attempts to disperse and escape to one side, the robots closest to that side will proactively move diagonally forward in the direction of escape to block the escape route, while the robots on the opposite side will appropriately shrink their positions to fill gaps in the formation; when encountering narrow passages, the robots will automatically form a front-to-back queue, with the robots in front guiding the flock into the passage, and the robots behind maintaining an appropriate distance at the end of the flock to prevent them from falling behind; when a robot's path is blocked by an obstacle, adjacent robots will proactively adjust their own trajectories, taking on greater responsibility for herding while leaving detour space for their obstructed companions.

[0128] The individual grouping decision module collects the grouping channel occupancy status and waiting queue length after livestock enter the grouping waiting area. It integrates the body condition information estimated from visual recognition and the historical health records obtained from radio frequency identification to construct an individual profile for each livestock. Using the grouping channel occupancy status, the current individual profile, and the waiting queue length as the state, and the allocation to different grouping channels as the action, the module obtains the individual grouping strategy through dual deep Q-network optimization. The process of collecting the occupancy status of the group channels and the length of the waiting queue is as follows: The herd waiting area is a transitional zone connecting the pasture area with the entrances of each herd passage. Its function is to accommodate sheep that have been driven here and are waiting to be assigned to their respective destinations. On one side of the exit of this waiting area, the system has pre-set several herd passages arranged side by side. In this scheme, the number of passages is set to 5, numbered from 1 to 5 respectively. Each herd passage is a narrow fenced passage with a width of about 1.0 to 1.5 meters, allowing only one animal to pass through at a time. Each passage is physically isolated from the others by fence partitions, ensuring that animals passing through a certain passage will not come into physical contact with animals in adjacent passages or accidentally enter between passages.

[0129] These five grouping channels each lead to different destinations; channels 1 to 4 correspond to different feeding and management groups, such as fattening group, breeding group, supplementary feeding group, and isolation observation group. The feeding programs, feed formulas, and daily management measures of each group are different; channel 5 is set up as a release channel, which is used to allow individuals that do not need to be grouped or whose grouping conditions are not yet met to return to the main pasture area.

[0130] To achieve real-time monitoring of the occupancy status of each passageway, the system installs a pair of through-beam photoelectric sensors at the entrance and exit of each group passageway. These sensors consist of an infrared transmitter and an infrared receiver, mounted on fence posts on either side of the passageway, facing each other at a height of approximately 0.6 meters above the ground, roughly corresponding to the mid-section of an adult sheep's torso. Under normal conditions without livestock, the infrared beam emitted by the transmitter directly illuminates the receiver, causing it to output a high-level signal. When livestock enter the passageway, their bodies block the infrared beam, preventing the receiver from receiving the light, and the output signal immediately flips to a low level. By simultaneously monitoring the changes in the levels of the two sets of photoelectric sensors at the entrance and exit, the system can accurately determine whether the passageway is currently occupied by livestock. The specific judgment logic is as follows: if the entrance sensor is triggered earlier than the exit sensor, and there is a time difference between the two, it indicates that a livestock is passing through the passageway; if neither the entrance nor exit sensor is triggered, the passageway is idle.

[0131] Within each decision cycle of the individual grouping decision module, the system performs occupancy status checks on each of the five grouping channels. For the grouping channel numbered g, the occupancy status is represented by a binary variable: a value of 0 indicates that the channel is currently completely empty, with no livestock passing through, and the next livestock to be grouped can be accepted; a value of 1 indicates that the channel is currently occupied, meaning that one livestock is passing through inside, and no new livestock can be accepted at this time.

[0132] The specific logic flow of the detection is as follows: The system first reads the current signal level of the photoelectric sensor at the entrance of channel g; if the entrance sensor signal is high, it indicates that there are no livestock blocking the entrance and the front of the channel is empty; if the entrance sensor signal is low, it indicates that there are livestock entering or waiting to enter at the entrance; reads the current signal level of the photoelectric sensor at the exit of the same channel; if the exit sensor is also high, combined with the entrance being high, it can be determined that there are no livestock inside the channel, and the occupancy status is set to 0; if the entrance is low and the exit is high, it indicates that the head of the livestock has entered but has not yet reached the exit, and the channel is being occupied, and the occupancy status is set to 1; if the entrance changes from low to high, and the exit changes from high to low and then back to high, it indicates that the livestock has completed the entire passage and left the channel, the channel is now empty, and the occupancy status switches back from 1 to 0; if both the entrance and exit sensors show a low level for a long time, it may mean that the livestock is stuck in the middle of the channel, which is an abnormal occupancy status, and the system also marks its occupancy status as 1 and triggers an abnormal prompt to notify the feeding management personnel to check.

[0133] The occupancy status detection results of the 5 channels are summarized into a 5-dimensional binary vector in each decision cycle. The value of the g-th element in the vector is 0 or 1, which corresponds to the idle and occupied status of the g-th channel, respectively. This 5-dimensional vector constitutes the "grouping channel occupancy status" component in the individual grouping decision state space. Through this vector, the individual grouping decision agent can grasp the availability of all available destination channels in real time, so as to prioritize the selection of idle channels during allocation and avoid pushing livestock into channels that are already occupied.

[0134] Since the grouping passage can only allow one animal to pass at a time, when the entire flock is gathered in the grouping waiting area, it is impossible for all the animals to enter the passage at the same time. They must be processed one by one in order. Therefore, the system needs to organize all the animals waiting to be grouped in the waiting area into an orderly waiting queue and count the queue length in real time for decision-making.

[0135] The waiting queue is organized according to the principle of first-come, first-served. Based on the continuous tracking of the individual identity of each animal by the multi-source perception fusion module, the system records the timestamp of each animal's first entry into the group waiting area. The area of ​​the group waiting area is pre-defined as a polygonal geofence on the map. When the center point coordinates of a certain animal's mask fall into the polygonal area for the first time, the system records the current time as the animal's arrival time.

[0136] All livestock that have entered the grouping and waiting area but have not yet been assigned a passage are arranged in ascending order of their arrival time, forming a logical waiting queue. The livestock that arrived earliest and waited the longest are at the front, and the livestock that arrived latest are at the back. The length of the waiting queue is the total number of livestock currently in the queue that are waiting to be processed, represented by a non-negative integer. When the length of the waiting queue is 0, it means that there are no livestock waiting to be grouped, and all livestock that have arrived at the waiting area have been assigned.

[0137] Each animal's individual file also records its current rank in the waiting queue; a rank of 1 indicates that the animal is at the front of the queue and will be processed in the next round of grouping decisions; a rank equal to the queue length indicates that the animal is at the back of the queue and must wait for all the animals in front of it to be processed before it is its turn; when an animal is assigned to a channel by the individual grouping decision agent and begins to enter the channel, the system removes the animal from the waiting queue, decrements the queue length by 1, and the rank of all subsequent animals is moved forward in sequence to fill the vacancy; if a new animal enters the grouping waiting area, the system adds it to the end of the waiting queue, and the queue length is increased by 1.

[0138] Through this explicit queue management mechanism, the system transforms the disordered flock of sheep into an ordered sequence to be processed, ensuring that the grouping process is orderly and that no individual is missed, nor are there any instances of queue jumping or duplicate processing.

[0139] At the beginning of each decision cycle, the individual group decision module simultaneously performs the two operations mentioned above: channel occupancy status detection and waiting queue length statistics, and summarizes the obtained data into the complete basic state at the current moment.

[0140] The aggregated data consists of three components: First, a 5-dimensional binary vector representing the occupancy status of each grouping channel, reflecting the current availability of each channel; second, the current waiting queue length, i.e., the total number of livestock still waiting to be processed; and third, the individual profile of the livestock at the front of the queue, which is the grouping target to be processed in the current decision-making cycle. Its individual profile has been constructed in the previous steps by combining visually estimated body condition information and historical health records read by radio frequency identification, and includes fields such as estimated weight, body condition score, recent health score, breed code, physiological stage, and age in months.

[0141] The process of creating an individual file for each animal is as follows: The acquisition of body condition information relies on the multi-dimensional individual features extracted by the multi-source perception fusion module in the preceding steps, as well as the body condition estimation algorithm executed on this basis. The system needs to estimate two core body condition indicators from visual data: weight and body condition score.

[0142] The task of weight estimation is to infer the true weight of livestock from 2D images and 3D point clouds. The system uses a binocular camera and instance segmentation masks to jointly locate the 3D region of the livestock's back. In the fused point cloud, each spatial point has been labeled with its corresponding livestock individual number, so the 3D spatial points belonging to the current livestock's back region can be directly filtered out. After statistical filtering to remove stray noise, these point clouds form a relatively clean 3D point set of the back. Body size parameters are extracted from this point set, including the curve length of the back along the spine, and the width and height values ​​at multiple cross-sections perpendicular to the spine. These body size parameters are fed into a pre-trained weight regression network. This network is a lightweight fully connected neural network. The input layer receives multiple body size parameters and several front-end dimensions of the appearance feature vector. After nonlinear transformation in the hidden layer, the output layer directly outputs a scalar value, namely the estimated weight of the livestock, in kilograms. The training data for this network comes from labeled samples collected from real weighing equipment, ensuring that the accuracy of weight estimation is within the range required for practical use.

[0143] Body condition scoring focuses on assessing the livestock's condition, specifically the degree of subcutaneous fat accumulation. This is a classification task, with an output of a discrete score from 1 to 5, where 1 indicates extreme emaciation, 5 indicates excessive obesity, and around 3 indicates ideal body condition. The estimation is based on two factors: first, the curvature characteristics of the back point cloud. Livestock in good condition have fuller back muscles and fat layers, resulting in a rounded arc outline in the point cloud; livestock in poor condition have prominent spine and rib outlines, and the point cloud surface is uneven with drastic curvature variations. Second, the ratio of body width to body length, and the deviation of this ratio from the standard parameters for the same breed and age. The body condition scoring classifier also uses a lightweight fully connected network. The input is a curvature feature vector and body size ratio parameters. The output layer, after being mapped by a normalized exponential function, provides a probability distribution for five categories, and the category with the highest probability is taken as the final score.

[0144] In addition to the two core indicators of weight and body condition score, the system also extracts auxiliary body condition features from visual data, including coat condition assessment and gait abnormality detection. Coat condition is assessed through visible light image texture analysis, and rough and messy coat is often associated with malnutrition or disease. Gait abnormalities are detected through continuous multi-frame motion feature sequence analysis. If an animal is found to have uneven strides, limping, or unsteady standing, it is marked as having a motion abnormality. These auxiliary features are recorded as supplementary fields to the body condition information.

[0145] While the vision system estimates the condition of the livestock, the radio frequency identification (RFID) system works simultaneously to confirm the unique identity of the livestock. Each animal is fitted with a low-frequency or ultra-high-frequency RFID electronic ear tag when it enters the livestock management system. The ear tag stores a globally unique identification code. Multiple RFID read / write antennas are deployed at the entrance and inside the passage of the group waiting area. When the livestock enters the effective reading range of the read / write antenna, the ear tag is activated and transmits its identification code back to the reader.

[0146] The RFID system provides an identification code and uses this code as an index key to initiate a query request to the livestock management database. The livestock management database is a centralized data storage system that maintains a complete record for each animal throughout its entire life cycle. Key historical fields stored in the database include: the animal's date of birth and current age, breed code and genetic background information, parent information, weight records and timestamps from each weighing, manual assessment records of each body condition score, past medical history and medication records, vaccination records, breeding records, recent feed intake statistics, and recent activity statistics. Among these, feed intake and activity data from the past 7 days are extracted separately and combined into a comprehensive health score using a preset health scoring algorithm. This score reflects the animal's overall health trend over the past week, with a value ranging from 0 to 100, where a higher value indicates better health.

[0147] Another key function of RFID association is to bind the individual livestock tracked in visual data to the identity record in the database one-to-one. During the multi-target tracking process of the multi-source perception fusion module, each livestock has a temporary tracking number assigned by the system. When the livestock enters the RFID reading and writing range and its ear tag is successfully read, the temporary tracking number establishes a permanent correspondence with the globally unique identification code. All subsequent data records, feature extractions and file updates for the livestock will use this identification code as the primary key.

[0148] After acquiring real-time physical condition data from visual estimation and historical health data retrieved from the database, the system enters the data alignment and fusion stage. The core challenge of this stage is that the two types of data have different time attributes and confidence levels. Visual estimation data reflects the immediate state at the current moment and has high timeliness but may be affected by estimation errors. Historical health data reflects long-term accumulated trend information and has high reliability but is lagging in time. The goal of fusion is to fully absorb the advantages of both while avoiding their respective limitations.

[0149] The alignment process first matches two types of data to the same animal based on the identification code. Then, timestamp alignment ensures that the historical data used is the most recently updated version. If some fields in the database have not been updated for a long time, such as the most recent weighing record being more than 30 days ago, the system prioritizes the visually estimated weight and marks the database record as an expired reference. If the two weighing records are within 7 days, the visually estimated value and the database record value are weighted and averaged, with the visually estimated value having a weight of 0.6 and the historical record value having a weight of 0.4, thus balancing real-time performance and reliability.

[0150] The fusion strategy for body condition scores is similar; visual scores are the primary reference, and historical manual scores are used for auxiliary verification; if the difference between the two exceeds 1 point, the system issues a data conflict warning, which is then remotely reviewed and confirmed by the animal husbandry management personnel; if the difference is within 1 point, the visual score is directly adopted.

[0151] The health score is constructed in a comprehensive manner. The system combines multiple sub-items such as food intake over the past 7 days, activity level statistics, recovery status in past medical history, and the degree of deviation of the current body condition score from the standard value according to preset weights. Specifically, food intake and activity level each account for 0.25, representing the recent physiological function status; the deviation of the current body condition score from the standard value accounts for 0.25, representing the nutritional and body condition status; and the recovery status of past medical history accounts for 0.25, representing potential health risks. After normalization, each sub-indicator is weighted and summed to obtain the current comprehensive health score.

[0152] After data alignment and fusion are completed, the system organizes all information into a fixed-length individual profile vector according to a preset data structure, so that the individual grouping decision-making agent can directly read and process it.

[0153] The individual profile vector consists of the following fields in order: Estimated weight (in kilograms), a comprehensive estimate of body weight; Body condition score, an integer between 1 and 5 or a real value rounded to one decimal place; Overall health score, a value between 0 and 100; Breed code, a predefined category number used to distinguish different breeds and genetic strains; Age in months, the number of whole months from the date of birth to the current date; Physiological stage, a coded value indicating the current reproductive or growth stage, such as lactation, rearing, pregnancy, non-pregnant, etc.; Coat condition marker, 0 for normal and 1 for abnormal roughness; Movement abnormality marker, 0 for normal gait and 1 for lameness or abnormal gait.

[0154] The process of obtaining individual grouping strategies is as follows: Each individual group decision-making agent faces a composite state consisting of three parts of information in each decision cycle. These three parts of information are concatenated into a fixed-dimensional state vector, which serves as the direct input to the subsequent Q-network.

[0155] The first part is the grouping channel occupancy status vector; this vector consists of 5 binary values ​​arranged in sequence, each corresponding to the current occupancy status of the 5 grouping channels; the value at each position is 0 or 1, 0 indicates that the channel is idle and available, and 1 indicates that the channel is occupied or in an abnormal state and is unavailable; these 5 binary values ​​together constitute the basis for the legality constraint of the current selectable action, because an occupied channel should logically not be selected as the current destination of the livestock.

[0156] The second part is the individual profile vector of the livestock to be grouped; this vector is the 8-dimensional vector constructed in the previous steps, with each dimension being: estimated weight, body condition score, comprehensive health score, breed code, age in months, physiological stage, coat condition marker, and movement abnormality marker. These 8 values ​​describe the overall status of the livestock to be processed from different dimensions, providing an individualized basis for the agent to determine which management group the livestock should be assigned to.

[0157] The third part is the waiting queue length; this is a separate scalar integer value representing the total number of livestock still waiting to be processed in the current waiting queue. The impact of queue length on decision-making is that when the queue is long, the agent needs to appropriately favor the channel with faster processing speed to alleviate backlog pressure; when the queue is short or about to be emptied, the agent can more leisurely pursue the grouping accuracy.

[0158] The above three parts of information are concatenated in a fixed order. The first part is a 5-dimensional channel occupancy vector, the second part is an 8-dimensional individual profile vector, and the third part is a 1-dimensional queue length scalar. The complete state vector after concatenation has 5 plus 8 plus 1 dimensions, for a total of 14 dimensions. This 14-dimensional vector is refreshed at the beginning of each decision cycle and sent to the dual deep Q network for forward propagation.

[0159] The action space of an individual grouping decision-making agent is a discrete space containing 5 selectable actions, represented by integers 0 to 4 respectively; each action corresponds to a target grouping channel, with action 0 corresponding to channel 1, action 1 corresponding to channel 2, and so on up to action 4 corresponding to channel 5.

[0160] The five channels each lead to a different management group; Channel 1 leads to the fattening group, receiving individuals whose weight and body condition are suitable for intensive fattening; Channel 2 leads to the breeding group, receiving ewes that have entered the mating period or are pregnant; Channel 3 leads to the supplementary feeding group, receiving individuals with low body condition scores who need enhanced nutritional supplementation; Channel 4 leads to the isolation and observation group, receiving individuals with low health scores who are at risk of disease and need to be observed separately; Channel 5 is the release channel, leading to the main pasture area, for individuals that do not need to be separated into groups for the time being.

[0161] At each decision point, the agent needs to select one of these five optional actions as the output. It is important to note that currently occupied channels are theoretically unselectable, because pushing another animal into a channel that is already being used by an animal would cause blockage or physical injury between the animals. Therefore, when making the final action selection, the system applies a mask to the Q-value of the action corresponding to the occupied channel, temporarily setting its Q-value to negative infinity to ensure that the agent will not select these unavailable actions.

[0162] The dual deep Q-network is the core algorithm for decision optimization in this module. Compared with the standard deep Q-network, its key improvement lies in decoupling the two stages of action selection and action value evaluation, which are handled by two sets of network parameters respectively. This effectively alleviates the problem of overestimation of Q value that is common in conventional deep Q-networks, making the learned strategy more robust and reliable.

[0163] The dual-depth Q-network in this scheme consists of two Q-networks with identical structures, referred to as the online network and the target network, respectively. Each network adopts a four-layer fully connected feedforward neural network structure. The input layer contains 14 neurons, corresponding to each component of the 14-dimensional state vector. The two hidden layers contain 128 and 64 neurons respectively, both using linear rectified functions as activation functions. The output layer contains 5 neurons, corresponding to the Q-value estimation of 5 selectable actions. The Q-value is a scalar that represents the expected value of the long-term cumulative reward obtained after performing the action in a given state. Actions with higher Q-values ​​are considered better.

[0164] The difference between online networks and target networks lies in the role of parameters and their update method. The parameters of an online network are the main parameters currently being learned and optimized, and are directly updated in each training step through backpropagation and gradient descent. The parameters of a target network are copied from the online network, but do not participate in backpropagation. Instead, the parameters of the online network are completely copied to the target network every certain number of steps. When calculating the training target value, the target network is used to evaluate the Q-values ​​of various actions in the next state, rather than the online network, thereby breaking the overestimation bias chain caused by using the same network for selection and evaluation in bootstrapping estimation.

[0165] The reward function for individual grouping decisions consists of three weighted terms, corresponding to the three optimization objectives of grouping accuracy, queue fluency, and channel load balancing. In each decision cycle, after the agent selects a channel for the current livestock and completes the grouping, the environment provides a comprehensive reward value based on the result of the selection. The calculation methods for each reward term are explained below.

[0166] The grouping accuracy reward is the core component of the reward function, with a weight of 0.5. The system pre-determines a target channel for each animal to be grouped. This rule is pre-defined by the feeding and management personnel based on the needs of animal husbandry and management. The specific logic is as follows: if the animal's health score is below 60, the target channel is the No. 4 isolation observation channel; if the health score is greater than or equal to 60 and the body condition score is below 2.5, the target channel is the No. 3 supplementary feeding channel; if the body condition score is greater than or equal to 2.5 and the physiological stage is pregnancy or mating period, the target channel is the No. 2 breeding channel; if the body condition score is greater than or equal to 2.5 and the weight has exceeded the slaughter standard, the target channel is the No. 1 fattening channel; otherwise, the target channel is the No. 5 release channel. If the agent's selected action is consistent with the target channel, a reward of +1 is given; if inconsistent, a penalty of -1 is given. This reward mechanism directly drives the agent to learn to accurately identify the individual state of each animal and assign it to the correct management group.

[0167] The weight of the queue smoothness reward is set to 0.3; this reward is directly linked to the length of the waiting queue, and the penalty value is the current waiting queue length divided by the preset maximum tolerable queue length; the penalty is smaller when the queue is short and larger when the queue is long; this design prompts the agent to appropriately speed up the decision-making pace when the queue is severely backlogged, so as to avoid aggravating the waiting congestion due to excessive caution; when the queue length is 0, the smoothness penalty is also 0, and no negative impact is generated.

[0168] The weight of the channel load balancing reward is set to 0.2. The reward is calculated by counting the number of livestock received by each of the five channels since the start of this round of grouping and calculating the variance of these five values. The smaller the variance, the more balanced the load of each channel is. The larger the variance, the more busy some channels are while others are too idle. The load balancing penalty is the variance divided by the preset maximum tolerance variance value. This encourages the agent to prioritize allocating livestock to the channel with the lighter current load when multiple channels are equally reasonable choices, thereby making full use of the parallel processing capabilities of each channel and improving the overall grouping efficiency.

[0169] The total reward value is obtained by weighting the above three items according to their respective weights. During the training process, this comprehensive reward signal is continuously fed back to the agent, enabling it to gradually learn to find the optimal balance between grouping accuracy, processing speed and load balancing through repeated trial and error.

[0170] With states, actions, network structure, and reward functions in place, the agent enters the formal training phase. The training data first comes from offline pre-training in a simulation environment. The simulation environment simulates the complete physical layout of the grouped waiting area and five channels, and generates a large number of individual profiles of virtual livestock according to the distribution of real breeding data. Each virtual livestock has the correct target channel calculated according to rules. In the simulation environment, the agent can interact with the simulation environment tens of thousands of times, quickly accumulating experience without relying on the slow and costly trial and error process in a real ranch.

[0171] In each training round, a small batch of historical experience data is randomly sampled from the experience replay buffer. The format of each experience data is: the current state vector, the action number selected by the agent, the immediate reward obtained after executing the action, and the state vector of the next time step. For each sampled experience, the target value corresponding to the experience is calculated. The calculation logic is as follows: First, find out which of the 5 actions in the next state is evaluated by the online network as having the highest Q value, and record the action number. Then, query the Q value of the action with that number in the target network. Then, multiply the Q value by the discount factor and add the immediate reward to obtain the target value. The discount factor is usually set to a value close to 1 but less than 1, such as 0.95, which indicates that the agent attaches great importance to future rewards, but also retains a certain time deduction.

[0172] The loss function of the online network is: the squared error between the online network's predicted Q-values ​​for the current state and the current action and the aforementioned target value; the gradient of this loss function is calculated using the backpropagation algorithm, and all parameters of the online network are updated using the Adam optimizer; every 100 such parameter updates are performed, the parameters of the online network are completely copied to the target network once.

[0173] After tens of thousands of rounds of simulation training, the Q-value estimation of the online network gradually converges, and the agent learns to accurately select the target channel for each virtual animal while taking into account both smoothness and load balancing. At this point, the trained online network parameters are saved and migrated and deployed to the edge computing nodes of a real ranch. In the real environment, the agent will also continue to perform low-frequency online fine-tuning to adapt to the subtle differences between the real data distribution and the simulation data, ensuring the long-term effectiveness of the strategy.

[0174] In the online decision-making phase after training, each decision cycle lasts 0.5 seconds. The agent performs the following operations: First, it obtains the current state vector, i.e., the latest 14-dimensional data, from the perception and queue management modules. Then, it sends the state vector into the online network for a forward propagation calculation, obtaining 5 output values, each corresponding to the Q value of one of the 5 selectable channels. Next, it checks the channel occupancy state vector and forcibly replaces the Q value corresponding to the currently occupied channel with negative infinity, completing the action mask. Finally, among the remaining valid channels, it selects the channel with the largest Q value as the output action for the current decision.

[0175] In the collaborative execution and safety management module, each robot tracks its trajectory in real time through distributed model predictive control, based on speed commands and individual grouping strategies, while satisfying kinematic constraints and safety distance constraints between robots and between robots and livestock. The real-time trajectory tracking process is as follows: The system deploys an independent embedded control unit on each mobile robot. This unit integrates a real-time dynamic differential global positioning system receiver module, an inertial measurement unit, a wireless communication module, and a processor with edge computing capabilities. The control unit continuously receives speed commands from the herding decision module via a wireless network. These commands are two-dimensional vectors representing the robot's desired movement speed along the horizontal and vertical axes of the world coordinate system on the ground plane, in meters per second. Simultaneously, the control unit also receives phased commands for individual grouping strategies. For example, when a grouping task is initiated, some robots need to switch from a herding formation to a guard formation, forming a mobile fence around the grouping standby area to prevent the flock from scattering and escaping.

[0176] The robot's underlying actuator consists of four independently driven omnidirectional wheels. By controlling the coordinated rotation of each wheel's speed, it can achieve translation in any direction within a plane and rotation around a vertical axis. However, speed commands issued directly from the decision-making level cannot be directly applied to the motors because the commands may have abrupt changes, violate physical limits, or introduce collision risks. The role of distributed model predictive control is to faithfully track the intended command while incorporating all kinematic constraints and safe distance constraints into the optimization framework, automatically generating a smooth and safe actual motion trajectory.

[0177] In order to make accurate predictions of the motion state for several steps in the future within the control unit, it is first necessary to establish a concise and sufficiently accurate robot kinematic model. This solution adopts the kinematic model of an omnidirectional mobile platform, which abstracts the robot's motion into three degrees of freedom planar motion: translation along the horizontal axis of the world coordinate system, translation along the vertical axis, and rotation about the vertical axis.

[0178] At the beginning of each discrete control cycle, the robot's state is described by three quantities: its horizontal coordinate, vertical coordinate, and orientation angle in the world coordinate system; the control input consists of two translational velocity components and one rotational angular velocity, which can freely take values ​​within the maximum range allowed by the hardware; during the time interval of the control cycle, assuming that the control quantities remain unchanged, the robot's new position and orientation at the next moment can be calculated based on geometric relationships, which is the basic equation for predicting the state recursion.

[0179] This kinematic model is linear or approximately linear, and can be easily embedded in constrained optimization problems of model predictive control, ensuring real-time solution.

[0180] In each control cycle, the robot needs to solve a trajectory optimization problem in a finite time domain. The goal of this problem is to find an optimal set of control inputs in the predicted future steps, such that the deviation between the robot's actual motion trajectory and the desired speed command is minimized, the change of control variables is as smooth as possible, and all safety constraints are strictly satisfied.

[0181] The objective function consists of a weighted sum of two parts. The first part is the speed tracking error penalty, which is the sum of the squares of the differences between the robot's predicted speed and the speed command in each future prediction step. The smaller the difference, the more faithfully the robot executes the driving intention of the decision layer. The second part is the control increment penalty, which is the sum of the squares of the differences between the control inputs of two adjacent steps. This term is used to suppress drastic changes in control commands, ensure smooth and stable robot movement, and avoid sudden acceleration or sharp turns. The weights of the two parts can be set according to actual debugging needs. Usually, the tracking error weight is larger to ensure the task execution effect.

[0182] The design of the constraints covers three levels: robot physical limits, static environment safety, and dynamic interaction safety. The physical limit constraints include: the control input for each step must not exceed the maximum value allowed by the hardware, such as a translation speed limit of 3 meters per second and a rotation speed limit of 180 degrees per second; the change in control input between two adjacent steps must not exceed the upper limit corresponding to the maximum acceleration capability of the motor, ensuring that the instructions can be physically executed.

[0183] Safety distance constraints are crucial for ensuring the safe coexistence of multi-robot systems and livestock herds; these constraints fall into three categories: collision avoidance between robots and static obstacles, collision avoidance between robots and other robots, and maintaining a safe distance between robots and the sheep flock.

[0184] For static obstacle avoidance, the robot utilizes a gridded semantic map constructed by a multi-source perception fusion module. In each prediction step, the robot predicts its future position and queries whether the grid containing that position is passable. If the passability requirement is not met, the predicted state is determined to be infeasible, and the optimizer must find other paths. To improve optimization efficiency, the system also transforms the obstacle distance vector into a linear constraint form, constraining the robot's future position within a feasible space beyond a certain safety margin from the obstacle.

[0185] For collision avoidance between robots, each robot broadcasts its currently solved predicted trajectory sequence to other robots at a fixed frequency via a wireless communication network; at the same time, it also receives predicted trajectories broadcast by other robots. When constructing its own optimization problem, the robot uses the predicted positions of other robots as dynamic obstacles, constraining the Euclidean distance between its future position and the positions of other robots at the same step to always be greater than the preset safety distance of 1.0 meter. If communication is interrupted for some reason, the robot will use the last predicted trajectory broadcast by other robots in the previous cycle as a reference, and temporarily expand the safety distance to 1.5 meters to increase the conservative margin. This distributed collision avoidance method based on predicted trajectory sharing does not require a central coordination node, and each robot can achieve collision-free cooperative motion within the formation by solving independently.

[0186] To constrain the safe distance between the robot and the sheep flock, the system utilizes the sheep density distribution information and the real-time position of each sheep in the state space. The robot treats each animal in the flock as a moving obstacle to be avoided, constraining the minimum distance between its future position and the sheep's current position to be no less than 1.5 meters. Since the sheep's movement trajectory is difficult to predict accurately, the system uses a short-time linear extrapolation method to predict the sheep's position at the next moment based on its current velocity vector, and uses this to construct the distance constraint. At the same time, this distance constraint is linked to the animal welfare risk index: when the welfare risk index is high, the safe distance threshold automatically increases to 2.0 meters, forcing the robot to maintain a greater distance from the sheep flock and avoid further exacerbating stress. This mechanism achieves a closed-loop connection between navigation safety and animal welfare risk at the physical execution level.

[0187] The solution of distributed model predictive control is repeated every 0.1-second control cycle. Each time, the robot uses its latest measured position, velocity, and orientation angle as the initial state, and solves the constrained optimization problem online according to the latest received velocity command and constraints to obtain the optimal control sequence for the next 10 steps. The solution algorithm uses an efficient numerical optimizer suitable for small-to-medium scale quadratic programming problems, which can ensure that the calculation is completed within the control cycle.

[0188] Following the principle of rolling time-domain control, only the first step of the optimized sequence is executed each time, i.e., the speed command applied to the motor in the current 0.1 second. In the next 0.1 second cycle, the robot remeasures its state, receives the latest command and the predicted trajectory from its neighbors, solves the optimization problem again, and then executes the new first step of the control. This continuous rolling update gives the system a strong anti-interference capability: whether the actual movement deviates from the prediction due to ground bumps or the sheep suddenly change their direction of movement, the control system can correct the deviation in time in each cycle and achieve closed-loop trajectory tracking.

[0189] When performing herding tasks, all robots primarily track targets based on speed commands. Once the flock enters the grouping waiting area and the individual grouping decision module is activated, the collaborative execution module automatically switches the robots' working mode to grouping assistance mode. At this time, some robots continue to cruise at low speed within the waiting area according to speed commands, maintaining the flock's compact state. Other robots, according to the individual grouping strategy, move to designated positions on both sides of the grouping channel entrance, forming a moving barrier that leaves only the entrance to the currently activated channel open, guiding the sheep to enter the channel in sequence.

[0190] Example 2: Please see Figure 2 Based on Example 1, Example 2 of this application also provides a livestock breeding management method using reinforcement learning, including the following specific steps: Step 1: Acquire multi-source data and perform spatiotemporal alignment and point-by-point fusion to generate a fused point cloud; perform instance segmentation and motion feature-guided multi-level correlation tracking on the fused point cloud, and extract multi-dimensional individual features based on the multi-level correlation tracking; simultaneously construct a pasture-rasterized semantic map based on the fused point cloud. Step 2: Using multiple deployed robots as multiple agents, construct a state space by combining multi-dimensional individual features and semantic maps. Based on the state space, design an animal welfare risk index and a reward function with navigation safety as a weighted term. Through the multi-agent deep deterministic policy gradient algorithm, obtain a group driving strategy for driving livestock to the grouping area. Based on the group driving strategy, output the speed command of each robot. Step 3: After the livestock enter the grouping waiting area, the occupancy status of the grouping channel and the length of the waiting queue are collected. The body condition information estimated from visual recognition and the historical health records obtained from radio frequency identification are fused to construct an individual profile for each animal. Using the occupancy status of the grouping channel, the current individual profile and the length of the waiting queue as the state, and the allocation to different grouping channels as the action, the individual grouping strategy is obtained through dual deep Q network optimization. Step 4: Based on speed commands and individual grouping strategies, each robot performs real-time trajectory tracking through distributed model predictive control, while satisfying kinematic constraints and safe distance constraints between robots and between robots and livestock.

[0191] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0192] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0193] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A livestock breeding management system applying reinforcement learning, characterized in that, The system includes: The multi-source perception fusion module acquires multi-source data and performs spatiotemporal alignment and point-by-point fusion to generate a fused point cloud; it performs instance segmentation and motion feature-guided multi-level correlation tracking on the fused point cloud, extracts multi-dimensional individual features based on the multi-level correlation tracking; and it simultaneously constructs a pasture-grid semantic map based on the fused point cloud. The group driving decision module uses multiple deployed robots as multiple agents, constructs a state space by combining multi-dimensional individual features and semantic maps, designs an animal welfare risk index based on the state space, and a reward function with navigation safety as a weighted term. Through a multi-agent deep deterministic policy gradient algorithm, it obtains a group driving strategy for driving livestock to the grouping area, and outputs speed commands for each robot based on the group driving strategy. The individual grouping decision module collects the grouping channel occupancy status and waiting queue length after livestock enter the grouping waiting area. It integrates the body condition information estimated from visual recognition and the historical health records obtained from radio frequency identification to construct an individual profile for each livestock. Using the grouping channel occupancy status, the current individual profile, and the waiting queue length as the state, and the allocation to different grouping channels as the action, the module obtains the individual grouping strategy through dual deep Q-network optimization. In the collaborative execution and safety management module, each robot tracks its trajectory in real time through distributed model predictive control, based on speed commands and individual grouping strategies, while satisfying kinematic constraints and safe distance constraints between robots and between robots and livestock.

2. The livestock breeding management system applying reinforcement learning according to claim 1, characterized in that, The process of extracting multidimensional individual features is as follows: For each tracked animal, the position is predicted using a Kalman filter and strongly correlated with the current detection. For strongly correlated but unmatched items, weak correlation matching is performed by calculating Mahalanobis distance based on the movement features of the tracked animals, thus maintaining the continuity of the tracked animals' identities.

3. A livestock breeding management system applying reinforcement learning according to claim 2, characterized in that, The process of constructing a rasterized semantic map of the pasture based on the fused point cloud is as follows: Geometric feature point matching is used to estimate the pose of LiDAR frames and the resulting data is stitched together to generate a 3D point cloud map. The voxels in the 3D point cloud map are semantically labeled, with semantic labels including at least ground, fence, water source, and obstacle. The labeled 3D point cloud map is vertically projected into a 2D grid. The semantic label that appears most frequently in each grid is used as the dominant semantic label for that grid. The passability is calculated based on the proportion of ground points. Grids with a ground point proportion higher than a preset threshold are labeled as passable areas, forming a gridded semantic map.

4. A livestock breeding management system applying reinforcement learning according to claim 1, characterized in that, The process of constructing a state space from multidimensional individual features and semantic maps is as follows: The robot's state is defined by its relative distance and angle to the target area, as well as its current speed; the group state is defined by the distance and angle between the geometric center of the livestock herd and the robot, as well as the density distribution of the livestock herd in multiple sectors around the robot. The distances to obstacles obtained from multi-directional detection in the perimeter of the rasterized semantic map are used as the environmental perception state. Animal welfare risk index, calculated based on the degree of deviation of body surface temperature from the baseline and the proportion of individuals moving at high speed in the total number of livestock, is used as the welfare representation state, and the historical values ​​of the welfare risk index at multiple consecutive decision-making moments are used as the welfare risk trend state. The individual state, group state, environmental perception state, welfare representation state and welfare risk trend state are spliced ​​together to form a complete state space.

5. A livestock breeding management system applying reinforcement learning according to claim 1, characterized in that, Animal welfare risks include: Animal welfare risk items are quantified based on the welfare risk index, which is a weighted average of the deviation of each animal’s body surface temperature from the dynamic baseline temperature of the individual, and the proportion of individuals with high-speed movement exceeding the preset warning threshold in the total population. The navigation safety measures include: constructing a graded penalty based on the circumferential obstacle distance vector, applying incremental penalties to the number and degree of directions entering the danger zone and warning zone, and superimposing collision penalties based on the safety distance constraints between robots and between robots and livestock.

6. A livestock breeding management system applying reinforcement learning according to claim 5, characterized in that, The process of outputting the speed commands for each robot is as follows: Each robot acts as an independent intelligent agent, internally maintaining an actor network and a critic network, employing a centralized training and decentralized execution paradigm. During the training phase, it outputs continuous actions based on local observation states and superimposes exploration noise. After interacting with the environment, it stores joint observations, joint actions, and shared rewards in an experience replay buffer. It uses global joint states and joint actions to perform centralized value evaluation, and updates policy parameters with the obtained value gradient direction. The target network slowly approximates the online network parameters through soft updates. During the execution phase, it outputs a normalized velocity adjustment amount based on forward propagation of real-time local observation states. After inverse normalization, this is restored to a physical velocity command and superimposed on the current velocity. After smoothing and limiting, the command is sent to the robot's underlying motion controller.

7. A livestock breeding management system applying reinforcement learning according to claim 1, characterized in that, The process of creating an individual file for each animal is as follows: Based on the fusion of point cloud and instance segmentation mask, a 3D point set of the livestock back is extracted. After filtering, body size parameters are extracted. Weight is estimated by a pre-trained weight regression network. Body condition score is evaluated by a classification network based on the curvature features of the back point cloud and body size ratio. Identification codes are read by RFID ear tags. Historical health data, including feed intake, activity level, medical history and breeding records, are retrieved from the breeding management database. The real-time body condition data estimated by vision is timestamped and weighted with the historical records to generate a comprehensive health score. The tissues are labeled as individual profile vectors and synchronously written back to the database after grouping.

8. A livestock breeding management system applying reinforcement learning according to claim 7, characterized in that, The process of obtaining the individual grouping strategy through dual deep Q-network optimization is as follows: The state is constructed by combining the occupancy status of the grouping channel, the individual files of the livestock to be grouped, and the length of the waiting queue. The discrete action is to be assigned to different grouping channels. A dual-path structure of online network and target network is constructed. The online network selects the action and the target network evaluates the value. The reward function is a weighted sum of grouping accuracy, queue smoothness, and channel load balancing. After training converges, only the online network is used to output the channel allocation action during the deployment phase.

9. A livestock breeding management system applying reinforcement learning according to claim 1, characterized in that, The process of real-time trajectory tracking using distributed model predictive control is as follows: Each robot takes its current state, received speed commands, and individual grouping strategy as inputs, and constructs a finite-time domain optimization problem within the prediction time domain. The optimization objectives are speed tracking error and control increment smoothness, and the constraints are physical limit constraints, obstacle accessibility constraints based on semantic maps, inter-robot safety distance constraints based on the sharing of predicted trajectories of other robots, and safety distance constraints between robots and livestock. The optimization problem is solved online in a rolling manner, with only the first step of the control variable executed in each control cycle. During execution, the threshold of the animal and livestock safety distance constraints is applied based on the animal welfare risk index.

10. A livestock breeding management method applying reinforcement learning, applied to a livestock breeding management system applying reinforcement learning as described in any one of claims 1-9, characterized in that, The method includes: Step 1: Acquire multi-source data and perform spatiotemporal alignment and point-by-point fusion to generate a fused point cloud; perform instance segmentation and motion feature-guided multi-level correlation tracking on the fused point cloud, and extract multi-dimensional individual features based on the multi-level correlation tracking; simultaneously construct a pasture-rasterized semantic map based on the fused point cloud. Step 2: Using multiple deployed robots as multiple agents, construct a state space by combining multi-dimensional individual features and semantic maps. Based on the state space, design an animal welfare risk index and a reward function with navigation safety as a weighted term. Through the multi-agent deep deterministic policy gradient algorithm, obtain a group driving strategy for driving livestock to the grouping area. Based on the group driving strategy, output the speed command of each robot. Step 3: After the livestock enter the grouping waiting area, the occupancy status of the grouping channel and the length of the waiting queue are collected. The body condition information estimated from visual recognition and the historical health records obtained from radio frequency identification are fused to construct an individual profile for each animal. Using the occupancy status of the grouping channel, the current individual profile and the length of the waiting queue as the state, and the allocation to different grouping channels as the action, the individual grouping strategy is obtained through dual deep Q network optimization. Step 4: Based on speed commands and individual grouping strategies, each robot performs real-time trajectory tracking through distributed model predictive control, while satisfying kinematic constraints and safe distance constraints between robots and between robots and livestock.

Images