A Method and System for Identifying Abnormal Behavioral Features of Laying Hens Based on Depth Images

Abstract

This invention discloses a method and system for identifying abnormal behavior features in laying hens based on depth images, belonging to the field of intelligent identification technology in livestock and poultry farming. The method includes acquiring a sequence of depth images of laying hens in cages within a preset time period; preprocessing the image sequences and extracting the torso region of the laying hens to generate a sequence of depth maps showing the hens' postures. An improved temporal segmentation algorithm is used to process the image sequences, and the segmentation boundaries are optimized based on the similarity of the motion trajectories of key points in the skeleton, resulting in a preliminary set of behavioral segments with different behavioral semantics. The spatiotemporal joint behavioral features of the depth image subsequences corresponding to each behavioral segment are extracted. These features are input into a pre-trained abnormal behavior recognition model, and the probability of each behavioral segment belonging to different abnormal behavior categories is calculated. Based on the probability, the behavior category is determined, and the abnormal behavior recognition result, including timestamp, behavior category, and confidence level, is output. This method is adapted to cage-raised laying hen farming scenarios, optimizing the accuracy of temporal behavior segmentation and the representation effect of behavioral features.

Description

Technical Field

[0001] This invention belongs to the field of intelligent identification technology in livestock and poultry farming, specifically a method and system for identifying abnormal behavioral features of laying hens based on depth images. Background Technology

[0002] Large-scale cage rearing has become the mainstream method for egg-laying hen farming. Abnormal behavior of laying hens during the rearing process is an important indicator of their growth and health. Currently, the industry mostly relies on visible light images to capture images of laying hen farming, combined with ordinary image segmentation methods and fixed-interval time-domain division to process image sequences, relying on basic visual features to complete the initial identification of laying hen behavior. Conventional processing methods only rely on pixel grayscale differences to complete image segmentation, and the behavior time-domain division uses a uniform fixed frame interval method, without dynamically adjusting it based on the target's own motion characteristics.

[0003] Visible light imaging is highly susceptible to interference from external factors such as changes in lighting in poultry sheds, cage obstruction, and shadows cast by debris, making it difficult to reliably isolate the effective area of ​​the hen's torso and resulting in insufficient completeness of posture information extraction. Fixed-interval temporal segmentation methods cannot adapt to the varying speeds of hen behavior, leading to discrepancies between the behavioral semantic segmentation boundaries and actual action switching nodes, easily resulting in misaligned behavioral segments and semantically mixed segments. Single-dimensional feature extraction modes can only capture static morphology or single-frame temporal changes, failing to correlate with dynamic behavioral evolution patterns, and their feature representation dimensions are limited, making it difficult to accurately distinguish various types of abnormal hen behaviors. Summary of the Invention

[0004] This invention aims to solve at least one of the technical problems existing in the prior art; Therefore, this invention proposes a method for identifying abnormal behavioral features of laying hens based on depth images, including: Obtain depth image sequences of laying hens in target cages within a preset time period; The depth image sequence is preprocessed to extract the torso region of the laying hen and generate a depth map sequence of laying hen postures. An improved temporal segmentation algorithm is applied to the depth map sequence of laying hens to divide it into continuous time periods with different behavioral semantics, resulting in a preliminary set of behavioral segments. The improved temporal segmentation algorithm optimizes the boundaries based on the similarity measure of the motion trajectories of the skeleton key points in the depth image sequence. For each preliminary behavioral segment in the set of preliminary behavioral segments, spatiotemporal joint behavioral features are extracted based on its corresponding depth image subsequence; The spatiotemporal joint behavioral features are input into a pre-trained abnormal behavior recognition model to calculate the probability that the preliminary behavioral fragment belongs to different abnormal behavior categories. Based on the probability, the behavior category corresponding to the preliminary behavior segment is determined, and the abnormal behavior identification result containing timestamp, behavior category and confidence level is output.

[0005] Further, the depth image sequence is preprocessed to extract the torso region of the laying hen, generating a laying hen pose depth map sequence, including: For each frame of the depth image sequence, a background modeling and foreground segmentation algorithm is applied to separate the foreground region of the laying hen from the background environment; Cluster analysis was performed on the foreground area of ​​the laying hens based on depth values ​​to distinguish the laying hen bodies from static objects such as cage railings, feed troughs, and nipple drinkers. Using a 3D skeleton estimation algorithm, key points are extracted from the segmented point cloud of the laying hen's body to locate the head, neck, back, torso, wings, legs, and claws in 3D space. Centered on the key points of the hen's torso, a three-dimensional bounding box is drawn on the original depth image, and the depth values ​​within the three-dimensional bounding box are resampled into a fixed-size depth image block; The sequence of depth image blocks arranged in chronological order is used as the depth map sequence of the laying hen posture.

[0006] Furthermore, an improved temporal segmentation algorithm is applied to the hen pose depth map sequence to divide it into continuous time periods with different behavioral semantics, resulting in a preliminary set of behavioral segments, including: Extract consecutive depth image frames from the sequence of egg-laying hen posture depth maps in chronological order; For each frame of depth image, the skeleton pose vector of the current frame is calculated based on the 3D key point information it contains, and the skeleton pose vectors of all time frames constitute the pose trajectory. The improved temporal segmentation algorithm is used to analyze the attitude trajectory. The improved temporal segmentation algorithm calculates the local similarity matrix of the attitude trajectory and detects the boundary points of similarity abrupt changes along the diagonal direction in the local similarity matrix. After detecting the boundary point, the improved temporal segmentation algorithm performs fine-tuning of the boundary position based on the temporal smoothing constraint of the poses on both sides of the boundary point, ensuring that the pose changes within the behavior segment are continuous and that there are significant differences in pose between segments. The time interval between adjacent boundary points is marked as a preliminary behavior segment, and all preliminary behavior segments constitute the set of preliminary behavior segments.

[0007] Furthermore, the improved temporal segmentation algorithm optimizes the boundary based on a similarity metric of the motion trajectories of skeleton keypoints in the depth image sequence, including: Calculate the three-dimensional motion trajectory of a specific set of key points in consecutive time frames from the sequence of egg-laying hen posture depth maps; Define a metric function to measure the similarity between two short-term trajectories, which combines the Euclidean distance of the trajectory shapes with the cosine similarity of the motion directions; Slide a fixed-length time window on the attitude trajectory and calculate the similarity metric between the sub-trajectory within the window and the sub-trajectory of the adjacent window; The similarity measure value is used as a change curve over time. Local minimum points are detected on the change curve, and these local minimum points are used as preliminary potential boundaries. For each initial potential boundary, the similarity metric is recalculated within its neighborhood using more refined trajectory sampling. The boundary points are then adjusted to the precise positions where the similarity metric value is minimized, thus completing the boundary optimization.

[0008] Furthermore, for each preliminary behavioral segment in the set of preliminary behavioral segments, spatiotemporal joint behavioral features are extracted based on its corresponding depth image subsequence, including: For the depth image subsequence corresponding to the initial behavior segment, the displacement field of the three-dimensional key points between consecutive frames is calculated to obtain the dense optical flow field sequence; The depth image subsequence is subjected to three-dimensional voxelization processing, and the depth value change of each voxel grid on the time axis is constructed into a spatiotemporal cube; The spatiotemporal cube is subjected to feature extraction using a three-dimensional convolutional neural network to obtain a low-level feature map representing the local spatiotemporal pattern; The dense optical flow field sequence is processed using a graph convolutional network. The key points of the skeletal structure of the laying hen are used as graph nodes, and the skeleton connections and inter-frame correspondences are used as graph edges. The spatiotemporal dependencies between nodes are learned, and spatiotemporal features based on the skeleton graph are output. The low-level feature map is fused and pooled with the spatiotemporal features based on the skeleton graph to generate a fixed-dimensional spatiotemporal joint behavioral feature vector.

[0009] Furthermore, the dense optical flow field sequence is processed using a graph convolutional network. Key points of the hen's skeleton are used as graph nodes, and the skeleton connections and inter-frame correspondences are used as graph edges. The spatiotemporal dependencies between nodes are learned, and spatiotemporal features based on the skeleton graph are output, including: A spatiotemporal graph is constructed for the initial behavioral segment, and the node set of the spatiotemporal graph consists of all skeletal key points on all time frames; The spatiotemporal graph has spatial edges and temporal edges. The spatial edges connect pairs of key points with physiological connections within the same frame, and the temporal edges connect the same key point in adjacent frames. Each node's features consist of its three-dimensional coordinates, depth value, and motion vector at that point extracted from the dense optical flow field. The constructed spatiotemporal graph is input into a multi-layer graph convolutional network, which aggregates neighborhood node information along the spatial edge in the spatial dimension and aggregates continuous frame information along the temporal edge in the temporal dimension. After multiple layers of information transmission, the features of all nodes output from the last layer are globally pooled to obtain the spatiotemporal features based on the skeleton graph.

[0010] Further, the spatiotemporal joint behavioral features are input into a pre-trained abnormal behavior recognition model to calculate the probability that the preliminary behavioral fragment belongs to different abnormal behavior categories, including: The pre-trained abnormal behavior recognition model includes a feature encoder and a classifier; The feature encoder receives the spatiotemporal joint behavior feature vector and converts it into a high-dimensional hidden layer representation through multi-layer nonlinear mapping. The classifier receives the high-dimensional hidden layer representation output by the feature encoder and calculates its score for each predefined abnormal behavior category, which includes vent pecking, feather pecking, fighting, stereotyped behavior, lethargy, and abnormal appetite. Applying a normalized exponential function to the scores converts them into probability values, yielding the probability distribution of the preliminary behavioral fragments belonging to different abnormal behavior categories.

[0011] Furthermore, the pre-trained abnormal behavior recognition model is trained through the following steps: We collected a large number of labeled deep image sequence samples of laying hen behavior, with each sample labeled with the abnormal behavior category and the corresponding time period; For each training sample, the following steps are taken: acquiring depth image sequences, preprocessing to generate pose depth map sequences, performing improved temporal segmentation to obtain preliminary behavioral fragments, and extracting spatiotemporal joint behavioral features to obtain the corresponding spatiotemporal joint behavioral feature vector and category label. The initial neural network model containing the feature encoder and classifier is trained using labeled training samples; During training, the model parameters are optimized using the backpropagation algorithm to minimize the difference between the class probability distribution output by the model and the true label. When the model's recognition accuracy on the independent validation set no longer improves, training is stopped, and the pre-trained abnormal behavior recognition model is obtained.

[0012] Further, based on the probability, the behavior category corresponding to the initial behavior fragment is determined, and an abnormal behavior identification result including a timestamp, behavior category, and confidence level is output, including: A probability threshold is preset for each category of abnormal behavior; The probability that the preliminary behavioral fragment belongs to a certain abnormal behavior category is compared with the corresponding probability threshold; If the probability is greater than or equal to the probability threshold, then the preliminary behavioral fragment is determined to belong to the abnormal behavior category, and the probability value is used as the confidence level. If the probability is less than the probability threshold of all abnormal behavior categories, then the initial behavior segment is determined to be normal behavior; For the initial behavioral segments identified as abnormal behavior, record their start timestamp, end timestamp, the identified behavior category, and the corresponding confidence level to form the structured abnormal behavior identification result.

[0013] Furthermore, the present invention also includes a depth image-based system for identifying abnormal behavior features of laying hens, the system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the steps of the depth image-based method for identifying abnormal behavior features of laying hens as described above.

[0014] Compared with the prior art, the beneficial effects of the present invention are: By optimizing the temporal segmentation boundary based on the similarity measurement of the motion trajectories of key points in the skeleton, the processing method of mechanically dividing behavior periods at fixed frame intervals is changed. Instead, the changes in the limb movements of the laying hen itself are used as the basis for temporal segmentation. In the breeding scenario, cage structure occlusion and ambient light fluctuations do not interfere with the temporal segmentation logic. The segmentation nodes of behavior segments can fit the actual switching time of the behavior actions, reducing the semantic confusion caused by the boundary offset of behavior segments. The temporal segmentation process can adaptively match the behavior changes of different action rhythms. The segmented behavior segments all have independent and complete behavioral semantics, and the regularity of temporal data is improved. Subsequent feature analysis can rely on the accurately segmented behavior segments to carry out temporal information mining.

[0015] This method extracts spatiotemporal joint behavioral features from deep image subsequences, simultaneously integrating spatial body region structural information with temporal continuous motion change information, thus broadening the information coverage dimensions of behavioral features. It departs from extraction modes based on single static or single temporal features, fully preserving the intrinsic correlation information in the continuous evolution of behavior and enriching the hierarchical structure of feature expression. The inherent attributes of different types of abnormal behaviors can be differentiated through multidimensional features, strengthening the distinguishing attributes of behavioral features and adapting to the feature representation needs of diverse behavioral states in complex farming environments. The behavior recognition model can rely on refined spatiotemporal joint features to complete category probability calculations, adapting to the normalized behavior analysis scenarios of laying hens in caged environments, and achieving stable parsing of behavioral information in continuous image sequences. Attached Figure Description

[0016] Figure 1This is a flowchart of the method for identifying abnormal behavior features of laying hens based on depth images according to the present invention; Figure 2 A flowchart for preprocessing and generating a sequence of laying hen pose depth maps; Figure 3 This is a flowchart for extracting behavioral features of laying hens and performing spatiotemporal graph convolution processing. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 This invention provides a method for identifying abnormal behavioral features of laying hens based on depth images. The specific method includes: By acquiring depth image sequences of laying hens in target cages within a preset time period, the depth image sequences are preprocessed to extract the torso region of the hens and generate a sequence of hen posture depth maps. An improved temporal segmentation algorithm is then applied to the hen posture depth map sequence to divide it into continuous time periods with different behavioral semantics, thus obtaining a preliminary set of behavioral segments. This improved temporal segmentation algorithm optimizes the boundaries based on the similarity measure of the motion trajectories of skeletal key points in the depth image sequence. For each preliminary behavioral segment in the preliminary behavioral segment set, spatiotemporal joint behavioral features are extracted based on its corresponding depth image subsequence. These spatiotemporal joint behavioral features are input into a pre-trained abnormal behavior recognition model to calculate the probability that the preliminary behavioral segment belongs to different abnormal behavior categories. Based on this probability, the behavior category corresponding to the preliminary behavioral segment is determined, and an abnormal behavior recognition result including timestamp, behavior category, and confidence level is output.

[0019] In one embodiment of the present invention, the depth image sequence is preprocessed to extract the torso region of the laying hen and generate a laying hen pose depth map sequence, see reference. Figure 2 The process includes applying background modeling and foreground segmentation algorithms to each frame of the depth image sequence to separate the foreground region of the laying hen from the background environment; performing cluster analysis within the foreground region of the laying hen based on depth values ​​to distinguish the laying hen's body from static objects such as cage railings, feed troughs, and nipple drinkers; using a 3D skeleton estimation algorithm to extract key points from the segmented point cloud of the laying hen's body to locate the head, neck, back, torso, wings, legs, and claws in 3D space; using the key points of the laying hen's torso as the center to delineate a 3D bounding box on the original depth image, and resampling the depth values ​​within the 3D bounding box into fixed-size depth image blocks; and using the sequence of depth image blocks arranged in chronological order as the laying hen pose depth map sequence.

[0020] In practice, depth image sequences of laying hens within a preset time period are acquired from a depth camera installed directly above each cage in a multi-layered cage-raising hen house. The depth camera continuously acquires images at a fixed frame rate, and its field of view completely covers the interior area of ​​the target cage. Each frame of the depth image records the distance values ​​from each point in the cage scene to the camera plane. Background modeling and foreground segmentation algorithms are applied to each frame of the depth image sequence. The background model is initialized by taking the median value of multiple frames of static scene depth images and updated with a learning rate during the acquisition process. For the current frame of the depth image, the difference between the depth value and the background model is calculated pixel by pixel. When the difference exceeds a preset threshold, the pixel is marked as the foreground area of ​​the laying hen. The foreground segmentation determination criteria are as follows: in: This represents the depth image in frame t at pixel coordinates. Depth value at that location, This represents the depth value of the background model at the same coordinate position. As the segmentation threshold, when Pixels with a depth of 1 are assigned to the foreground region of the laying hen. After obtaining the foreground region, clustering analysis is performed within it based on depth values. Specifically, a clustering method using a weighted average of depth values ​​and pixel planar coordinates in Euclidean space is employed. Pixels that are spatially adjacent and have the same depth value are grouped into several clusters. Based on the geometric shape, location range, and depth distribution characteristics of each cluster, clusters of laying hen bodies are distinguished from clusters of cage railings, feed troughs, and nipple drinkers. Static object clusters corresponding to cage railings, feed troughs, and nipple drinkers are removed from the foreground, retaining only the point cloud region corresponding to the laying hen body. In some embodiments, the initial clustering... Initially, several seed points are set based on the known layout of static objects within the breeding cage to accelerate convergence. A 3D skeleton estimation algorithm is used to extract key points from the segmented point cloud of the laying hen's body. This algorithm employs a model trained on a deep neural network, taking normalized point cloud data as input and outputting the coordinates of the hen's head, neck, back, torso, wing, leg, and claw key points in 3D space. After extracting all key points, a 3D bounding box is drawn on the original depth image, centered on the hen's torso key points. The size of the 3D bounding box is based on the egg... Typical body dimensions for chicken breeds are predefined, with length, width, and height covering the axial, lateral, and dorsal / ventral regions of the torso, respectively. Depth values ​​within the 3D bounding box are resampled into fixed-size depth image patches using bilinear interpolation. In some embodiments, the fixed size is set to 64×64 pixels to balance the precision of pose representation with computational overhead. It is understood that, due to the fixed perspective of the depth camera, key points on the torso, acting as body centers, can stably frame the body region containing pose information. The sequence of depth image patches arranged sequentially over time is directly used as the egg-laying hen pose depth map sequence. Each element in the egg-laying hen pose depth map sequence... The depth image blocks retain only the depth information of the hen's torso and adjacent body parts, eliminating interference from cage railings, feed troughs, and nipple drinkers. It can be understood that the temporal continuity of the hen posture depth map sequence allows subsequent behavioral segmentation to accurately reflect the temporal changes in hen posture. Optionally, when generating the hen posture depth map sequence, depth values ​​of each frame's depth image block are normalized to map the depth values ​​to a standard range to eliminate individual and locational differences. Optionally, for sudden depth loss pixels appearing in several consecutive frames, the temporal median is used to fill in the gaps, maintaining the integrity of the depth image block sequence.

[0021] In one embodiment of the present invention, an improved temporal segmentation algorithm is performed on the hen posture depth map sequence to obtain a preliminary set of behavioral segments. This includes extracting consecutive depth image frames from the hen posture depth map sequence in chronological order, calculating the skeleton pose vector of each depth image frame based on its contained 3D keypoint information, and the skeleton pose vectors of all time frames constitute the pose trajectory. The improved temporal segmentation algorithm is used to analyze the pose trajectory, calculate the local similarity matrix of the pose trajectory, and detect boundary points with abrupt similarity changes along the diagonal direction in the local similarity matrix. After detecting the boundary points, the boundary positions are fine-tuned according to the temporal smoothing constraints of the poses on both sides of the boundary points, so that the pose changes within the behavioral segments are continuous, while there are significant differences in pose between segments. Meanwhile, the improved temporal segmentation algorithm optimizes the boundaries based on the similarity metric of the motion trajectories of skeleton keypoints in the depth image sequence. Specifically, it calculates the 3D motion trajectories of a specific set of keypoints in consecutive time frames from the chicken pose depth map sequence, defines a metric function to measure the similarity between two short-term trajectories, and combines the Euclidean distance of the trajectory shape with the cosine similarity of the motion direction. A fixed-length time window is slid across the pose trajectory, and the similarity metric between the sub-trajectory within the window and the sub-trajectory of the adjacent window is calculated. The sequence of this similarity metric over time is used as a change curve, and local minima are detected on the change curve. These local minima are used as preliminary potential boundaries. For each preliminary potential boundary, the similarity metric is recalculated in its neighborhood using finer trajectory sampling, and the boundary points are adjusted to the precise position with the minimum similarity metric value, thus completing the boundary optimization. The time interval between adjacent boundary points is marked as a preliminary behavior segment, and all preliminary behavior segments constitute a set of preliminary behavior segments.

[0022] In practice, continuous depth image frames are extracted sequentially from the hen posture depth map sequence. Each depth image frame corresponds to a set of 3D coordinates of the hen's head, neck, back, torso, wing, leg, and claw keypoints output by a 3D skeleton estimation algorithm. For each depth image frame, the skeleton posture vector of the current frame is calculated based on the 3D keypoint information it contains. The skeleton posture vector is constructed by taking the torso keypoint coordinates as the reference origin, subtracting the 3D coordinates of all other keypoints from the torso keypoint coordinates to obtain the offset vector of each keypoint relative to the torso, and then concatenating these offset vectors into a high-dimensional vector according to the fixed anatomical order of head, neck, back, wings, legs, and claws. This high-dimensional vector serves as the skeleton posture vector of the current frame. The skeleton posture vectors of all time frames are arranged in chronological order to form the posture trajectory. It can be understood that the posture trajectory represents the posture evolution of various body parts relative to the torso during the continuous movement of the hen in the time domain. When the hen's behavior changes, the local statistical characteristics of the posture trajectory will produce detectable abrupt changes. An improved temporal segmentation algorithm is used to analyze the attitude trajectory. First, the local similarity matrix of the attitude trajectory is calculated. The element in the i-th row and j-th column of the local similarity matrix is ​​the cosine similarity between the skeleton attitude vector of the i-th frame and the skeleton attitude vector of the j-th frame. A square detection window of a fixed size slides along the main diagonal of the local similarity matrix. The sum of the singular values ​​of the sub-matrices within the window is calculated window by window as an indicator of local similarity. When the sliding window crosses the action semantic transition time, the sum of the singular values ​​of the sub-matrices within the window decreases significantly, thereby detecting the boundary points of the similarity abrupt change. After detecting the boundary point, the boundary position is fine-tuned based on the temporal smoothing constraints of the poses on both sides of the boundary point. Specifically, a neighboring frame interval is taken as the center of the initially detected boundary point. Within the neighboring frame interval, the boundary point is attempted to be translated position by position with a single frame step size. The sum of the average inter-frame change amplitude of the skeleton pose vector in the segment to the left of the boundary point and the average inter-frame change amplitude of the skeleton pose vector in the segment to the right of the boundary point is calculated under each translation. The translation position corresponding to the minimum amplitude is taken as the fine-tuned boundary point. This makes the pose change within the behavior segment continuous, while the pose difference between adjacent behavior segments is obvious.

[0023] In its implementation, the improved temporal segmentation algorithm further optimizes the boundaries based on the similarity metric of the motion trajectories of skeletal keypoints in the depth image sequence, thereby enhancing the accuracy of boundary localization. From the hen pose depth map sequence, a specific set of keypoints is selected, including head, back, and torso keypoints. The 3D positions of these keypoints are tracked across consecutive time frames, forming 3D motion trajectories for the head, back, and torso keypoints, respectively. Before calculating the similarity metric, the 3D coordinates of each sampling point in the head, back, and torso keypoint trajectories are normalized. This normalization involves dividing each component of the keypoint's 3D coordinates at each moment by the length of the hen's torso, defined as the Euclidean distance between the torso and back keypoints in the same frame. After normalization, the spatial coordinates of all keypoints are converted to dimensionless scale values, and all subsequent distance calculations are performed in this normalized coordinate space. Define the metric function for measuring the similarity between two short-term trajectories as follows: in: Represents two short-time trajectories and Similarity metric, This represents the number of time sampling points contained within the short-time trajectory. To balance the weighting coefficients of the Euclidean distance term for trajectory shape and the cosine similarity term for motion direction, and Trajectories and trajectory In the The normalized three-dimensional spatial coordinates of each sampling time point and Trajectories and trajectory In the The motion direction vector at each sampling time is obtained by normalizing the position difference between adjacent sampling points.

[0024] In the specific implementation, a fixed-length time window is slid along the combined posture motion trajectory formed by stitching together the 3D motion trajectories of the head keypoints, back keypoints, and torso keypoints according to time alignment. The window length corresponds to the motion duration of N consecutive frames. For each sliding position, a sub-trajectory within the window and its adjacent sub-trajectory within the next window of the same length are considered as a trajectory pair. A similarity metric function is used to calculate the similarity metric value between the two sub-trajectories. This process is repeated window by window along the time axis to obtain a sequence of similarity metric values ​​over time, which serves as a change curve. On the change curve, all local minima are detected using the rule that the first-order difference is zero-crossing and the second-order difference is positive. These local minima are used as preliminary potential boundaries. For each preliminary potential boundary, a series of candidate boundary positions are generated within the temporal neighborhood of the preliminary potential boundary with a finer time sampling step. The similarity metric value of the two sub-trajectories before and after each candidate boundary position is recalculated. The candidate boundary position with the smallest similarity metric value is determined as the precise boundary, thus completing boundary optimization. The time interval between adjacent precise boundaries is labeled as a preliminary action segment, and all preliminary action segments constitute a set of preliminary action segments. In some embodiments, weighting coefficients... The value is set to 0.6 to give greater weight to the normalized Euclidean distance term of the trajectory shape, thus focusing more on spatial position changes. In some embodiments, the width of the fixed-length time window is set to 20 frames, corresponding to approximately 1 second of hen motion segments, so that the similarity metric can capture fine-grained changes in behavior transitions. Optionally, non-maximum suppression is used during the detection of boundary points on the diagonal of the local similarity matrix to avoid detecting overly dense false boundaries in neighboring locations. Optionally, when optimizing boundaries based on keypoint motion trajectory similarity metrics, the joint similarity of the head keypoint 3D motion trajectory, the back keypoint 3D motion trajectory, and the torso keypoint 3D motion trajectory is considered simultaneously to enhance the robustness of boundary segmentation in complex posture change scenarios. It can be understood that the combination of boundary detection through local similarity of posture trajectories and boundary optimization based on normalized skeleton keypoint motion trajectory similarity metrics can accurately divide continuous hen posture depth map sequences into continuous time periods with different behavioral semantics, thereby generating a high-quality preliminary set of behavioral segments.

[0025] In one embodiment of the present invention, spatiotemporal joint behavioral features are extracted for each preliminary behavioral segment in the preliminary behavioral segment set based on its corresponding depth image subsequence, see [reference]. Figure 3This includes calculating the displacement field of three-dimensional key points between consecutive frames for the depth image subsequence corresponding to the initial behavioral segment, and obtaining a dense optical flow field sequence; performing three-dimensional voxelization processing on the depth image subsequence, and constructing a spatiotemporal cube by the depth value change of each voxel grid on the time axis; and using a three-dimensional convolutional neural network to extract features from the spatiotemporal cube to obtain a low-level feature map representing the local spatiotemporal pattern. The process of processing dense optical flow field sequences using graph convolutional networks is as follows: A spatiotemporal graph is constructed for the initial behavioral segments. The node set of the spatiotemporal graph consists of all skeleton keypoints across all time frames. The edges of the spatiotemporal graph include spatial and temporal edges. Spatial edges connect paired keypoints with physiological connections within the same frame, while temporal edges connect the same keypoint in adjacent frames. The features of each node consist of its 3D coordinates, depth value, and the motion vector extracted from the dense optical flow field at that point. The constructed spatiotemporal graph is input into a multi-layer graph convolutional network. Neighborhood node information is aggregated along spatial edges in the spatial dimension, and continuous frame information is aggregated along temporal edges in the temporal dimension. After multi-layer information transfer, all node features output from the last layer are globally pooled to obtain spatiotemporal features based on the skeleton graph. The low-level feature map and the spatiotemporal features based on the skeleton graph are then fused and pooled to generate a fixed-dimensional spatiotemporal joint behavioral feature vector.

[0026] In specific implementation, for each preliminary behavioral segment in the preliminary behavioral segment set, a depth image subsequence corresponding to that preliminary behavioral segment is obtained. The depth image subsequence consists of continuous depth image frames extracted from the chicken posture depth map sequence within the start and end timestamp range of that preliminary behavioral segment. When calculating the displacement field of 3D keypoints between consecutive frames, for each pair of adjacent depth image frames in the depth image subsequence, using the set of 3D keypoints that have been successfully located in both the previous and next frames, the 3D displacement vector is calculated for each keypoint. The 3D displacement vector is the difference between the 3D coordinates of the keypoint in the next frame and the 3D coordinates of the corresponding keypoint in the previous frame. The 3D displacement vectors of all keypoints are summarized to form the sparse motion field of the adjacent frame pair. Radial basis function interpolation is performed on the sparse motion field in the spatial dimension to fill the blank areas between keypoints, thereby obtaining a dense optical flow field sequence. Each frame of the dense optical flow field sequence corresponds one-to-one with each frame in the depth image subsequence except for the last frame. When performing 3D voxelization on depth image subsequences, the depth image subsequences are stacked along the time axis to form a 3D data volume. In the spatial dimension, each frame of depth image is uniformly divided into grids, with the grid cell size adapted to the size of the laying hen. In the temporal dimension, the depth image frame number is used as the third coordinate. The average depth value of several consecutive depth image frames covered by each voxel grid on the time axis is taken and filled into the voxel grid to construct a spatiotemporal cube. It can be understood that the spatiotemporal cube simultaneously preserves the spatial depth distribution pattern of the laying hen's posture and the temporal evolution sequence. A three-dimensional convolutional neural network is used to extract features from a spatiotemporal cube. The three-dimensional convolutional neural network contains alternating three-dimensional convolutional layers, three-dimensional batch normalization layers, and three-dimensional pooling layers. The three-dimensional convolutional kernel slides simultaneously in two spatial dimensions and one temporal dimension to capture the pattern changes of the chicken's posture in the local spatiotemporal neighborhood. After multiple layers of three-dimensional convolution and three-dimensional pooling operations, the feature tensor output by the top layer is compressed in space dimension through global average pooling to obtain a low-level feature map representing the local spatiotemporal pattern. The low-level feature map is a feature vector with a fixed dimension.

[0027] In the specific implementation, a graph convolutional network is used to process the dense optical flow field sequence to construct a spatiotemporal graph for the initial behavioral segment. The node set of the spatiotemporal graph consists of all skeleton keypoints on all time frames in the depth image subsequence, and each node uniquely corresponds to a keypoint in a certain frame. The edges of the spatiotemporal graph include two types: spatial edges and temporal edges. Spatial edges connect pairs of keypoints with physiological connections within the same frame. Specifically, the head keypoint is connected to the neck keypoint, the neck keypoint is connected to the back keypoint, the back keypoint is connected to the torso keypoint, the torso keypoint is connected to the keypoints of both wings respectively, the torso keypoint is connected to the keypoints of both legs respectively, and the leg keypoint is connected to the claw keypoint. Temporal edges connect nodes representing the same keypoint type in adjacent frames. Specifically, the head keypoint node in frame t is connected to the head keypoint node in frame t+1, and so on for other keypoint types. Each node's features consist of its keypoint 3D coordinates, keypoint depth value, and motion vector at the corresponding keypoint extracted from the dense optical flow field sequence. The motion vector is directly sampled from the corresponding spatial position of the corresponding frame in the dense optical flow field sequence. Thus, each node obtains an initial feature vector that integrates spatial position information, depth information, and motion information.

[0028] In the specific implementation, the constructed spatiotemporal graph is input into a multi-layer graph convolutional network, which contains multiple spatial graph convolutional layers and multiple temporal graph convolutional layers stacked sequentially. In each spatial graph convolutional layer, for nodes within the same frame in the spatiotemporal graph, neighboring node information is aggregated along the spatial edges. The aggregation function is a weighted sum of the current node's features and the features of all spatially adjacent nodes, with the weights calculated by a normalized exponential function based on the feature similarity between the current node and its neighboring nodes. In each temporal graph convolutional layer, information transmitted by nodes of the same keypoint type in consecutive frames is aggregated along the temporal edges. A gated recurrent unit structure is used to memorize and update the node features of historical time frames, enabling the current frame node to incorporate the motion history information of the same keypoint from several past frames. After multi-layer information transmission, the top layer of the multi-layer graph convolutional network outputs the deep features of all nodes across all time frames. Global pooling is performed on all node features, taking the mean and maximum values ​​along the node dimension. The mean pooling vector and the maximum pooling vector are concatenated into a single overall feature vector, serving as the spatiotemporal feature based on the skeleton graph.

[0029] In practical implementation, the process of fusing and pooling low-level feature maps with spatiotemporal features based on skeleton maps is as follows: in: This represents a low-level feature map extracted by a 3D convolutional neural network. This represents the spatiotemporal features based on the skeleton graph output by a multi-layer graph convolutional network. Concat represents the operation of concatenating two feature vectors end-to-end along the feature channel dimension. The weight matrix is ​​a learnable linear transformation. For bias terms, The fused feature vector is then subjected to global average pooling to map it to a fixed-dimensional spatiotemporal joint behavior feature vector. In some embodiments, the kernel size of the 3D convolutional neural network is set to 3×3×3, and the number of spatial and temporal convolutional layers in the spatiotemporal graph convolution is set to 3 to achieve a balance between feature abstraction levels and computational complexity. In some embodiments, the voxel grid of the spatiotemporal cube is divided into 32×32 grids in the spatial dimension and fixed to a length of 16 frames in the temporal dimension. Multiple spatiotemporal cubes are extracted from a longer preliminary behavior segment using a sliding window approach, and the average value is taken. Optionally, jump spatial edges are added when constructing the spatiotemporal graph. These jump spatial edges connect distant keypoint pairs that are not directly physiologically connected but move in coordination in specific behaviors to capture cross-body region motion coordination patterns. Optionally, when extracting motion vectors from dense optical flow field sequences, the direction of the motion vectors is quantized and encoded to enhance the discriminative power of motion direction information. It is understandable that by complementing and fusing the dense spatiotemporal local patterns captured by the 3D convolutional neural network with the spatiotemporal dependencies of the skeleton explicitly modeled by the graph convolutional network, the generated spatiotemporal joint behavioral feature vector can simultaneously and completely represent the behavior of laying hens from both the perspectives of appearance depth changes and skeleton motion structure.

[0030] In one embodiment of the present invention, spatiotemporal joint behavioral features are input into a pre-trained abnormal behavior recognition model to calculate the probability that a preliminary behavioral fragment belongs to different abnormal behavior categories. The pre-trained abnormal behavior recognition model includes a feature encoder and a classifier. The feature encoder receives spatiotemporal joint behavioral feature vectors and converts them into high-dimensional hidden layer representations through multi-layer nonlinear mapping. The classifier receives the high-dimensional hidden layer representations output by the feature encoder and calculates the score of each predefined abnormal behavior category. The predefined abnormal behavior categories include vent pecking, feather pecking, fighting, stereotyped behavior, lethargy, and abnormal appetite. A normalized exponential function is applied to the scores to convert them into probability values, thereby obtaining the probability distribution of the preliminary behavioral fragment belonging to different abnormal behavior categories. The training steps of this pre-trained abnormal behavior recognition model include: collecting a large number of labeled depth image sequence samples of laying hen behavior, each sample being labeled with an abnormal behavior category and the corresponding time period; for each training sample, processing is performed according to the method of acquiring depth image sequence, preprocessing to generate pose depth map sequence, performing improved temporal domain segmentation to obtain preliminary behavior fragments, and extracting spatiotemporal joint behavior features to obtain the corresponding spatiotemporal joint behavior feature vector and category label; using labeled training samples to train an initial neural network model containing a feature encoder and classifier; during training, the model parameters are optimized through the backpropagation algorithm to minimize the difference between the class probability distribution output by the model and the true label; when the recognition accuracy of the model on the independent validation set no longer improves, training is stopped, and the pre-trained abnormal behavior recognition model is obtained.

[0031] In practice, the spatiotemporal joint behavior feature vector is input into a pre-trained abnormal behavior recognition model. This model comprises a feature encoder and a classifier. The feature encoder receives the spatiotemporal joint behavior feature vector and consists of four stacked fully connected layers. A modified linear unit (MLU) is inserted between each pair of fully connected layers as a non-linear activation function. The number of neurons in the fully connected layers is set to 1024, 2048, 4096, and 2048, respectively. Through multi-layer non-linear mapping, the spatiotemporal joint behavior feature vector is converted into a high-dimensional hidden layer representation with a dimension of 2048. The classifier receives the high-dimensional hidden layer representation output by the feature encoder. It contains a fully connected output layer with a number of nodes equal to the number of predefined abnormal behavior categories. There are six predefined abnormal behavior categories: vent pecking, feather pecking, fighting, stereotyped behavior, lethargy, and abnormal appetite. The classifier performs a linear transformation on the high-dimensional hidden layer representation and calculates its score for each predefined abnormal behavior category. Applying a normalized exponential function to all six scores converts them into probability values, yielding the probability distribution of the initial behavioral fragment belonging to different abnormal behavior categories. The sum of the six probability values ​​is always 1, and the category with the highest probability value is considered the most likely behavior category of the initial behavioral fragment.

[0032] In specific implementation, the training process of the pre-trained abnormal behavior recognition model is as follows: A large number of labeled depth image sequence samples of laying hen behavior are collected. Each sample contains depth image frames of laying hens continuously acquired from a depth camera above the cage within a preset time period. Trained observers view the synchronously acquired visible light video frame by frame and label the abnormal behavior categories exhibited by the laying hens in each time interval. The labeled abnormal behavior categories are limited to six categories: vent pecking, feather pecking, fighting, stereotyped behavior, lethargy, and abnormal appetite. Time intervals without labeled abnormal behavior are considered normal behavior by default. For each training sample, the corresponding depth image sequence is extracted from the original depth camera record in the same way as the depth image sequence acquisition. The depth image sequence is processed according to the preprocessing procedure described in the previous embodiment to generate a laying hen posture depth map sequence. The improved temporal segmentation algorithm described in the previous embodiment is used to perform temporal segmentation on the laying hen posture depth map sequence to obtain multiple preliminary behavior segments. A spatiotemporal joint behavior feature vector is extracted from each preliminary behavior segment according to the feature extraction method described in the previous embodiment. The extracted spatiotemporal joint behavior feature vector is stored together with the timestamp of the preliminary behavior segment. For each preliminary behavior segment, calculate its temporal intersection-union ratio (CIU) with the labeled sample segments. When the CIU with a certain abnormal behavior labeled segment exceeds 0.7, the spatiotemporal joint behavior feature vector of the preliminary behavior segment is labeled as the abnormal behavior category. Preliminary behavior segments that do not meet the CIU condition with any abnormal behavior labeled segments are labeled as normal behavior categories. In this way, a training sample set of spatiotemporal joint behavior feature vectors with category labels is constructed.

[0033] The initial neural network model, containing a feature encoder and a classifier, is trained using the constructed training sample set. In each iteration, a fixed number of samples are randomly selected from the training sample set. The spatiotemporal joint behavioral feature vectors of these samples are fed into the initial neural network model. After multiple layers of nonlinear mapping by the feature encoder and linear transformation by the classifier, the predicted class probability distribution is output. The cross-entropy loss is calculated using the predicted probability distribution and the true class label. The gradient of the cross-entropy loss with respect to all weight and bias parameters in the feature encoder and classifier is calculated using the backpropagation algorithm. A stochastic gradient descent optimizer is used to update the parameters along the negative gradient direction. This process is repeated multiple times to minimize the difference between the model's output class probability distribution and the true label. After each training round, the current model is evaluated using an independent validation set that has no intersection with the training sample set. The samples in the independent validation set also contain spatiotemporal joint behavioral feature vectors and class labels. The model's behavior classification accuracy on the independent validation set is calculated. Training stops when the model's recognition accuracy on the independent validation set no longer improves after five consecutive rounds, and the model parameters at this point are saved as a pre-trained abnormal behavior recognition model.

[0034] In some embodiments, stereotyped behaviors in predefined abnormal behavior categories include repetitive, aimless pecking at cages and pacing back and forth along fixed routes. Abnormal appetite is judged by hens pausing in front of the feed trough but pecking at a significantly lower frequency than normal. In some embodiments, during the training sample construction phase, the labeled abnormal behavior categories are artificially enhanced by randomly cropping time window boundaries and applying small translational perturbations to the depth image sequence to increase the diversity of training samples. Optionally, random deactivation operations are introduced between the fully connected layers of the feature encoder, randomly discarding 0.5% of neuron outputs during training to enhance the model's generalization ability. Optionally, temperature scaling techniques are applied to the classifier output scores, dividing all scores uniformly by a temperature coefficient greater than 1 before calculating the normalization exponential function, making the probability distribution smoother and helping to distinguish highly similar abnormal behavior categories. It can be understood that the feature encoder abstracts the spatiotemporal joint behavioral feature vector, which integrates depth appearance and skeletal motion information, layer by layer through multi-layer nonlinear mapping, ultimately resulting in a clustered distribution of features corresponding to different abnormal behavior categories in the hidden space. It is understandable that the pre-trained abnormal behavior recognition model does not rely on labeled information during actual operation. It can obtain the probability distribution by directly performing a forward propagation on the spatiotemporal joint behavioral feature vector of each preliminary behavior segment, which meets the timeliness requirements of real-time monitoring of abnormal behavior in aquaculture scenarios.

[0035] In one embodiment of the present invention, the behavior category corresponding to the preliminary behavior segment is determined according to probability, and an abnormal behavior identification result including timestamp, behavior category and confidence level is output. This includes: presetting a probability threshold for each abnormal behavior category; comparing the probability of the preliminary behavior segment belonging to a certain abnormal behavior category with the corresponding probability threshold; if the probability is greater than or equal to the probability threshold, the preliminary behavior segment is determined to belong to the abnormal behavior category, and the probability value is used as the confidence level; if the probability is less than the probability threshold of all abnormal behavior categories, the preliminary behavior segment is determined to be normal behavior; and for the preliminary behavior segment determined to be abnormal behavior, its start timestamp, end timestamp, determined behavior category and corresponding confidence level are recorded to form a structured abnormal behavior identification result.

[0036] In practice, based on the probability distribution calculated for each preliminary behavior segment by the pre-trained abnormal behavior recognition model, the behavior category corresponding to the preliminary behavior segment is determined and a structured abnormal behavior recognition result is output. A probability threshold is preset for each predefined abnormal behavior category. The probability thresholds for pecking, feather pecking, fighting, stereotyped behavior, lethargy, and abnormal appetite are set independently. All six probability thresholds are stored in the system configuration parameters for use in the judgment stage.

[0037] For the preliminary behavioral segment to be determined, the probability distribution of the preliminary behavioral segment belonging to different abnormal behavior categories, output by the pre-trained abnormal behavior recognition model of this embodiment, is obtained. The probability distribution is a six-dimensional vector, and each component of the six-dimensional vector corresponds to the probability value of the pecking vent category, the feather pecking category, the fighting category, the stereotyped behavior category, the drowsy category, and the abnormal appetite category, respectively. Each component in the six-dimensional vector is compared one by one with the probability threshold of the corresponding abnormal behavior category. The comparison process starts with the component with the highest probability value in the six-dimensional vector. If the probability value of the component is greater than or equal to the probability threshold of its corresponding abnormal behavior category, the preliminary behavioral segment is determined to belong to that abnormal behavior category, and the probability value of the component is directly used as the confidence level of this determination. If the probability value of the component is less than the probability threshold of its corresponding abnormal behavior category, the next highest probability value in the six-dimensional vector is checked until a component with a probability value greater than or equal to the probability threshold of its corresponding abnormal behavior category is found.

[0038] If, after traversing all six components of the six-dimensional vector, the probability value of each component is less than the probability threshold of its corresponding abnormal behavior category, then the preliminary behavior segment is determined to be normal behavior, and no abnormal behavior identification record is generated for the preliminary behavior segment. For preliminary behavior segments determined to be abnormal behavior, the time information of the preliminary behavior segment is integrated with the determination result into a structured abnormal behavior identification result, recording the start timestamp, end timestamp, determined behavior category, and corresponding confidence level of the preliminary behavior segment. The start timestamp is the acquisition time of the first frame of depth image within the preliminary behavior segment, and the end timestamp is the acquisition time of the last frame of depth image within the preliminary behavior segment. The timestamp precision is consistent with the acquisition frame rate of the depth camera.

[0039] In implementation, the structured abnormal behavior identification results of all preliminary behavioral segments judged as abnormal behaviors are arranged in chronological order by timestamp, forming a complete abnormal behavior identification output sequence. The output sequence includes the start and end time interval of each abnormal behavior, the specific name of the predefined abnormal behavior category to which it belongs, and the confidence level in numerical form. In some embodiments, the probability threshold for pecking the vent is set to 0.75, the probability threshold for feather pecking is set to 0.70, the probability threshold for fighting is set to 0.80, the probability threshold for stereotyped behavior is set to 0.65, the probability threshold for lethargy is set to 0.70, and the probability threshold for abnormal appetite is set to 0.60. These thresholds are uniformly adjusted through the parameter configuration interface before system deployment based on the need to balance the false alarm rate and the false negative rate. In some embodiments, the structured abnormal behavior identification results are sent to the farm monitoring terminal in real time. The monitoring terminal interface displays the duration and behavior category label of each abnormal behavior segment in the form of a timeline bar chart, and different warning colors are assigned to different abnormal behavior categories. At the same time, the confidence level is marked as a percentage value on the corresponding bar. It is understandable that setting differentiated probability thresholds for different abnormal behavior categories allows for the use of higher thresholds for behaviors that pose a greater threat to production, such as vent pecking and fighting, to reduce false alarms. Conversely, a moderate threshold is used for chronic abnormal behaviors, such as stereotyped behaviors and abnormal appetite, to improve early detection capabilities. This balances detection sensitivity and alarm reliability. Optionally, if two adjacent preliminary behavioral segments are classified as belonging to the same abnormal behavior category and the interval between their end and start timestamps is less than a preset merging interval, the abnormal behavior identification results of the two segments are merged into one, forming a continuous and complete record of abnormal behavior events. Optionally, for abnormal behavior identification results with a confidence level below 0.85, a low-confidence marker is added to the monitoring terminal interface to prompt livestock management personnel to manually verify the results in conjunction with video playback.

[0040] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for identifying abnormal behavior characteristics of laying hens based on depth images, characterized in that, The method includes: Obtain depth image sequences of laying hens in target cages within a preset time period; The depth image sequence is preprocessed to extract the torso region of the laying hen and generate a depth map sequence of laying hen postures. An improved temporal segmentation algorithm is applied to the depth map sequence of laying hens to divide it into continuous time periods with different behavioral semantics, resulting in a preliminary set of behavioral segments. The improved temporal segmentation algorithm optimizes the boundaries based on the similarity measure of the motion trajectories of the skeleton key points in the depth image sequence. For each preliminary behavioral segment in the set of preliminary behavioral segments, spatiotemporal joint behavioral features are extracted based on its corresponding depth image subsequence; The spatiotemporal joint behavioral features are input into a pre-trained abnormal behavior recognition model to calculate the probability that the preliminary behavioral fragment belongs to different abnormal behavior categories. Based on the probability, the behavior category corresponding to the preliminary behavior segment is determined, and the abnormal behavior identification result containing timestamp, behavior category and confidence level is output.

2. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 1, characterized in that, The depth image sequence is preprocessed to extract the torso region of the laying hen, generating a laying hen pose depth map sequence, including: For each frame of the depth image sequence, a background modeling and foreground segmentation algorithm is applied to separate the foreground region of the laying hen from the background environment; Cluster analysis was performed on the foreground area of ​​the laying hens based on depth values ​​to distinguish the laying hen bodies from static objects such as cage railings, feed troughs, and nipple drinkers. Using a 3D skeleton estimation algorithm, key points are extracted from the segmented point cloud of the laying hen's body to locate the head, neck, back, torso, wings, legs, and claws in 3D space. Centered on the key points of the hen's torso, a three-dimensional bounding box is drawn on the original depth image, and the depth values ​​within the three-dimensional bounding box are resampled into a fixed-size depth image block; The sequence of depth image blocks arranged in chronological order is used as the depth map sequence of the laying hen posture.

3. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 1, characterized in that, An improved temporal segmentation algorithm is applied to the hen pose depth map sequence to divide it into continuous time periods with different behavioral semantics, resulting in a preliminary set of behavioral segments, including: Extract consecutive depth image frames from the sequence of egg-laying hen posture depth maps in chronological order; For each frame of depth image, the skeleton pose vector of the current frame is calculated based on the 3D key point information it contains, and the skeleton pose vectors of all time frames constitute the pose trajectory. The improved temporal segmentation algorithm is used to analyze the attitude trajectory. The improved temporal segmentation algorithm calculates the local similarity matrix of the attitude trajectory and detects the boundary points of similarity abrupt changes along the diagonal direction in the local similarity matrix. After detecting the boundary point, the improved temporal segmentation algorithm performs fine-tuning of the boundary position based on the temporal smoothing constraint of the poses on both sides of the boundary point, ensuring that the pose changes within the behavior segment are continuous and that there are significant differences in pose between segments. The time interval between adjacent boundary points is marked as a preliminary behavior segment, and all preliminary behavior segments constitute the set of preliminary behavior segments.

4. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 3, characterized in that, The improved temporal segmentation algorithm optimizes the boundary based on the similarity measure of the motion trajectories of skeleton key points in the depth image sequence, including: Calculate the three-dimensional motion trajectory of a specific set of key points in consecutive time frames from the sequence of egg-laying hen posture depth maps; Define a metric function to measure the similarity between two short-term trajectories, which combines the Euclidean distance of the trajectory shapes with the cosine similarity of the directions of motion; Slide a fixed-length time window on the attitude trajectory and calculate the similarity metric between the sub-trajectory within the window and the sub-trajectory of the adjacent window; The similarity measure is used as a time sequence as a change curve, local minima are detected on the change curve, and the local minima are used as preliminary potential boundaries. For each initial potential boundary, the similarity metric is recalculated within its neighborhood using more refined trajectory sampling. The boundary points are then adjusted to the precise positions where the similarity metric value is minimized, thus completing the boundary optimization.

5. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 1, characterized in that, For each preliminary behavioral segment in the set of preliminary behavioral segments, spatiotemporal joint behavioral features are extracted based on its corresponding depth image subsequence, including: For the depth image subsequence corresponding to the initial behavior segment, the displacement field of the three-dimensional key points between consecutive frames is calculated to obtain the dense optical flow field sequence; The depth image subsequence is subjected to three-dimensional voxelization processing, and the depth value change of each voxel grid on the time axis is constructed into a spatiotemporal cube; The spatiotemporal cube is subjected to feature extraction using a three-dimensional convolutional neural network to obtain a low-level feature map representing the local spatiotemporal pattern; The dense optical flow field sequence is processed using a graph convolutional network. The key points of the skeletal structure of the laying hen are used as graph nodes, and the skeleton connections and inter-frame correspondences are used as graph edges. The spatiotemporal dependencies between nodes are learned, and spatiotemporal features based on the skeleton graph are output. The low-level feature map is fused and pooled with the spatiotemporal features based on the skeleton graph to generate a fixed-dimensional spatiotemporal joint behavioral feature vector.

6. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 5, characterized in that, The dense optical flow field sequence is processed using a graph convolutional network. Key points of the hen's skeleton are used as graph nodes, and the skeleton connections and inter-frame correspondences are used as graph edges. The spatiotemporal dependencies between nodes are learned, and spatiotemporal features based on the skeleton graph are output, including: A spatiotemporal graph is constructed for the initial behavioral segment, and the node set of the spatiotemporal graph consists of all skeletal key points on all time frames; The spatiotemporal graph has spatial edges and temporal edges. The spatial edges connect pairs of key points with physiological connections within the same frame, and the temporal edges connect the same key point in adjacent frames. Each node's features consist of its three-dimensional coordinates, depth value, and motion vector at that point extracted from the dense optical flow field. The constructed spatiotemporal graph is input into a multi-layer graph convolutional network, which aggregates neighborhood node information along the spatial edge in the spatial dimension and aggregates continuous frame information along the temporal edge in the temporal dimension. After multiple layers of information transmission, the features of all nodes output from the last layer are globally pooled to obtain the spatiotemporal features based on the skeleton graph.

7. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 5, characterized in that, The spatiotemporal joint behavioral features are input into a pre-trained abnormal behavior recognition model to calculate the probability that the initial behavioral fragment belongs to different abnormal behavior categories, including: The pre-trained abnormal behavior recognition model includes a feature encoder and a classifier; The feature encoder receives the spatiotemporal joint behavior feature vector and converts it into a high-dimensional hidden layer representation through multi-layer nonlinear mapping. The classifier receives the high-dimensional hidden layer representation output by the feature encoder and calculates its score for each predefined abnormal behavior category, which includes vent pecking, feather pecking, fighting, stereotyped behavior, lethargy, and abnormal appetite. Applying a normalized exponential function to the scores converts them into probability values, yielding the probability distribution of the preliminary behavioral fragments belonging to different abnormal behavior categories.

8. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 7, characterized in that, The pre-trained abnormal behavior recognition model is trained through the following steps: We collected a large number of labeled deep image sequence samples of laying hen behavior, with each sample labeled with the abnormal behavior category and the corresponding time period; For each training sample, the following steps are taken: acquiring depth image sequences, preprocessing to generate pose depth map sequences, performing improved temporal segmentation to obtain preliminary behavioral fragments, and extracting spatiotemporal joint behavioral features to obtain the corresponding spatiotemporal joint behavioral feature vector and category label. The initial neural network model containing the feature encoder and classifier is trained using labeled training samples; During training, the model parameters are optimized using the backpropagation algorithm to minimize the difference between the class probability distribution output by the model and the true label. When the model's recognition accuracy on the independent validation set no longer improves, training is stopped, and the pre-trained abnormal behavior recognition model is obtained.

9. The method for identifying abnormal behavior features of laying hens based on depth images according to claim 7, characterized in that, Based on the probability, the behavior category corresponding to the preliminary behavior fragment is determined, and an abnormal behavior identification result including a timestamp, behavior category, and confidence level is output, including: A probability threshold is preset for each category of abnormal behavior; The probability that the preliminary behavioral fragment belongs to a certain abnormal behavior category is compared with the corresponding probability threshold; If the probability is greater than or equal to the probability threshold, then the preliminary behavioral fragment is determined to belong to the abnormal behavior category, and the probability value is used as the confidence level. If the probability is less than the probability threshold of all abnormal behavior categories, then the initial behavior segment is determined to be normal behavior; For the initial behavioral segments identified as abnormal behavior, record their start timestamp, end timestamp, the identified behavior category, and the corresponding confidence level to form the structured abnormal behavior identification result.

10. A system for identifying abnormal behavior features of laying hens based on depth images, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for identifying abnormal behavior features of laying hens based on depth images as described in any one of claims 1 to 9.

Images