Livestock number recognition method and system combining AI vision and multi-target tracking

Abstract

The application provides a livestock quantity identification method and system combining AI vision and multi-target tracking, which collects continuous video streams of livestock activity areas, generates video frame sequences with timestamp markers for dynamic background modeling operation, extracts contour feature sets of livestock objects in each frame image, generates multi-dimensional motion representation vectors, performs adaptive multi-scale feature enhancement, decouples pixel-level features of livestock cluster areas, outputs shape-motion joint coding sequences of each independent livestock individual, constructs a space-time correlation matrix based on this, extracts topological features of livestock motion trajectories through a lightweight feature extraction network, generates a global ID mapping table containing unique identification of individuals, performs real-time statistics of livestock quantity based on the global ID mapping table, and performs behavior tracing analysis on livestock individuals corresponding to abnormal motion trajectories, to obtain quantity statistics results and abnormal behavior identification results. The application can improve the accuracy of livestock quantity statistics.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method and system for livestock number recognition that combines AI vision with multi-target tracking. Background Technology

[0002] In the field of livestock farming management, accurate identification and dynamic tracking of livestock numbers are crucial for achieving refined management, disease prevention and control, and improved production efficiency. The core objective is to use technology to monitor real-time changes in livestock numbers and individual activity patterns within the farming area. Current technologies typically employ manual counting or simple visual recognition methods for livestock counting. Manual counting is not only time-consuming and labor-intensive but also susceptible to human error. Traditional visual recognition methods often extract foreground targets by subtracting from static backgrounds and rely on basic features such as color and shape from a single frame for individual detection and counting. However, in practical applications, complex situations arise in farming environments, including dynamic background interference, occlusion and overlap caused by livestock herding, changes in individual movement postures, and similar appearances. Traditional methods often struggle to effectively extract stable individual features, leading to issues such as missed detections, duplicate counting, or frequent tracking ID switching. This results in low accuracy in counting and an inability to effectively trace individuals with abnormal movements, failing to meet the demands of modern livestock farming for precise and intelligent management. Summary of the Invention

[0003] In view of this, the present invention provides a method and system for livestock number identification that combines AI vision and multi-target tracking.

[0004] According to one aspect of the present invention, a livestock quantity identification method combining AI vision and multi-target tracking is provided. The method includes: acquiring a continuous video stream of a livestock activity area to generate a video frame sequence with timestamps; performing dynamic background modeling on the video frame sequence to extract a set of contour features of livestock objects in each frame image, generating a multi-dimensional motion representation vector based on the temporal variation law of the contour feature set, wherein the multi-dimensional motion representation vector includes contour deformation parameters and motion trend descriptors; performing adaptive multi-scale feature enhancement on the multi-dimensional motion representation vector to decouple the pixel-level features of the livestock cluster area and output a morphology-motion joint encoding sequence for each independent livestock individual; constructing a spatiotemporal correlation matrix based on the morphology-motion joint encoding sequence, extracting the topological features of the livestock movement trajectory through a lightweight feature extraction network, and generating a global ID mapping table containing a unique identifier for each individual; performing real-time livestock quantity statistics based on the global ID mapping table, and performing behavioral tracing analysis on livestock individuals corresponding to abnormal movement trajectories to obtain quantity statistics results and abnormal behavior identification results.

[0005] According to another aspect of the present invention, a computer system is provided, comprising: a processor; and a memory, wherein the memory stores computer-readable code that, when executed by the processor, causes the processor to perform the method as described above.

[0006] This invention provides a livestock number recognition method combining AI vision and multi-target tracking. Through deep fusion of dynamic background modeling and the temporal variation law of contour features, it can adaptively handle dynamic background interference in livestock activity scenarios, effectively extracting multi-dimensional motion features including contour deformation and movement trends, and improving the recognition stability of livestock objects in complex environments. Through the collaborative design of adaptive multi-scale feature enhancement and decoupling of cluster regions, it specifically strengthens the expression of detailed features in cluster regions, achieving accurate separation of overlapping individuals in livestock cluster scenarios, and improving the completeness and accuracy of individual recognition. Based on the morphology-motion joint encoding sequence, a spatiotemporal correlation matrix is ​​constructed, and combined with a lightweight feature extraction network to extract trajectory topology features, which can effectively capture the global structural relationship of livestock movement trajectories, reduce tracking errors caused by occlusion, similar appearances, etc., and improve the stability and continuity of multi-target tracking. Through a global ID mapping table, it achieves integrated processing of real-time livestock number statistics and abnormal movement trajectory tracing, which can not only accurately count the current total number of livestock, but also associate abnormal behavior with specific individuals, providing more comprehensive decision support for livestock management. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the architecture for an application scenario provided by the present invention;

[0008] Figure 2 This is a flowchart illustrating a livestock number identification method combining AI vision and multi-target tracking provided by the present invention.

[0009] Figure 3 This is a schematic diagram of the structure of a computer system provided in an embodiment of the present invention. Detailed Implementation

[0010] To facilitate a clearer understanding of this invention, we will first introduce the application scenarios of the media data processing method for implementing this invention, such as... Figure 1 As shown, the scenario includes a computer system 10 and a terminal cluster. The terminal cluster can include one or more terminals; the number of terminals will not be limited here. Figure 1 As shown, the terminal cluster may specifically include terminal 1, terminal 2, ..., terminal n; it can be understood that terminal 1, terminal 2, terminal 3, ..., terminal n are all connected to the computer system 10 via a network so that each terminal can interact with the computer system 10 through the network connection.

[0011] Understandably, computer system 10 can refer to a device that performs a livestock number recognition method combining AI vision and multi-target tracking. Computer system 10 can also be used to store surveillance video streams. Computer system 10 can be a server, such as a standalone physical server, or a server cluster or distributed system consisting of multiple physical servers, with the monitoring equipment as the terminal.

[0012] Further, please see Figure 2 This is a flowchart illustrating a livestock quantity identification method combining AI vision and multi-target tracking, provided by an embodiment of the present invention. Figure 2 As shown, this method can be derived from... Figure 1 The method for livestock number identification, which combines AI vision and multi-target tracking, is executed by a computer system 10, and may include the following steps:

[0013] Step S100: Collect continuous video streams of the livestock activity area and generate a video frame sequence with timestamps.

[0014] Specifically, video capture devices, such as cameras, can be used to continuously record video of the activity area of ​​livestock (such as pigs) to obtain a continuous sequence of images reflecting the dynamic behavior of the livestock. Timestamps are time information added to video frames to accurately identify the temporal position of each frame in the video stream, which helps in subsequent temporal correlation and analysis of video frames.

[0015] Step S200: Perform dynamic background modeling on the video frame sequence, extract the contour feature set of livestock objects in each frame image, and generate a multidimensional motion representation vector based on the temporal change law of the contour feature set. The multidimensional motion representation vector includes contour deformation parameters and motion trend descriptors.

[0016] Dynamic background modeling involves analyzing and processing video frame sequences to create a model that reflects changes in the background over time. Since the background may be affected by factors such as changes in lighting and slight scene shifts during livestock activity, dynamic background modeling can effectively separate the foreground (i.e., the livestock object) from the background. The contour feature set is a collection of features describing the outline of the livestock object, including edge point coordinate sequences, multi-scale curvature variation features, region area parameters, and directional gradient distribution features. These features can reflect the livestock's shape and posture from different angles.

[0017] In one implementation, step S200 can be implemented as the following steps S210~S260:

[0018] Step S210: Perform spatiotemporal segmentation on the video frame sequence to divide it into spatiotemporal units containing spatial location encoding and time sequence information. The spatial size of the spatiotemporal unit is dynamically adjusted according to the video resolution, and the time span is matched with the livestock movement cycle.

[0019] Spatiotemporal segmentation divides a video frame sequence into spatiotemporal units along both spatial and temporal dimensions. Spatial location encoding assigns a unique identifier to each spatiotemporal unit, representing its spatial position within the video frame. Temporal sequence information records the order and duration of each spatiotemporal unit over time. When video resolution is high, the spatial size can be relatively small to capture finer details; when video resolution is low, the spatial size needs to be appropriately increased to ensure each spatiotemporal unit contains sufficient information. The livestock movement cycle refers to the time required for a livestock to complete a full movement cycle. Matching the time span to the livestock movement cycle ensures that each spatiotemporal unit can completely record one stage of the livestock's movement.

[0020] Step S220: Dynamic background estimation of spatiotemporal units is performed through background probability modeling. Based on the time dimension model parameter update mechanism, the probability value of each spatiotemporal unit belonging to the background region is calculated, and a background probability distribution map that evolves over time is generated.

[0021] Background probability modeling uses probabilistic and statistical methods to analyze pixel values ​​within spatiotemporal units and establish a model describing the probability of a pixel belonging to a background region. Dynamic background estimation considers that the background may change over time and continuously updates the background model to adapt to these changes. The time-dimensional model parameter update mechanism dynamically adjusts the parameters of the background probability model according to the passage of time to reflect real-time changes in the background. Through this mechanism, the probability value of each spatiotemporal unit belonging to the background region can be calculated. The background probability distribution map is a visual graph based on spatiotemporal units, displaying the probability of each spatiotemporal unit belonging to the background region. It evolves continuously over time, intuitively reflecting the dynamic changes of the background.

[0022] Step S230: Determine the foreground region determination conditions based on the spatiotemporal distribution characteristics of the background probability distribution map, mark the regions that meet the determination conditions with probability values ​​lower than the determination conditions for multiple consecutive frames as candidate livestock regions, and generate a binarized foreground mask map with spatiotemporal correlation.

[0023] The spatiotemporal distribution characteristics of the background probability distribution map refer to the distribution patterns of background probabilities in spatial and temporal dimensions, including the magnitude of probability values, distribution range, and trends. The foreground region determination criterion is a threshold set based on these spatiotemporal distribution characteristics to distinguish between the foreground (livestock objects) and the background. When the probability value of a region is lower than this determination criterion in multiple consecutive frames, it is marked as a candidate livestock region. A binarized foreground mask is an image where each pixel has only two states: 0 and 1. 0 represents the background region, and 1 represents the foreground region (i.e., the candidate livestock region). Spatiotemporal correlation means that the mask not only considers information from the current frame but also incorporates information from adjacent frames, making the marked candidate livestock regions continuous in both time and space.

[0024] Step S240: Perform morphological optimization on the binarized foreground mask image, sequentially performing dilation processing to connect broken contours and erosion processing to restore detailed features, generating an optimized foreground mask image containing the complete livestock object region, wherein the size of the processing kernel is dynamically adjusted according to the estimated body shape features of the livestock.

[0025] Morphological optimization operations utilize mathematical morphology methods to process binarized foreground masks to improve image quality and shape. Dilation is a morphological operation that expands the foreground region outward by adding pixels to the image, thus connecting broken contours. Erosion, on the other hand, is the opposite operation, shrinking the foreground region inward by removing pixels from image edges to restore detailed features. The processing kernel is a small template used in morphological operations, and its size affects the operation's effectiveness. Dynamically adjusting the kernel size based on the estimated body shape of the livestock ensures that the operation can better adapt to livestock of different body shapes. When processing the binarized foreground mask, dilation is performed first to connect broken livestock contours. Then, erosion is performed to remove the excess parts generated during dilation, restoring the detailed features of the livestock object.

[0026] Step S250: Extract the contour feature set of livestock objects from the original video frame based on the optimized foreground mask map. The contour feature set includes edge point coordinate sequence, multi-scale curvature change features, region area parameters and directional gradient distribution features.

[0027] An optimized foreground mask is a binarized image that accurately identifies the livestock object region after morphological optimization. The original video frame is the initially captured video image containing livestock activity. By optimizing the foreground mask, the livestock object can be accurately located from the original video frame, and its contour feature set can be extracted. The edge point coordinate sequence refers to the coordinate values ​​of each edge point on the livestock object's contour, which can accurately describe the livestock's external outline. Multi-scale curvature variation features describe the curvature changes of the livestock contour at different scales, reflecting the detailed changes in the livestock contour. The region area parameter refers to the size of the area occupied by the livestock object, which can reflect the size of the livestock macroscopically. The directional gradient distribution feature refers to the gradient direction distribution of each point on the livestock contour, which can reflect the livestock's posture and orientation.

[0028] In one implementation, step S250 can be implemented as steps S251 to S256:

[0029] Step S251: Perform edge extraction processing on the optimized foreground mask image, distinguish strong edges from weak edges through a dynamic threshold adjustment mechanism, and generate a hierarchical edge point coordinate sequence containing continuous edge structures. The dynamic threshold adapts to the complexity of the regional texture.

[0030] Edge extraction involves identifying regions of dramatic grayscale value change (i.e., edges) in an image from an optimized foreground mask. A dynamic threshold adjustment mechanism automatically adjusts the threshold based on different image regions, distinguishing between strong and weak edges. Strong edges are those with significant grayscale value changes and high confidence; weak edges have relatively small grayscale value changes and lower confidence. A hierarchical edge point coordinate sequence contains coordinates of edge points at different levels, providing a more comprehensive description of the edge structure of the object. Region texture complexity refers to the degree of texture complexity in an image region; the dynamic threshold adaptively adjusts based on this complexity to ensure accurate edge extraction across different texture regions.

[0031] Step S252: Perform polygon approximation processing on the layered edge point coordinate sequence, calculate the curvature change rate of adjacent edge points, mark the region with the curvature change rate exceeding the preset change rate threshold as the key feature region and retain its inflection point, mark the region with the curvature change rate not exceeding the preset change rate threshold as the regular feature region and simplify its redundant edge points, and generate a simplified contour sequence containing the inflection point coordinates and corresponding curvature level.

[0032] Polygon approximation processing uses polygons to approximate the contour described by the sequence of layered edge point coordinates, reducing data volume and highlighting key features. The rate of curvature change between adjacent edge points refers to the speed at which the curvature of the contour changes between two adjacent edge points. A preset rate of curvature threshold is a pre-defined value used to distinguish between critical feature regions and regular feature regions. Critical feature regions are areas in the contour with drastic curvature changes and important features, such as the joints or head of livestock; regular feature regions are areas in the contour with relatively gentle curvature changes. Inflection points are points in critical feature regions where the contour direction changes significantly. By marking regions with a rate of curvature change exceeding the preset threshold as critical feature regions and retaining their inflection points, and marking regions with a rate of curvature change below the preset threshold as regular feature regions and simplifying their redundant edge points, a simplified contour sequence containing inflection point coordinates and corresponding curvature levels can be generated.

[0033] Step S253: Perform multi-scale curvature analysis on the simplified contour sequence, calculate the change in the tangent angle between adjacent turning points at different scale levels, and generate multi-scale curvature change features that describe the changes in contour details. The number of scale levels is adjusted according to the contour complexity.

[0034] Multi-scale curvature analysis analyzes the curvature of a simplified contour sequence at different scales to capture detailed changes in the contour at various scales. Tangent angle variation refers to the change in the angle between the contour tangents between adjacent inflection points, reflecting the degree of curvature of the contour in that region. The number of scale levels is adjusted according to the contour complexity; complex contours require more scale levels to comprehensively describe their detailed changes, while simple contours require fewer scale levels.

[0035] Step S254: Calculate the number of pixels in each connected region in the foreground mask image through region filling processing, convert it into the actual physical region area parameter by combining the spatial coordinate system, and simultaneously record the rate of change of the area parameter in the time dimension.

[0036] Region filling processing involves filling all pixels within each connected foreground region (i.e., the livestock object region) in the optimized foreground mask image with the same value to accurately calculate its pixel count. A connected region is a continuous area composed of adjacent pixels. By calculating the number of pixels in each connected region, the size of that region can be obtained. Spatial coordinate transformation involves converting the pixel count into the actual physical area, which requires knowledge of the video image resolution and the actual scene dimensions. The rate of change of the area parameter over time refers to how quickly the area of ​​the livestock object region changes over time, reflecting the livestock's growth, posture changes, etc. For example, in a livestock farm, region filling processing is performed on the connected regions of each pig in the optimized foreground mask image. By calculating the number of pixels in the connected regions of each pig and converting the pixel count into the actual physical area based on the ratio between resolution and actual size, the area parameter values ​​for each pig are recorded at different time points, and its rate of change over time is calculated. If a pig's area parameter changes significantly over a period of time, it may indicate that it has undergone significant growth or posture changes.

[0037] Step S255: Perform directional gradient statistics on the coordinate sequence of layered edge points, divide the gradient direction of the edge points into multiple intervals, calculate the gradient magnitude ratio for each interval, and generate directional gradient distribution features that describe the contour direction characteristics.

[0038] Oriented gradient statistics involve statistically analyzing the gradient direction and magnitude of each edge point in a hierarchical edge point coordinate sequence. The gradient direction is the direction of the greatest change in grayscale value at the edge point, while the gradient magnitude refers to the magnitude of this change. Dividing the gradient direction of edge points into multiple intervals allows for a more detailed description of the directional distribution of edge points. Calculating the gradient magnitude percentage for each interval refers to calculating the proportion of the sum of the gradient magnitudes of all edge points within that interval to the total gradient magnitude. Oriented gradient distribution characteristics reflect the directional properties of livestock silhouettes and can help determine the posture and orientation of livestock.

[0039] Step S256: Integrate the layered edge point coordinate sequence, multi-scale curvature change features, physical region area parameters and directional gradient distribution features, and construct a contour feature set containing spatial morphology and spatiotemporal dynamic characteristics after feature standardization.

[0040] Integration refers to combining several different types of features, such as the hierarchical edge point coordinate sequence, multi-scale curvature variation characteristics, physical region area parameters, and directional gradient distribution characteristics, into a unified feature representation. Feature standardization is a normalization operation performed on these features to give them the same dimensions and range, thereby eliminating scale differences between different features. The contour feature set, which includes spatial morphology and spatiotemporal dynamic characteristics, is a comprehensive feature set that reflects both the spatial morphological characteristics of the livestock object (such as its outline and area size) and its dynamic changes in the temporal dimension (such as contour deformation and direction of movement).

[0041] Step S260: Generate a multidimensional motion representation vector based on the temporal variation pattern of the contour feature set. The multidimensional motion representation vector includes contour deformation parameters and motion trend descriptors.

[0042] The temporal variation pattern of a contour feature set refers to the pattern of change of the contour feature set over time, including changes in shape, area, and direction. By analyzing these variation patterns, a multidimensional motion representation vector can be generated. This multidimensional motion representation vector is a vector containing features across multiple dimensions, integrating contour deformation parameters and motion trend descriptors to more comprehensively represent the movement characteristics of livestock. Contour deformation parameters describe how the shape of the livestock contour changes over time, such as stretching, compression, and twisting; motion trend descriptors predict the future movement state of livestock, such as changes in direction and speed.

[0043] In one implementation, step S260 can be implemented as steps S261 to S266:

[0044] Step S261: Perform time-axis alignment processing on the contour feature set in consecutive video frames, and establish the correspondence between cross-frame contour features through the matching algorithm of edge point coordinate sequence. The matching priority is determined based on the similarity of multi-scale curvature change features.

[0045] Timeline alignment involves arranging contour feature sets from consecutive video frames in chronological order to ensure the accuracy of the time points corresponding to each feature set. Edge point coordinate sequence matching algorithms are methods for finding correspondences between edge point coordinate sequences in different frames. This algorithm establishes cross-frame contour feature correspondences, thus determining the contour features of the same animal in different frames. Matching priority is determined based on the similarity of multi-scale curvature variation features, meaning that during the matching process, contour feature sets with higher similarity in multi-scale curvature variation features are prioritized for matching. This is because multi-scale curvature variation features reflect detailed contour changes, and feature sets with higher similarity are more likely to belong to the same animal.

[0046] Step S262: Calculate the offset of edge point coordinates of the contour feature set between adjacent frames, supplement the offset information of the occluded area to generate a complete contour deformation displacement field, and sparse the contour deformation displacement field to reduce the data dimensionality.

[0047] The offset of edge point coordinates in the contour feature set between adjacent frames refers to the change in the coordinates of edge points in the contour feature set of the same livestock object between two adjacent video frames. In real-world scenarios, livestock may occlude each other, leading to inaccurate acquisition of some edge point coordinates. Supplementing the offset information of occluded areas involves using estimation methods (such as inference based on the movement trends of surrounding visible edge points) to compensate for the coordinate offset information of occluded areas, thereby generating a complete contour deformation displacement field. The contour deformation displacement field is a field describing the deformation of the contour between adjacent frames, containing the coordinate offset information of all edge points. Sparse representation of the contour deformation displacement field involves using algorithms (such as principal component analysis) to remove redundant information, retaining only important features to reduce data dimensionality and computational load.

[0048] Step S263: Analyze the vector distribution characteristics of the profile deformation displacement field, extract the deformation parameters in the main direction and the deformation amplitude values ​​in each region, calculate the spatial distribution uniformity index of the deformation amplitude, and integrate the deformation parameters in the main direction, the deformation amplitude values ​​in each region and the spatial distribution uniformity index of the deformation amplitude into profile deformation parameters that describe the changes in profile shape.

[0049] The vector distribution characteristics of the profile deformation displacement field refer to the distribution of the direction and magnitude of vectors in the profile deformation displacement field. The principal direction deformation parameter refers to the parameter of change of the profile in the main deformation direction, which can reflect the overall deformation trend of the profile. The deformation amplitude value of each region refers to the degree of deformation in different regions of the profile. The spatial distribution uniformity index of deformation amplitude is a numerical value used to measure the degree of uniformity of the profile deformation amplitude in space. By integrating the principal direction deformation parameter, the deformation amplitude values ​​of each region, and the spatial distribution uniformity index of deformation amplitude, a profile deformation parameter describing the change of profile shape can be constructed.

[0050] Step S264: Construct a statistical model of motion direction distribution based on contour deformation parameters, subdivide the direction intervals in the active motion region, merge the direction intervals in the gentle motion region, and generate motion direction features that include the proportion of direction and the rate of change of proportion.

[0051] The motion direction distribution statistical model is a model built based on contour deformation parameters for statistically analyzing the distribution of livestock motion directions. Active motion areas refer to regions where livestock move frequently and at high speeds; flat motion areas are regions where livestock move relatively slowly and are relatively still. Subdividing active motion areas into specific direction intervals allows for a more precise description of livestock motion directions within these areas; merging direction intervals in flat motion areas reduces data complexity. Direction percentage refers to the proportion of livestock movement within each direction interval; the percentage change rate refers to the rate at which the direction percentage changes over time. This step generates motion direction features that include both direction percentage and percentage change rate.

[0052] Step S265: Combining the motion direction characteristics and the temporal change rate of the physical region area parameter, predict the motion state within a preset time interval in the future, and generate a motion trend descriptor that includes the direction probability distribution, velocity change characteristics and prediction error range.

[0053] The movement direction feature contains information on the distribution and changes in the direction of livestock movement, while the temporal rate of change of the physical area parameter reflects the area change of livestock over time. Combining these two pieces of information, the movement trend of livestock within a preset time interval can be predicted. The direction probability distribution refers to the probability of livestock moving in various directions in the future; the velocity change characteristics refer to the trend of livestock speed changes, such as acceleration and deceleration; and the prediction error range refers to the range of uncertainty in the prediction result. The movement trend descriptor is a comprehensive feature that integrates the direction probability distribution, velocity change characteristics, and prediction error range to describe the future movement trend of livestock.

[0054] Step S266: Combine the contour deformation parameters and motion trend descriptions into a multi-dimensional motion representation vector with a unified dimension. Weight each feature dimension through an attention mechanism, and the weight values ​​are adaptively adjusted according to the contribution of the features in the historical recognition task.

[0055] Combining contour deformation parameters and motion trend descriptors into a unified multidimensional motion representation vector combines these two distinct features into a more comprehensive feature vector. Attention mechanisms automatically focus on important features and ignore secondary features, allowing for weighting of each feature dimension within the multidimensional motion representation vector. Adaptive adjustment of weights based on feature contribution in historical recognition tasks dynamically adjusts the weight of each feature in the multidimensional motion representation vector according to its impact on past livestock recognition tasks. For example, for each pig, the contour deformation parameters and motion trend descriptors are merged into a unified multidimensional motion representation vector. Attention-based neural networks (such as Transformer networks, which incorporate multi-head attention mechanisms) are used to weight each feature dimension. During training, the contribution of each feature dimension to the recognition task is recorded. Feature dimensions with higher contributions in historical recognition tasks are given higher weights, while those with lower contributions are given lower weights. Through continuous training and adjustment, the weights adaptively reflect the importance of the features.

[0056] Step S300: Perform adaptive multi-scale feature enhancement on the multi-dimensional motion representation vector, decouple the pixel-level features of the livestock herd region, and output the morphology-motion joint encoding sequence for each individual livestock.

[0057] Adaptive multi-scale feature enhancement is a method that automatically adjusts feature enhancement strategies based on the characteristics of multi-dimensional motion representation vectors and the requirements of different scales. This method can highlight important features in multi-dimensional motion representation vectors and improve feature expressiveness. Livestock herding areas are regions where multiple livestock are gathered together. In these areas, pixel-level features are intertwined, making it difficult to distinguish the features of each individual livestock. Decoupling pixel-level features refers to using algorithms (such as deep learning-based decoupling networks) to separate these intertwined features, enabling accurate identification of the features of each individual livestock. A morphological-motor joint coding sequence is a coding sequence that combines the morphological features (such as shape and size) and motion features (such as direction and speed) of an individual livestock, providing a more comprehensive description of the state of each individual livestock.

[0058] In one implementation, step S300 can be implemented as the following steps S310~S360:

[0059] Step S310: Input the multi-dimensional motion representation vector into the multi-scale feature extraction network, and process it in parallel through the feature extraction paths of different receptive fields to generate a multi-scale feature map set containing low-scale detail features, medium-scale structural features and high-scale semantic features.

[0060] Multi-scale feature extraction networks are neural networks specifically designed to extract features at different scales, incorporating multiple feature extraction paths with different receptive fields. A receptive field refers to the size of the input image region that each neuron in a convolutional neural network focuses on. By processing feature extraction paths with different receptive fields in parallel, features at different scales can be extracted simultaneously from multi-dimensional motion representation vectors. Low-scale detail features reflect subtle information in an image, such as the hair and texture of livestock; mid-scale structural features describe the overall structure of objects in an image, such as the body outline and limb proportions of livestock; high-scale semantic features have a higher level of abstraction and reflect the semantic information of the image, such as the species and behavior of livestock. A multi-scale feature map set is a collection of low-scale, mid-scale, and high-scale feature maps, comprehensively describing the feature information of multi-dimensional motion representation vectors. For example, a multi-scale feature extraction network based on the ResNet architecture can be used, inputting multi-dimensional motion representation vectors into the network. The network contains convolutional kernels of different sizes, each corresponding to a different receptive field. Through parallel processing, low-scale detail feature maps (such as those extracted using 3×3 convolution kernels), medium-scale structural feature maps (such as those extracted using 5×5 convolution kernels), and high-scale semantic feature maps (such as those extracted using 7×7 convolution kernels) are generated, forming a multi-scale feature map set.

[0061] Step S320: Perform feature fusion processing on the multi-scale feature map set. Based on the attention-guided weighted fusion strategy, calculate the regional saliency weights of the feature maps at each scale and perform weighted summation to generate a fused feature map that retains multi-scale information. During the fusion process, the original features are introduced through residual connections to enhance the ability to preserve details.

[0062] Feature fusion processing combines feature maps of different scales from a multi-scale feature map set to fully utilize the information from each scale. Attention-guided weighted fusion is a method that assigns weights based on the saliency of regions in the feature maps. This strategy calculates the regional saliency weights of each scale's feature map. These regional saliency weights reflect the importance of each region in the feature map. Weighted summation adds the feature maps from each scale according to their regional saliency weights, resulting in a fused feature map. A fused feature map that preserves multi-scale information contains low-scale detail, mid-scale structural information, and high-scale semantic information.

[0063] In one implementation, step S320 can be implemented as steps S321 to S325:

[0064] Step S321: Sort the multi-scale feature map set according to the scale level, perform regional saliency analysis on each scale feature map, and generate a regional saliency weight map that matches the size of the feature map through the class activation mapping method. The weight value reflects the contribution of the region to the livestock identification task.

[0065] Sorting by scale hierarchy arranges the feature maps in a multi-scale feature map set in the order of low-scale, medium-scale, and high-scale. Region saliency analysis assesses the importance of different regions in each scale feature map. Class activation mapping (Grad-CAM) methods can be used to generate region saliency weight maps that match the feature map sizes. The weight values ​​reflect the contribution of the region to the livestock identification task; higher weight values ​​indicate greater importance in livestock identification. For example, for low-scale detail feature maps, medium-scale structural feature maps, and high-scale semantic feature maps in a multi-scale feature map set, they are first sorted by scale hierarchy. Then, a class activation mapping method (such as Grad-CAM) is used to perform region saliency analysis on each scale feature map. For each feature map, the activation value of each region is calculated, and the activation value is converted into a weight value to generate a region saliency weight map.

[0066] Step S322: Perform size alignment processing on feature maps of different scales. Use bilinear interpolation to upsample the semantic feature maps output by the deep layers of the network to the same spatial size as the detail feature maps output by the shallow layers of the network. Simultaneously adjust the size of the corresponding saliency weight maps. Normalize the saliency weight maps at each scale. Use a soft maximization function to map the weight values ​​to a preset numerical range to obtain normalized weight maps.

[0067] Size alignment adjusts feature maps at different scales to the same spatial size for subsequent fusion operations. Bilinear interpolation is a commonly used image upsampling method that upsamples semantic feature maps from deep layers of the network to the same spatial size as detail feature maps from shallow layers. Synchronous adjustment of the corresponding saliency weight map means adjusting the size of the feature map and its corresponding saliency weight map simultaneously. Normalization adjusts the weight values ​​of saliency weight maps at different scales to ensure they have the same range and distribution. A soft maximization function maps weight values ​​to a preset numerical range (e.g., [0,1]), resulting in a normalized weight map.

[0068] Step S323: Perform a weighted summation operation on the size-aligned feature maps based on the normalized weight map to generate a preliminary fused feature map. During the weighting process, the weight allocation ratio is dynamically adjusted according to the scale level of the feature maps. The semantic feature maps output by the deep layers of the network receive higher weights in regions rich in semantic information, while the detail feature maps output by the shallow layers of the network receive higher weights in regions rich in detail features.

[0069] The weighted summation operation involves summing size-aligned feature maps according to normalized weight maps to obtain a preliminary fused feature map. Dynamically adjusting the weight allocation ratio based on the scale level of the feature maps means assigning different weights to feature maps at different scales during the weighting process, taking into account their characteristics. Semantic feature maps output from deeper layers of the network receive higher weights in semantically rich regions (such as regions that can distinguish livestock species), because these regions are crucial for identifying livestock species and behaviors. Detail feature maps output from shallower layers receive higher weights in regions rich in detail features (such as livestock hair and texture), because these regions provide more detailed information. For example, a weighted summation operation is performed on size-aligned low-scale detail feature maps, mid-scale structural feature maps, and high-scale semantic feature maps, along with their corresponding normalized weight maps. In regions rich in semantic information, the weight of the high-scale semantic feature map is increased; in regions rich in detail features, the weight of the low-scale detail feature map is increased. Through weighted summation, a preliminary fused feature map is generated, which contains both rich semantic information and retains a certain amount of detail information.

[0070] Step S324: Construct a multi-path residual connection structure, extract edge features and texture features from the original multi-dimensional motion representation vector, map them to the same dimension as the preliminary fused feature map through a feature transformation layer, and superimpose the features with the preliminary fused feature map through a skip connection method to obtain the superimposed fused feature map.

[0071] Multi-path residual connection is a connection method in neural networks that involves residual connections across multiple paths. Extracting edge and texture features from the original multi-dimensional motion representation vector can be achieved using feature extraction algorithms (such as the Sobel operator for edge features and Gabor filters for texture features). A feature transformation layer maps features to a specified dimension, allowing the extracted edge and texture features to be mapped to the same dimension as the initial fused feature map. Skip connections directly connect the original features to subsequent layers, superimposing them with the initial fused feature map. This method yields a superimposed fused feature map, enhancing its detail preservation. For example, a multi-path residual connection structure can be constructed, using the Sobel operator to extract edge features from the original multi-dimensional motion representation vector and the Gabor filter to extract texture features. Then, a fully connected layer is used as a feature transformation layer to map the edge and texture features to the same dimension as the initial fused feature map. Skip connections then superimpose these features with the initial fused feature map to obtain the superimposed fused feature map.

[0072] Step S325: Perform feature calibration processing on the superimposed fused feature map, dynamically select feature channels through the channel attention mechanism, enhance the channel response containing key livestock features, suppress the interference of background noise channels, and generate the final multi-scale fused feature map.

[0073] Feature calibration further optimizes and adjusts the superimposed fused feature map. Channel attention is a mechanism for automatically focusing on important feature channels and ignoring secondary ones, allowing for dynamic selection of feature channels. Enhancing the response of channels containing key livestock features means increasing the response values ​​of feature channels important for livestock identification; suppressing background noise channels means reducing the response values ​​of feature channels that are background-related and unhelpful for livestock identification. The final multi-scale fused feature map is one that, after feature calibration, retains multi-scale information while highlighting key livestock features. For example, a module based on channel attention (such as the CBAM module) can be used to perform feature calibration on the superimposed fused feature map. This module calculates the importance weight of each feature channel and dynamically selects the feature channels based on the weight. For channels containing key livestock features, their weights are increased to enhance their response; for background noise channels, their weights are decreased to suppress interference. In this way, the final multi-scale fused feature map is generated.

[0074] Step S330: Construct a dual-channel attention mechanism module, calculate attention weights from the spatial dimension and the feature channel dimension respectively, and generate an attention-enhanced feature map that can highlight the key areas and important feature channels of livestock.

[0075] The dual-channel attention mechanism module comprises two branches: spatial attention and channel attention. Calculating attention weights from the spatial dimension involves evaluating the importance of different spatial locations in the feature map and determining the attention weight for each location. Calculating attention weights from the channel dimension involves evaluating the importance of different feature channels in the feature map and determining the attention weight for each channel. Through these two dimensions of attention calculation, attention-enhanced feature maps that highlight key regions and important feature channels of livestock can be generated. For example, a dual-channel attention mechanism module based on a convolutional neural network can be constructed. In the spatial attention branch, convolutional layers are used to process the feature map and calculate the attention weight for each spatial location. In the channel attention branch, global average pooling and fully connected layers are used to process the feature map and calculate the attention weight for each feature channel. The attention weights from these two branches are multiplied to obtain the final attention weight. This attention weight is then applied to the original feature map to generate an attention-enhanced feature map.

[0076] Step S340: Identify livestock herding areas based on attention-enhanced feature maps, predict the individual density distribution of the herding areas through density estimation, determine the boundaries of the herding areas based on the density distribution characteristics, and generate a herding area mask.

[0077] Identifying livestock herding areas based on attention-enhanced feature maps involves locating regions of concentrated livestock within the attention-enhanced feature map. Density estimation is a method used to predict the density distribution of individuals within the herding area, providing an approximate number of livestock in each region. Determining the herding area boundary based on density distribution features involves identifying the edge locations of the herding area based on variations in the density distribution. A herding area mask is a binary image where pixels within the herding area have a value of 1, while pixels in the background have a value of 0, accurately identifying the location of livestock herding areas.

[0078] In one implementation, step S340 can be implemented as the following steps S341~S346:

[0079] Step S341: Input the attention-enhanced feature map into the density estimation network, and perform multi-scale density prediction through an encoder-decoder structure. The encoder part uses progressive downsampling to extract contextual features, and the decoder part fuses feature information from different levels through skip connections to generate a density prediction map with the same size as the input feature map.

[0080] A density estimation network is a neural network used to predict the density distribution of individuals within a cluster region, employing an encoder-decoder structure. The encoder, the first half of the network, extracts contextual features through progressive downsampling (e.g., using convolutional and pooling layers). Contextual features refer to the association information between different regions in an image. The decoder, the second half of the network, fuses feature information from different levels of the encoder through skip connections, restoring the spatial size of the feature map. This structure enables multi-scale density prediction, generating a density prediction map with the same size as the input feature map. For example, using a density estimation network based on the U-Net architecture, an attention-enhanced feature map is input. In the encoder, progressive downsampling using convolutional and pooling layers extracts contextual features at different scales. In the decoder, skip connections fuse feature information from different levels of the encoder with the features of the current layer, performing an upsampling operation. The final result is a density prediction map with the same size as the input feature map, where the value of each pixel represents the livestock density at that location.

[0081] Step S342: Post-processing optimization of the density prediction map is performed. Adaptive Gaussian kernel smoothing is used to eliminate density prediction noise. The kernel size is dynamically adjusted according to the estimated livestock size. Larger livestock correspond to larger kernel sizes, and smaller livestock correspond to smaller kernel sizes.

[0082] Post-processing optimization involves further processing the density prediction map to improve its quality. Adaptive Gaussian kernel smoothing is a method that uses a Gaussian kernel function to smooth the image, eliminating noise in the density prediction map. Dynamic kernel size adjustment based on estimated livestock size refers to selecting an appropriate Gaussian kernel size according to the size of the livestock. Larger livestock require larger kernel sizes for smoothing to avoid over-smoothing and loss of detail; smaller livestock require smaller kernel sizes.

[0083] Step S343: Analyze the spatial distribution characteristics of the optimized density prediction map, calculate the spatial gradient change of the density value, mark the regions with drastic gradient changes as potential cluster boundaries, and generate a cluster boundary candidate map.

[0084] Analyzing the spatial distribution characteristics of the optimized density prediction map refers to studying the spatial distribution of density values ​​in the density prediction map. The spatial gradient change of density values ​​refers to the rate of change of density values ​​between adjacent pixels. Regions with drastic gradient changes are marked as potential cluster boundaries because density values ​​typically change significantly at the boundaries of clustered regions. The cluster boundary candidate map is a binary image where regions with drastic gradient changes are marked as 1, and other regions are marked as 0, serving as a reference for determining cluster boundaries. For example, for the optimized density prediction map, the Sobel operator is used to calculate the spatial gradient change of density values. Regions with gradient changes exceeding a certain threshold are marked as potential cluster boundaries. These marking results are then displayed as a binary image, generating the cluster boundary candidate map.

[0085] Step S344: Perform region growth based on density values ​​and boundary information, starting from the density peak point and expanding the region outwards until the density gradient threshold or boundary candidate region is encountered, generating the initial cluster region.

[0086] Region growing is an image segmentation method based on pixel similarity. Starting from a seed point, it continuously expands the region outwards until certain conditions are met. Region growing based on density values ​​and boundary information involves combining density values ​​from the density prediction map with information from the cluster boundary candidate map, starting the region expansion from the density peak point. The density peak point is the point with the highest density value in the density prediction map, usually representing the area where livestock are most densely clustered. Expanding the region outwards means starting from the density peak point and sequentially checking the density values ​​and boundary information of its neighboring pixels. When a density gradient threshold (i.e., a region with excessively large density value changes) or a boundary candidate region is encountered, the expansion stops, and the initial cluster region is generated. For example, for the optimized density prediction map and cluster boundary candidate map, all density peak points are identified. Starting from one of the density peak points, the region is expanded outwards to its surrounding neighboring pixels. The density values ​​of the neighboring pixels and their density gradients with the current region are checked. If the density gradient exceeds the threshold or a boundary region in the cluster boundary candidate map is encountered, the expansion stops. This process is repeated until all density peak points have undergone region growing, generating the initial cluster region.

[0087] Step S345: Perform morphological optimization on the initial cluster region, and perform region merging and splitting operations. Based on the comparison results between the distance between the centroids of the cluster regions in the density map and the preset distance threshold, merge small clusters with a distance less than the preset distance threshold and continuous density distribution. Calculate the rate of change of the density gradient inside the cluster region. When the rate of change exceeds the preset gradient threshold, it is determined to be a density distribution discontinuity. Split large clusters with density distribution discontinuities to generate the optimized cluster region.

[0088] Morphological optimization further adjusts and optimizes the initial cluster region to more accurately represent the livestock cluster area. Region merging merges small clusters that are close together and have continuous density distribution into a larger cluster; region splitting splits large clusters with discontinuous density distribution into multiple smaller clusters. The centroid of a cluster region in the density map refers to the geometric center of each cluster region. By comparing the distance between centroids with a preset distance threshold, it can be determined whether two clusters should be merged. Calculating the rate of change of the density gradient within a cluster region involves calculating the density gradient of each pixel within the cluster region and statistically analyzing its rate of change. When the rate of change exceeds a preset gradient threshold, it is considered a density distribution discontinuity, requiring a splitting operation.

[0089] Step S346: Convert the optimized cluster region into a binary mask image, set the pixel value within the cluster region to 1, and the background region to 0, to generate a cluster region mask containing complete cluster boundary information.

[0090] Converting the optimized clump region to a binary mask represents the optimized clump region as a binary image. Pixel values ​​within the clump region are set to 1, and background pixels are set to 0, clearly identifying the boundaries of the clump region. A clump region mask containing complete clump boundary information is a binary image that accurately represents the location and boundaries of the livestock clump region, which can be used for subsequent individual boundary delineation and livestock identification tasks. For example, for the optimized clump region, a binary image of the same size as the original image is created. Pixel values ​​within the clump region are set to 1, and background pixels are set to 0. In this way, a clump region mask containing complete clump boundary information is generated. In this mask image, the boundaries of the livestock clump region are clearly displayed.

[0091] Step S350: Perform individual boundary segmentation on the cluster region mask, optimize the selection of individual centers and separate the sticky regions based on the clustering algorithm, and generate an accurate region mask for each individual livestock.

[0092] Individual boundary segmentation further divides the clustered regions in the clustered region mask into regions for each individual livestock. Clustering algorithms are used to group data points into different clusters; optimizing individual center selection based on clustering algorithms means finding the center position of each individual livestock using clustering algorithms and optimizing it. Separating adhered regions refers to separating livestock individuals that are stuck together. A precise region mask is a binary image where the region of each individual livestock is accurately identified, with a pixel value of 1, while the background region has a pixel value of 0.

[0093] Step S360: Based on the precise region mask, extract the morphological and motion features of the corresponding region from the attention-enhanced feature map. After unifying the dimensions of features at different scales through feature mapping, perform temporal encoding to generate a morphological-motion joint encoding sequence containing individual spatiotemporal features.

[0094] Extracting morphological and kinematic features from attention-enhanced feature maps based on precise region masks refers to extracting the feature information of each individual livestock from the attention-enhanced feature map based on the region of each individual livestock in the precise region mask. Morphological features refer to the livestock's outline, size, and other characteristics; kinematic features refer to the livestock's direction of movement, speed, and other characteristics. Feature mapping is a method that transforms features at different scales into the same dimension, unifying morphological and kinematic features of different scales into a single dimension. Temporal coding encodes the unified features in chronological order, generating a morphological-kinematic joint encoding sequence containing the spatiotemporal features of an individual. For example, for the precise region mask of each individual livestock, feature information of the corresponding region is extracted from the attention-enhanced feature map. Principal component analysis is used to perform feature mapping on these features, unifying features at different scales to the same dimension. Then, the unified features are arranged in chronological order and temporally encoded using a recurrent neural network (such as an LSTM network). Finally, a morphological-kinematic joint encoding sequence containing the spatiotemporal features of an individual is generated.

[0095] Step S400: Construct a spatiotemporal correlation matrix based on the morphology-motor joint coding sequence, extract the topological features of livestock movement trajectories through a lightweight feature extraction network, and generate a global ID mapping table containing unique individual identifiers.

[0096] Constructing a spatiotemporal correlation matrix based on morpho-motor joint coding sequences involves calculating the degree of correlation between different individuals at different time points based on the morpho-motor joint coding sequence of each individual livestock, and constructing a matrix to represent this correlation. The rows and columns of the spatiotemporal correlation matrix correspond to different individuals and time points, respectively, and the elements in the matrix represent the correlation strength of the corresponding individuals at the corresponding time points. A lightweight feature extraction network is a relatively simple neural network with low computational cost. This network can extract the topological features of livestock movement trajectories. Topological features refer to features describing the structure and shape of livestock movement trajectories, such as trajectory continuity and branching. A global ID mapping table is a table containing a unique identifier for each individual livestock individual, which can be used to track and identify each individual livestock.

[0097] In one implementation, step S400 can be implemented as the following steps S410~S450:

[0098] Step S410: Perform temporal dimension synchronization processing on the morphology-motion joint coding sequence in consecutive video frames, and establish a temporal correlation index for cross-frame coding sequences. The index value is generated according to the timestamp order of the video frames to ensure the temporal correlation of adjacent frame coding sequences.

[0099] Temporal synchronization processing arranges and aligns the morphology-motor joint coding sequences in consecutive video frames according to their temporal order, ensuring correct temporal correlation between coding sequences in different frames. Establishing a temporal correlation index for cross-frame coding sequences involves assigning a time-related index to each coding sequence, with the index value generated according to the timestamp order of the video frames. This ensures that the coding sequences of adjacent frames are temporally continuous, facilitating subsequent feature matching and trajectory tracking. For example, in monitoring videos of a pig farm, for each pig's morphology-motor joint coding sequence, a unique index value is assigned to each coding sequence based on the timestamp of the video frame. If the video frame rate is 25 frames per second, then the time interval between two adjacent frames is 0.04 seconds, and the corresponding coding sequence indexes will increase sequentially according to this temporal order. Through this temporal synchronization processing and the establishment of temporal correlation indexes, the movement state and morphological feature changes of each pig at different points in time can be accurately reflected.

[0100] Step S420: Calculate the feature similarity between adjacent frame morphology-motion joint coding sequences, adjust the weight contribution of historical frame features by combining the time decay factor, and generate a dynamically updated inter-frame similarity matrix.

[0101] Calculating the feature similarity between morphological-motion joint coding sequences of adjacent frames is the process of measuring the similarity between the coding sequences of the same animal in two adjacent frames. Methods such as cosine similarity and Euclidean distance can be used for calculation. The time decay factor is a parameter used to adjust the weights of features from historical frames; as time progresses, the influence of features from historical frames on the current frame gradually decreases. Adjusting the weight contribution of features from historical frames using the time decay factor can more accurately reflect the current motion state of the animal. The dynamically updated inter-frame similarity matrix is ​​a matrix that records the similarity of coding sequences between adjacent frames and is continuously updated as the video frames progress.

[0102] Step S430: Construct a spatiotemporal correlation matrix based on the inter-frame similarity matrix. The row dimension of the matrix corresponds to the encoding sequence index of the previous frame, the column dimension corresponds to the encoding sequence index of the current frame, and the matrix element values ​​represent the correlation strength of the corresponding encoding sequence pairs. The matrix content is dynamically updated through a time sliding window mechanism.

[0103] Constructing a spatiotemporal correlation matrix based on the inter-frame similarity matrix extends inter-frame similarity information to the spatiotemporal dimension, establishing correlations between encoded sequences at different time points. The matrix's rows correspond to the encoded sequence indices of the previous frame, and the columns correspond to the encoded sequence indices of the current frame. Matrix element values ​​represent the correlation strength between corresponding encoded sequence pairs, which can be adjusted based on inter-frame similarity. The time-sliding window mechanism considers only a certain number of frames near the current time point when processing video sequences. As time progresses, the window slides forward, dynamically updating the matrix content. In constructing the spatiotemporal correlation matrix, the encoded sequence indices of the previous frame are used as rows, and the encoded sequence indices of the current frame are used as columns. Matrix element values ​​are calculated and filled based on the similarity values ​​in the inter-frame similarity matrix. As the video plays, the time-sliding window moves forward, and the matrix content is updated accordingly to reflect the latest spatiotemporal correlation information.

[0104] Step S440: Input the spatiotemporal correlation matrix into the lightweight graph convolutional network. The lightweight graph convolutional network includes a local feature extraction layer and a global feature aggregation layer. The local feature extraction layer captures the local correlation features of adjacent nodes through graph convolution operations. The global feature aggregation layer models long-distance node dependencies through a self-attention mechanism to generate trajectory topology features that fuse local and global information.

[0105] The spatiotemporal correlation matrix is ​​input into a lightweight graph convolutional network to further extract the topological features of livestock movement trajectories. Lightweight graph convolutional networks are neural networks used to process graph-structured data, characterized by high computational efficiency and few parameters. The local feature extraction layer extracts features from neighboring nodes in the spatiotemporal correlation matrix through graph convolution operations, capturing local correlation information.

[0106] In one implementation, step S440 can be implemented as steps S441 to S446:

[0107] Step S441: Convert the spatiotemporal correlation matrix into a graph structure data representation. Each element in the matrix corresponds to the edge weight in the graph structure, the row and column indices correspond to the graph nodes, and the node feature vector is composed of the corresponding morphology-motion joint encoding sequence.

[0108] Converting a spatiotemporal correlation matrix into a graph-structured data representation involves transforming matrix-based data into a graph format for processing using graph convolutional networks. In a graph structure, each element in the matrix corresponds to an edge weight, representing the strength of the association between nodes. Row and column indices correspond to nodes in the graph, and each node represents an encoded sequence. The node feature vector is composed of the corresponding morphological-motor joint encoding sequence, thus incorporating the feature information of the encoded sequence into the graph structure. For example, for a spatiotemporal correlation matrix, converting it into a graph structure allows each element's value to serve as an edge weight in the graph; a larger element value indicates a stronger association between the corresponding two nodes. The row and column indices of the matrix correspond to nodes in the graph, and the feature vector of each node is composed of the morphological-motor joint encoding sequence corresponding to that node. This conversion method effectively represents the information of the spatiotemporal correlation matrix as graph-structured data, providing a foundation for subsequent graph convolutional operations.

[0109] Step S442: Construct a two-layer graph convolutional network as a local feature extraction layer. The first graph convolutional layer adopts a fixed-size neighborhood aggregation method to aggregate the direct neighbor features of each node. The second graph convolutional layer adopts an adaptive neighborhood aggregation method to dynamically adjust the neighborhood size according to the node similarity and generate local topological features containing local context information.

[0110] Constructing a two-layer graph convolutional network as a local feature extraction layer aims to more comprehensively extract local features of nodes in the graph structure. The first graph convolutional layer employs a fixed-size neighborhood aggregation method, meaning that for each node, only the features of its direct neighbors within a fixed range are considered and aggregated. This method can quickly obtain local information about nodes. The second graph convolutional layer uses an adaptive neighborhood aggregation method, dynamically adjusting the neighborhood size based on the similarity between nodes. For nodes with high similarity, the neighborhood range is expanded to obtain more relevant information; for nodes with low similarity, the neighborhood range is reduced. This approach allows for more flexible extraction of local features.

[0111] Step S443: Input local topological features into the multi-head self-attention mechanism layer as a global feature aggregation layer. Calculate long-distance dependencies between nodes through multiple parallel attention heads. Each attention head focuses on different types of trajectory association patterns to generate multi-view global association features.

[0112] Inputting local topological features into a multi-head self-attention mechanism layer as a global feature aggregation layer aims to capture long-distance dependencies between nodes in a graph structure. The multi-head self-attention mechanism computes the correlations between nodes using multiple parallel attention heads. Each attention head can focus on different types of trajectory association patterns; for example, some attention heads focus on temporal sequence associations between nodes, while others focus on spatial location associations. Through parallel computation of multiple attention heads, global association information between nodes can be obtained from multiple perspectives. For example, in a graph structure containing multiple livestock movement trajectories, local topological features are input into the multi-head self-attention mechanism layer. Assuming there are four attention heads, the first attention head focuses on the temporal sequence associations of nodes, the second attention head focuses on the spatial proximity of nodes, the third attention head focuses on the similarity associations of node features, and the fourth attention head focuses on the associations of node movement speeds. Each attention head independently computes the correlations between nodes, generating its own association features. Combining the output features of these four attention heads yields a multi-perspective global association feature.

[0113] Step S444: The output features of multiple attention heads are concatenated, and the concatenated features are mapped to a fixed dimension through a feature transformation layer. Layer normalization is used to stabilize the feature distribution and generate global topological features containing global structural information.

[0114] Concatenating the output features of multiple attention heads merges the features calculated by each attention head to integrate information from multiple perspectives. Mapping the concatenated features to a fixed dimension using a feature transformation layer unifies the feature dimensions, facilitating subsequent processing. Layer normalization is a method for stabilizing feature distribution, ensuring similar distributions across different samples and layers, improving model stability and training efficiency. For example, features from four attention heads output by a multi-head self-attention mechanism layer are concatenated along their feature dimensions. Then, a fully connected layer is used as a feature transformation layer to map the concatenated features to a fixed dimension, such as 128 dimensions. Next, layer normalization is performed on the mapped features, calculating the mean and standard deviation for each feature dimension and normalizing the feature values. Through these operations, global topological features containing global structural information are generated, accurately reflecting the global relationships and structural information between nodes in the graph structure.

[0115] Step S445: The local topological features and global topological features are fused through residual connections. The fusion ratio of the two features is dynamically adjusted through a gating mechanism to enhance the weight of local features in dense trajectory regions and enhance the weight of global features in sparse trajectory regions.

[0116] Fusing local and global topological features via residual connections aims to fully utilize both local and global information. Residual connections make it easier for the model to learn feature differences during training, avoiding the vanishing gradient problem. In regions with dense trajectories, local features provide more detailed information, thus increasing their weights; in regions with sparse trajectories, global features provide more macroscopic information, thus increasing their weights. By combining residual connections with gating mechanisms, effective fusion of local and global topological features is achieved, generating a more comprehensive feature representation.

[0117] Step S446: Perform nonlinear transformation on the fused features, enhance the feature representation ability through activation function, suppress noise feature interference, and generate trajectory topological feature vector.

[0118] Applying nonlinear transformations to the fused features enhances their expressive power. Activation functions, being nonlinear functions, can nonlinearly map input features, enabling the model to learn more complex feature relationships. Activation functions can enhance feature expressiveness while suppressing interference from noisy features. For example, the ReLU activation function can be used to process the fused features. This effectively removes some noisy features while enhancing the expression of useful features. After activation function processing, a trajectory topological feature vector is generated. This vector accurately represents the topological features of the livestock's movement trajectory, providing strong support for subsequent trajectory association and recognition tasks.

[0119] Step S450: Perform trajectory segment association processing on trajectory topology features. Adopt a trajectory matching strategy based on the Hungarian algorithm, and combine trajectory position continuity constraints and feature similarity constraints to associate the encoded sequences in continuous frames into complete trajectories. Assign a unique identifier code to each complete trajectory and generate a global ID mapping table containing information on currently active trajectories and historical trajectories.

[0120] Trajectory fragment association processing, which involves connecting trajectory fragments detected in different frames into complete trajectories, utilizes the topological features of the trajectories. A trajectory matching strategy based on the Hungarian algorithm is employed to find the optimal matching scheme. The Hungarian algorithm is used to solve the maximum weight matching problem in bipartite graphs, and it can quickly find the optimal match while ensuring matching quality. Combining trajectory position continuity constraints and feature similarity constraints improves the accuracy of trajectory matching. The trajectory position continuity constraint requires that the matched trajectories have continuity in position, without significant jumps; the feature similarity constraint requires that the matched trajectories have high similarity in features. After associating the coded sequences in consecutive frames into complete trajectories, a unique identifier code is assigned to each complete trajectory for easy tracking and management. A global ID mapping table containing information on currently active trajectories and historical trajectories is generated. This table records information on all trajectories, including their identifier code, start time, end time, and location information.

[0121] In one implementation, step S450 can be implemented as steps S451 to S457:

[0122] Step S451: Initialize the trajectory cache list to store the trajectory information of the livestock currently being tracked. Each trajectory information includes a historical morphology-motion joint coding sequence, timestamp record, location coordinate sequence and motion state parameters.

[0123] Initializing the trajectory cache list is to create a container for storing the trajectory information of the currently tracked individual livestock. Each trajectory information includes a historical morphology-motor joint encoding sequence, which records the morphology and movement characteristics of the livestock at different time points; a timestamp records the specific time of the trajectory at each time point; a position coordinate sequence records the position information of the livestock at different time points; and motion state parameters record information such as the livestock's movement speed and direction.

[0124] Step S452: Detect the starting point of the trajectory topology features. By analyzing the continuity and integrity of the feature sequence, mark the trajectory segment that appears for the first time and meets the duration condition as a new trajectory. Allocate the smallest unused integer from the identifier coding pool as the unique identifier code of the new trajectory.

[0125] Trajectory start-point detection, performed on trajectory topology features, aims to determine the origin of new trajectories. By analyzing the continuity and completeness of the feature sequence, it can be determined whether a trajectory segment is a new trajectory start-point. Continuity requires the trajectory to be consistent in time and location without interruption; completeness requires the trajectory to contain sufficient feature information. The first occurrence of a trajectory segment that meets a duration condition is marked as a new trajectory. The duration condition can be set according to actual conditions, such as requiring the trajectory segment to last at least 3 frames. The smallest unused integer is allocated from the identifier coding pool as a unique identifier code for the new trajectory. The identifier coding pool is a collection storing unused identifier codes; the smallest integer is selected each time a code is allocated.

[0126] Step S453: Calculate the matching degree between the existing trajectories in the trajectory cache list and the current frame morphology-motion joint coding sequence. The matching degree is determined by the cosine similarity of trajectory topological features, the Euclidean distance of position coordinates, and the consistency of motion direction, and a comprehensive matching degree matrix is ​​generated.

[0127] Calculating the matching degree between existing trajectories in the trajectory cache list and the current frame's morphology-motion joint coding sequence aims to find the best match between the current frame's coding sequence and existing trajectories. The matching degree is determined by the cosine similarity of trajectory topological features, the Euclidean distance of position coordinates, and the consistency of motion direction. The cosine similarity of trajectory topological features measures the similarity between two trajectories in terms of features; a value closer to 1 indicates greater similarity. The Euclidean distance of position coordinates measures the distance between two trajectories in terms of position; a smaller value indicates greater similarity. The consistency of motion direction measures whether two trajectories move in the same direction. These three factors are considered together to generate a comprehensive matching degree matrix. For example, for each existing trajectory in the trajectory cache list, its matching degree with each coding sequence in the current frame is calculated. All matching degree values ​​are then filled into the comprehensive matching degree matrix, where rows correspond to existing trajectories and columns correspond to the current frame's coding sequence.

[0128] Step S454: Perform the Hungarian matching algorithm based on the matching degree matrix to find the best match between the current frame encoding sequence and the trajectory in the trajectory cache list. Update the successfully matched trajectory, including appending the current frame encoding sequence, updating the timestamp record and motion state parameters.

[0129] The Hungarian matching algorithm, based on the matching degree matrix, aims to find the optimal matching scheme between the current frame's encoded sequence and the trajectories in the trajectory cache list. The Hungarian algorithm can quickly find the optimal match while ensuring matching quality. After performing the optimal matching between the current frame's encoded sequence and the trajectories in the trajectory cache list, the successfully matched trajectories need to be updated. Appending the current frame's encoded sequence involves adding the current frame's encoded sequence to the list of existing trajectory encoded sequences to record the latest state of the trajectory; updating the timestamp record involves adding the current frame's timestamp to the trajectory's timestamp list to record the trajectory's time information; and updating the motion state parameters involves updating the trajectory's speed, direction, and other parameters based on the motion information of the current frame.

[0130] Step S455: Perform new trajectory determination on the current frame encoding sequence that has not been matched. If the new trajectory conditions are met, assign a new identifier encoding. If the conditions are not met, mark it as a temporary trajectory segment and store it in the cache.

[0131] The purpose of determining new trajectories for unmatched encoded sequences in the current frame is to ascertain whether these unmatched sequences represent new trajectories. New trajectory criteria are assigned new identifier codes, which may include trajectory duration, feature completeness, etc. Trajectories that do not meet the criteria are marked as temporary trajectory fragments and stored in a buffer. Temporary trajectory fragments may be those that cannot be matched with existing trajectories temporarily due to occlusion, noise, or other reasons. These are stored in the buffer for further processing. This method allows for the appropriate handling of unmatched encoded sequences, preventing the omission of new trajectories.

[0132] Step S456: When the trajectory fails to match successfully in multiple consecutive frames, predict the possible future location area and morphological features based on the historical trajectory topology features and motion state parameters, and set the longest disappearance time threshold to control the trajectory lifecycle.

[0133] When a trajectory fails to match within multiple consecutive frames, it indicates that the trajectory may have been temporarily lost due to occlusion, leaving the monitoring range, or other reasons. Predicting the potential future location and morphological features based on historical trajectory topological features and motion state parameters is crucial for continuing tracking even when a trajectory is lost. Methods such as Kalman filters can be used for prediction, using historical location and motion information to forecast future location and morphology. Setting a maximum disappearance time threshold controls the trajectory lifecycle to avoid prolonged, ineffective tracking of trajectories that have left the monitoring range. If a trajectory fails to match within the maximum disappearance time threshold, it is considered to have ended. This method effectively manages the trajectory lifecycle and improves the efficiency of trajectory tracking.

[0134] Step S457: For the trajectory cache list, terminate the trajectory that exceeds the longest disappearance time threshold, release the identifier code back to the identifier code pool, update the global ID mapping table, and record the identifier codes of all currently active livestock individuals and their latest status information.

[0135] For the trajectory cache list, terminating trajectories exceeding the longest disappearance time threshold is to clean up invalid trajectory information. Releasing the identifier code back to the identifier code pool means putting the used identifier code back into the identifier code pool so that it can be used again later. Updating the global ID mapping table records the trajectory termination information in the global ID mapping table, and at the same time records the identifier codes of all currently active livestock individuals and their latest status information. For example, in a fish farm monitoring system, the trajectories in the trajectory cache list are checked periodically. For trajectories exceeding the longest disappearance time threshold (e.g., 15 frames), they are deleted from the cache list, and the identifier code of the trajectory is released back to the identifier code pool. The global ID mapping table is updated, recording the termination time and status information of the trajectory in the table, and also recording the identifier codes, latest positions, and movement status of all currently active fish. In this way, the information in the global ID mapping table can be ensured to be up-to-date and accurate, facilitating real-time monitoring and management of livestock movement trajectories.

[0136] Step S500: Perform real-time statistics on the number of livestock based on the global ID mapping table, and conduct behavioral tracing analysis on the individual livestock corresponding to abnormal movement trajectories to obtain the statistical results of the number of livestock and the results of abnormal behavior identification.

[0137] Real-time livestock counting based on a global ID mapping table determines the real-time livestock population by counting the number of currently active individual livestock identifiers in the table. Behavioral tracing analysis of livestock with abnormal movement trajectories aims to identify those with abnormal movement patterns and analyze the causes and processes of their abnormal behavior. The results yield both livestock count statistics and abnormal behavior identification results. The count statistics reflect the actual current livestock population, while the abnormal behavior identification results help managers promptly identify livestock health and safety issues.

[0138] In one implementation, step S500 can be implemented as follows: steps S510 to S560:

[0139] Step S510: Traverse the active identifier code records in the global ID mapping table, count the total number of identifier codes at the current time, generate basic livestock quantity statistics, and synchronously record the first appearance time and the latest update time of each identifier code.

[0140] Traversing the active identifier code records in the global ID mapping table is to obtain the identifier codes of all currently active livestock individuals. The total number of identifier codes at the current moment is counted and used as the basic livestock quantity statistic, reflecting the actual number of livestock at the current moment. The first appearance time and the latest update time of each identifier code are recorded synchronously. The first appearance time records when the livestock entered the monitoring range, and the latest update time records when the livestock was last detected.

[0141] Step S520: Perform multi-window time smoothing on the basic livestock quantity statistics, construct two sliding windows: a short-term window containing the most recent N frames and a long-term window containing the most recent M frames. Calculate the average quantity value within each of the two windows, and dynamically adjust the weight ratio of the two windows according to the scene to generate stable quantity statistics results.

[0142] Multi-window temporal smoothing of basic livestock population statistics aims to reduce fluctuations caused by noise, occlusion, and other factors. Two sliding windows are constructed: a short-term window containing the most recent N frames (e.g., frames 10-20) and a long-term window containing the most recent M frames (e.g., frames 50-100). The short-term window reflects recent rapid changes in livestock population, while the long-term window reflects long-term trends. The average population value within each window is calculated, and the weight ratio of the two windows is dynamically adjusted based on the scene to generate stable population statistics.

[0143] Step S530: Extract the complete motion trajectory data of each individual animal from the global ID mapping table, including the position coordinates of each frame, motion direction vector, velocity features and morphological feature sequence, and perform timestamp alignment processing on the trajectory data.

[0144] Extracting complete motion trajectory data for each individual livestock from the global ID mapping table is crucial for obtaining detailed motion information throughout the monitoring process. Frame position coordinates record the livestock's location at different time points; motion direction vectors record the direction of movement; velocity features record the livestock's speed; and morphological feature sequences record the livestock's morphological characteristics at different time points. Timestamp alignment of the trajectory data ensures temporal consistency, facilitating subsequent analysis and processing.

[0145] Step S540: Construct an abnormal trajectory detection model based on deep learning, convert motion trajectory data into trajectory feature vectors, reconstruct normal trajectory features through an autoencoder network, calculate the reconstruction error as a trajectory anomaly index, and comprehensively determine abnormal motion trajectories by combining the degree of trajectory deviation from the normal behavior pattern library.

[0146] The purpose of building a deep learning-based abnormal trajectory detection model is to automatically detect anomalies in motion trajectories. Converting motion trajectory data into trajectory feature vectors involves feature extraction and transformation of the original data for input into the model. Normal trajectory features are reconstructed using an autoencoder network, an unsupervised learning model that learns the latent representation of the data and attempts to reconstruct it. The reconstruction error is calculated as an indicator of trajectory anomaly; a larger reconstruction error indicates a greater difference between the trajectory and a normal trajectory, and a higher degree of anomaly. The degree to which the trajectory deviates from a normal behavior pattern database is combined to comprehensively determine abnormal motion trajectories. This database stores normal motion trajectory patterns; by comparing the trajectory with patterns in the database, anomaly can be determined.

[0147] In one implementation, step S540 can be implemented as the following steps S541~S547:

[0148] Step S541: Collect normal movement trajectory samples from historical monitoring data and construct a normal behavior pattern library. The samples cover normal movement trajectories under different time periods, different weather conditions, and different group densities.

[0149] Collecting samples of normal movement trajectories from historical monitoring data aims to obtain a large amount of normal movement trajectory data to construct a database of normal behavior patterns. The samples cover normal movement trajectories under different time periods, weather conditions, and population densities, ensuring the comprehensiveness and representativeness of the database. Movement trajectories at different time periods reflect livestock activity patterns at different times; movement trajectories under different weather conditions account for the impact of weather on livestock movement; and movement trajectories at different population densities reflect livestock behavior patterns in different environments.

[0150] Step S542: Extract features from normal motion trajectory samples, convert the position coordinate sequence into a relative displacement sequence, construct a multi-dimensional trajectory feature vector by combining motion direction, velocity and acceleration features, and standardize the feature vector.

[0151] Feature extraction from normal motion trajectory samples aims to extract useful features from the raw trajectory data. Converting the position coordinate sequence to a relative displacement sequence eliminates absolute position information, retaining only relative position changes, thus better reflecting the trajectory's motion pattern. A multi-dimensional trajectory feature vector is constructed by combining motion direction, velocity, and acceleration features; these features describe the trajectory's motion state from different perspectives. Standardizing the feature vector ensures it has a uniform scale and distribution, facilitating subsequent model training and computation. For example, for normal motion trajectory samples, the position coordinate sequence is converted to a relative displacement sequence. The motion direction, velocity, and acceleration features are calculated at each time point. These features are combined into a multi-dimensional trajectory feature vector containing information such as relative displacement, motion direction, velocity, and acceleration. Standardization methods, such as Z-score standardization, are used to process the feature vector so that each feature has a mean of 0 and a standard deviation of 1.

[0152] Step S543: Construct a deep autoencoder network as an anomaly detection model. The encoder part maps the trajectory feature vector to a low-dimensional latent space through an LSTM network and a fully connected layer. The decoder part reconstructs the latent vector into the original trajectory feature vector through a deconvolution layer and an LSTM network.

[0153] A deep autoencoder network is constructed as an anomaly detection model to learn the latent representation of normal trajectories and reconstruct the original trajectories. The encoder part maps the trajectory feature vectors to a low-dimensional latent space using an LSTM network and fully connected layers. LSTM is a long short-term memory network capable of processing sequential data and capturing the time-series information of the trajectory. The fully connected layers map the output of the LSTM network to a low-dimensional space, reducing the dimensionality of the data. The decoder part reconstructs the original trajectory feature vectors from the latent vectors using deconvolutional layers and an LSTM network. The deconvolutional layers restore the low-dimensional latent vectors to the original feature dimensions, and the LSTM network ensures that the reconstructed trajectory has the correct time series.

[0154] Step S544: Train an autoencoder network using the feature vectors of the normal trajectory, with the goal of minimizing the reconstruction error. Update the network parameters using the backpropagation algorithm. After training, save the reconstruction model of the normal trajectory.

[0155] Training the autoencoder network using feature vectors from normal trajectories allows the network to learn the features of normal trajectories. The optimization objective is to minimize the reconstruction error, i.e., by adjusting the network parameters to minimize the error between the reconstructed trajectory feature vector and the original trajectory feature vector. The network parameters are updated using the backpropagation algorithm, and the weights of each neuron in the network can be adjusted according to the magnitude of the error. After training, the reconstructed model of normal trajectories is saved for subsequent use in anomaly detection. The reconstruction error is set to mean squared error, and the Adam optimizer is used for optimization. During training, the network parameters are continuously adjusted to gradually reduce the reconstruction error. Training ends when a certain number of epochs are reached or the reconstruction error reaches a certain threshold. The trained network model is saved, and this model can be used to reconstruct new trajectories and detect anomalies.

[0156] Step S545: Convert the trajectory data to be detected into a trajectory feature vector, input it into the trained autoencoder network for reconstruction, and calculate the mean square error between the reconstructed trajectory feature vector and the original trajectory feature vector as the reconstruction error, i.e., the preliminary anomaly index.

[0157] Converting the trajectory data to be detected into trajectory feature vectors transforms the raw data into a form suitable for processing by an autoencoder network. The trained autoencoder network is then input for reconstruction, attempting to reconstruct the original trajectory feature vector based on the feature vector of the detected trajectory. The mean squared error (MSE) between the reconstructed and original trajectory feature vectors is calculated as the reconstruction error. MSE is a commonly used metric to measure the difference between two vectors, reflecting the accuracy of the reconstruction. This reconstruction error serves as a preliminary indicator of anomaly; a larger reconstruction error indicates a greater difference between the detected trajectory and a normal trajectory, suggesting a higher probability of anomaly.

[0158] Step S546: Retrieve the K reference trajectories most similar to the trajectory to be detected from the normal behavior pattern library, calculate the dynamic time warping distance between the trajectory to be detected and each reference trajectory, and take the average distance as the trajectory deviation index.

[0159] The process of retrieving the K most similar reference trajectories from a normal behavior pattern library aims to find the closest normal trajectory pattern to the target trajectory. The dynamic time-warped distance (DTW) between the target trajectory and each reference trajectory is calculated. DTW is a method for measuring the similarity between two time series, handling differences in time series length and time alignment issues. The average distance is used as an indicator of trajectory deviation; a larger average distance indicates a greater difference between the target trajectory and the normal trajectory pattern.

[0160] Step S547: Combine reconstruction error and trajectory deviation to generate a comprehensive anomaly score. When the comprehensive anomaly score exceeds a set threshold, the trajectory is determined to be an abnormal motion trajectory. The threshold is dynamically adjusted according to the scene complexity and detection accuracy requirements.

[0161] The purpose of fusing reconstruction error and trajectory deviation to generate a comprehensive anomaly score is to comprehensively consider information from both indicators and more accurately assess the degree of trajectory anomaly. A weighted summation method can be used to fuse the reconstruction error and trajectory deviation. When the comprehensive anomaly score exceeds a set threshold, the trajectory is determined to be an abnormal motion trajectory. The threshold is dynamically adjusted based on scene complexity and detection accuracy requirements. For complex scenes with many interfering factors, the threshold can be appropriately increased; for situations requiring high detection accuracy, the threshold can be appropriately decreased.

[0162] Step S550: Conduct behavioral tracing analysis on livestock individuals identified as having abnormal movement trajectories, extract morphological-motor joint coding sequences and trajectory topological features during abnormal periods, and analyze the start time, duration, location change patterns, and morphological feature evolution process of abnormal behavior.

[0163] Behavioral tracing analysis of livestock individuals identified as having abnormal movement trajectories aims to gain a deeper understanding of the causes and processes of abnormal behavior. Morphological-motor joint coding sequences and trajectory topological features are extracted during abnormal periods; these features can reflect the livestock's movement state and morphological characteristics during these periods.

[0164] As one implementation method, step S550 can be implemented as the following steps S551~S557:

[0165] Step S551: Determine the start time and duration of the abnormal behavior based on the timestamp information of the abnormal motion trajectory, and extract the video frame sequence within that duration as the data source for anomaly analysis.

[0166] Determining the start time and duration of abnormal behavior based on the timestamp information of abnormal movement trajectories is to accurately locate the time range in which the abnormal behavior occurred. Video frame sequences within this time period are extracted as the data source for anomaly analysis. These video frames contain detailed information about the livestock during the abnormal period.

[0167] Step S552: Extract the morphology-motor joint coding sequence of the corresponding livestock individual from the anomaly analysis data source, and divide the coding sequence into normal stage, transition stage and anomaly stage through time series segmentation algorithm, focusing on analyzing the characteristic change pattern of the transition stage.

[0168] Extracting the morphological-motor joint coding sequence of individual livestock from anomaly analysis data sources aims to obtain information on the movement and morphological characteristics of livestock during abnormal periods. A time-series segmentation algorithm is used to divide the coding sequence into three stages: normal, transition, and abnormal. The normal stage represents the livestock's normal movement state before the anomaly occurs; the transition stage represents the transition from the normal state to the abnormal state; and the abnormal stage represents the livestock's abnormal movement state. The focus is on analyzing the characteristic change patterns of the transition stage, as it may contain crucial information about the occurrence of abnormal behavior.

[0169] Step S553: ​​Combine trajectory topology features to analyze the evolution process of abnormal behavior, identify key transition nodes from normal behavior to abnormal behavior through feature mutation detection algorithm, and extract the trajectory feature differences before and after the transition nodes.

[0170] Analyzing the evolution of anomalous behavior using trajectory topology features aims to understand its development from a more macroscopic perspective. Feature mutation detection algorithms identify key transition points from normal to anomalous behavior, detecting points of sudden change in the feature sequence. Extracting the differences in trajectory features before and after these transition points reveals the causes and impacts of the anomalous behavior. For example, analyzing the evolution of anomalous behavior using trajectory topology features and employing feature mutation detection algorithms (such as statistically based mutation detection algorithms) to identify key transition points from normal to anomalous behavior.

[0171] Step S554: Perform spatial coordinate transformation on the position coordinate information at the transformation node. Convert the coordinates in the image coordinate system to the physical coordinate system coordinates in the actual monitoring scene through camera calibration parameters to determine the precise geographical location where the abnormal behavior occurred.

[0172] Spatial coordinate transformation of the location coordinates at the transformation node is to convert the location information in the image into location information in the actual physical scene. Camera calibration parameters are used to convert the coordinates in the image coordinate system to the physical coordinate system of the actual monitoring scene. These camera calibration parameters are obtained through the camera calibration process and establish a mapping relationship between the image coordinate system and the physical coordinate system. For example, in a pig farm monitoring video, for the location coordinates at the identified transformation node, spatial coordinate transformation is performed using camera calibration parameters. Assuming the camera calibration parameters are known, a series of matrix operations are used to convert the coordinates in the image coordinate system to the coordinates in the actual physical coordinate system. This allows the precise geographical location of the abnormal behavior to be determined as a specific area of ​​the pig farm, such as the northeast corner of the pigsty.

[0173] Step S555: Analyze the morphological changes during the abnormal phase, including the degree of deformation of the contour features, the frequency of changes in the direction of movement, and the fluctuation amplitude of the velocity features, and extract the abnormal behavior features with discriminative characteristics.

[0174] Analyzing morphological changes during abnormal phases aims to identify key features that distinguish abnormal behavior. These include the degree of deformation of the silhouette, the frequency of changes in movement direction, and the amplitude of fluctuations in velocity characteristics. These features reflect the movement and morphological characteristics of livestock in abnormal states. Extracting discriminative abnormal behavior features can then be used for subsequent abnormal behavior classification and early warning. For example, in monitoring videos of pig farms, analyzing morphological changes during abnormal phases reveals that a pig exhibits significant deformation of its silhouette during the abnormal phase, becoming more curled up. The frequency of changes in movement direction increases markedly, and the amplitude of fluctuations in velocity characteristics is also large.

[0175] Step S556: Construct an abnormal behavior classifier. Input the extracted abnormal behavior features into the classifier to determine the abnormality type. The classifier is trained based on historical labeled data and can identify a variety of typical abnormal behavior types.

[0176] The purpose of building an abnormal behavior classifier is to classify and identify abnormal behaviors. Extracted abnormal behavior features are input into the classifier to determine the type of abnormal behavior. The classifier can then identify the type of abnormal behavior based on the input features. The classifier is trained on historical labeled data, which contains labeled information about known abnormal behavior types. By training with this data, the classifier can learn feature patterns of different abnormal behavior types, thus enabling it to identify various typical abnormal behavior types. For example, an abnormal behavior classifier based on a support vector machine can be built. Extracted abnormal behavior features (such as the degree of contour deformation, frequency of changes in movement direction, and amplitude of speed fluctuations) are input into the classifier. The classifier is trained on historical labeled data, which contains various typical abnormal behavior types of livestock, such as illness, fright, and wandering.

[0177] Step S557: Integrate the start time, duration, physical location coordinates, anomaly type, and characteristic change description of the abnormal behavior to generate a structured abnormal behavior identification result. The result includes standardized data fields and a unified format specification.

[0178] Integrating the start time, duration, physical location coordinates, anomaly type, and characteristic change description of abnormal behavior is to organize and summarize relevant information. This generates structured abnormal behavior identification results, containing standardized data fields and a unified format, facilitating viewing and analysis by management personnel. For example, the start time of the abnormal behavior is recorded as a specific time point, the duration as the number of frames, the physical location coordinates as the actual geographic coordinates, the anomaly type as the specific type (e.g., injury, poisoning), and the characteristic change description as detailed information on the characteristic changes. This information is then integrated into a single table containing standardized data fields such as "start time," "duration," "physical location coordinates," "anomaly type," and "characteristic change description."

[0179] Step S560: Integrate the statistical results and abnormal behavior analysis information to generate a comprehensive identification result that includes the current total number of livestock, the number of livestock distributed in each region, abnormal individual identifiers, and descriptions of abnormal behavior.

[0180] Integrating livestock statistics and abnormal behavior analysis information aims to provide a comprehensive display of both livestock quantity and abnormal behavior data. This generates a comprehensive identification result that includes the current total number of livestock, the number of livestock distributed across different regions, abnormal individual identifiers, and descriptions of abnormal behavior. The current total number of livestock reflects the overall scale of the farm; the number of livestock distributed across different regions reflects the distribution of livestock in different areas; the abnormal individual identifiers identify which livestock exhibited abnormal behavior; and the abnormal behavior descriptions provide detailed explanations of the abnormal behavior.

[0181] It is understood that the various algorithms involved in the above descriptions of the embodiments of the present invention can all be obtained from relevant content in the prior art. To save space, they will not be elaborated on in the embodiments of the present invention. In addition, those skilled in the art can supplement the details based on common knowledge in the art when implementing the solutions of the present invention. For example, they can use normalization to eliminate dimensional conflicts before feature fusion, use interpolation to eliminate dimensional differences, reasonably set thresholds based on historical data, experience or business scenario requirements, train the model based on a general model training method, set the number of layers in the model structure based on actual needs, select activation functions, etc. The present invention will not provide redundant descriptions of overly detailed implementation processes here.

[0182] Please see Figure 3 This is a schematic diagram of the structure of the computer system 10 in the above-described application scenario provided by the embodiments of the present invention. Figure 3As shown, the computer system 10 described above may include: a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer system 10 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 3 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program. Figure 3 In the computer system 10 shown, the network interface 1004 provides network communication functions; the user interface 1003 is mainly used to provide an input interface; and the processor 1001 can be used to call the device control application stored in the memory 1005 to implement the methods provided in the above embodiments.

Claims

1. A method for livestock number recognition combining AI vision and multi-target tracking, characterized in that, The method includes: Collect continuous video streams of livestock activity areas and generate a sequence of video frames with timestamps; Dynamic background modeling is performed on the video frame sequence to extract the contour feature set of livestock objects in each frame image. Based on the temporal change law of the contour feature set, a multidimensional motion representation vector is generated. The multidimensional motion representation vector includes contour deformation parameters and motion trend descriptors. Adaptive multi-scale feature enhancement is performed on the multidimensional motion representation vector to decouple the pixel-level features of the livestock herd region and output the morphology-motion joint encoding sequence for each individual livestock. Based on the morphology-motion joint encoding sequence, a spatiotemporal correlation matrix is ​​constructed. A lightweight feature extraction network is used to extract the topological features of livestock movement trajectories, generating a global ID mapping table containing unique individual identifiers, including: Temporal synchronization processing is performed on the morphology-motion joint coding sequence in consecutive video frames to establish a temporal association index for cross-frame coding sequences. The index value is generated according to the timestamp order of the video frames. Calculate the feature similarity between adjacent frame morphology-motion joint coding sequences, adjust the weight contribution of historical frame features by combining the time decay factor, and generate a dynamically updated inter-frame similarity matrix. A spatiotemporal correlation matrix is ​​constructed based on the inter-frame similarity matrix. The row dimension of the matrix corresponds to the encoding sequence index of the previous frame, the column dimension corresponds to the encoding sequence index of the current frame, and the matrix element values ​​represent the correlation strength of the corresponding encoding sequence pairs. The matrix content is dynamically updated through a time sliding window mechanism. The spatiotemporal correlation matrix is ​​input into a lightweight graph convolutional network. The lightweight graph convolutional network contains a local feature extraction layer and a global feature aggregation layer. The local feature extraction layer captures the local correlation features of adjacent nodes through graph convolution operations, and the global feature aggregation layer models long-distance node dependencies through a self-attention mechanism to generate trajectory topology features that fuse local and global information. The trajectory topology features are processed to associate trajectory segments. Combining trajectory position continuity constraints and feature similarity constraints, the encoded sequences in consecutive frames are associated as complete trajectories. A unique identifier code is assigned to each complete trajectory, and a global ID mapping table containing information on current active trajectories and historical trajectories is generated. Based on the global ID mapping table, the number of livestock is counted in real time, and the behavior of individual livestock corresponding to abnormal movement trajectories is traced and analyzed to obtain the statistical results of the number of livestock and the results of abnormal behavior identification.

2. The method according to claim 1, characterized in that, The dynamic background modeling operation on the video frame sequence extracts the contour feature set of livestock objects in each frame image, and generates a multi-dimensional motion representation vector based on the temporal variation law of the contour feature set. The multi-dimensional motion representation vector includes contour deformation parameters and motion trend descriptors, including: The video frame sequence is spatiotemporally segmented to divide it into spatiotemporal units containing spatial location encoding and time sequence information. The spatial size of the spatiotemporal unit is dynamically adjusted according to the video resolution, and the time span is matched with the livestock movement cycle. Dynamic background estimation is performed on the spatiotemporal units through background probability modeling. Based on the time-dimensional model parameter update mechanism, the probability value of each spatiotemporal unit belonging to the background region is calculated, and a background probability distribution map that evolves over time is generated. Based on the spatiotemporal distribution characteristics of the background probability distribution map, the foreground region determination conditions are determined, and regions that satisfy the condition of having a probability value lower than the determination condition for multiple consecutive frames are marked as candidate livestock regions, thereby generating a binarized foreground mask map with spatiotemporal correlation. Morphological optimization operations are performed on the binarized foreground mask image, sequentially performing dilation processing to connect broken contours and erosion processing to restore detailed features, to generate an optimized foreground mask image containing the complete livestock object region. Based on the optimized foreground mask image, a set of contour features of livestock objects is extracted from the original video frame. The set of contour features includes edge point coordinate sequences, multi-scale curvature variation features, region area parameters, and directional gradient distribution features.

3. The method according to claim 2, characterized in that, The extraction of the livestock object's contour feature set from the original video frame based on the optimized foreground mask image includes: The optimized foreground mask image is subjected to edge extraction processing to generate a hierarchical edge point coordinate sequence containing continuous edge structures; The layered edge point coordinate sequence is subjected to polygon approximation processing. The curvature change rate of adjacent edge points is calculated. Regions with curvature change rate exceeding a preset change rate threshold are marked as key feature regions and their turning points are retained. Regions with curvature change rate not exceeding the preset change rate threshold are marked as regular feature regions and their redundant edge points are simplified. A simplified contour sequence containing turning point coordinates and corresponding curvature levels is generated. Multi-scale curvature analysis is performed on the simplified contour sequence. The change in the tangent angle between adjacent turning points is calculated at different scale levels to generate multi-scale curvature change features that describe the changes in contour details. The number of scale levels is adjusted according to the contour complexity. The number of pixels in each connected region of the optimized foreground mask image is calculated by region filling processing, and then converted into the actual physical region area parameter by combining the spatial coordinate system. The rate of change of the area parameter in the time dimension is recorded synchronously. The directional gradient statistics are performed on the coordinate sequence of the layered edge points. The gradient direction of the edge points is divided into multiple intervals and the gradient magnitude ratio of each interval is calculated to generate directional gradient distribution features that describe the contour direction characteristics. By integrating the layered edge point coordinate sequence, multi-scale curvature change features, physical region area parameters, and directional gradient distribution features, and performing feature standardization processing, a contour feature set containing spatial morphology and spatiotemporal dynamic characteristics is constructed.

4. The method according to claim 3, characterized in that, The generation of a multidimensional motion representation vector based on the temporal variation pattern of the contour feature set includes: The contour feature set in consecutive video frames is aligned on the time axis. The correspondence between contour features across frames is established by matching edge point coordinate sequences. The matching priority is determined based on the similarity of multi-scale curvature change features. Calculate the offset of edge point coordinates of the contour feature set between adjacent frames, supplement the offset information of the occluded area to generate a complete contour deformation displacement field, and perform sparse representation of the contour deformation displacement field. The vector distribution characteristics of the profile deformation displacement field are analyzed, the deformation parameters in the main direction and the deformation amplitude values ​​in each region are extracted, the spatial distribution uniformity index of the deformation amplitude is calculated, and the deformation parameters in the main direction, the deformation amplitude values ​​in each region and the spatial distribution uniformity index of the deformation amplitude are integrated into the profile deformation parameters that describe the changes in the profile shape. Based on the contour deformation parameters, a statistical model of motion direction distribution is constructed. The active motion region is subdivided into direction intervals, and the gentle motion region is merged into direction intervals, generating motion direction features that include the proportion of direction and the rate of change of proportion. By combining the motion direction characteristics and the temporal change rate of the physical region area parameter, the motion state within a preset time interval is predicted to generate a motion trend descriptor that includes the direction probability distribution, velocity change characteristics and prediction error range. The contour deformation parameters and motion trend descriptions are combined into a multi-dimensional motion representation vector with a unified dimension. Each feature dimension is weighted through an attention mechanism, and the weight values ​​are adaptively adjusted according to the contribution of the features in historical recognition tasks.

5. The method according to claim 1, characterized in that, The adaptive multi-scale feature enhancement of the multidimensional motion representation vector decouples the pixel-level features of the livestock herd region and outputs the morphology-motor joint encoding sequence for each individual livestock, including: The multidimensional motion representation vector is input into a multi-scale feature extraction network, and the feature extraction paths of different receptive fields are processed in parallel to generate a set of multi-scale feature maps. The multi-scale feature map set is subjected to feature fusion processing. Based on the attention-guided weighted fusion strategy, the regional saliency weights of each scale feature map are calculated and weighted summation is performed to generate a fused feature map that retains multi-scale information. An attention mechanism module is constructed to calculate attention weights from both the spatial dimension and the feature channel dimension, and to generate attention-enhanced feature maps. Based on the attention-enhanced feature map, livestock herding areas are identified, and the individual density distribution of the herding area is predicted by density estimation. The boundary of the herding area is determined according to the density distribution characteristics, and a herding area mask is generated. The cluster region mask is subjected to individual boundary division processing. Based on the clustering algorithm, the individual center selection is optimized and the sticky regions are separated to generate an accurate region mask for each individual livestock. Based on the precise region mask, the morphological and motion features of the corresponding regions are extracted from the attention-enhanced feature map. After unifying the dimensions of features at different scales through feature mapping, temporal encoding is performed to generate a morphological-motion joint encoding sequence containing individual spatiotemporal features.

6. The method according to claim 5, characterized in that, The feature fusion processing of the multi-scale feature map set, based on an attention-guided weighted fusion strategy, calculates the regional saliency weights of the feature maps at each scale and performs a weighted summation to generate a fused feature map that retains multi-scale information, including: The multi-scale feature map set is sorted by scale level, and regional saliency analysis is performed on each scale feature map to generate a regional saliency weight map that matches the size of the feature map. The weight value reflects the contribution of the region to the livestock identification task. Size alignment is performed on feature maps at different scales. The semantic feature maps output by the deep layers of the network are upsampled to the same spatial size as the detail feature maps output by the shallow layers of the network. The size of the corresponding saliency weight maps is adjusted synchronously. The saliency weight maps at each scale are normalized and the weight values ​​are mapped to a preset numerical range to obtain normalized weight maps. A weighted summation operation is performed on the size-aligned feature maps based on the normalized weight map to generate a preliminary fused feature map; A multi-path residual connection structure is constructed to extract edge and texture features from the original multi-dimensional motion representation vector. These features are then mapped to the same dimension as the initial fused feature map through a feature transformation layer. Finally, the features are superimposed on the initial fused feature map through a skip connection method to obtain the superimposed fused feature map. The superimposed fused feature map is subjected to feature calibration processing, and the feature channels are dynamically selected through a channel attention mechanism to generate the final multi-scale fused feature map.

7. The method according to claim 6, characterized in that, The process of identifying livestock herd regions based on the attention-enhanced feature map, predicting the individual density distribution of the herd region through density estimation, determining the herd region boundary based on the density distribution characteristics, and generating a herd region mask includes: The attention-enhanced feature map is input into the density estimation network, and multi-scale density prediction is performed through an encoder-decoder structure. The encoder part uses progressive downsampling to extract contextual features, and the decoder part fuses feature information from different levels through skip connections to generate a density prediction map with the same size as the input feature map. The density prediction map is post-processed and optimized by using adaptive Gaussian kernel smoothing to eliminate density prediction noise; Analyze the spatial distribution characteristics of the optimized density prediction map, calculate the spatial gradient change of density values, mark the regions with drastic gradient changes as potential cluster boundaries, and generate a cluster boundary candidate map. Region growth is performed based on density values ​​and boundary information. Starting from the density peak point, the region is expanded outwards until the density gradient threshold or boundary candidate region is encountered, generating the initial cluster region. Morphological optimization is performed on the initial cluster region, and region merging and splitting operations are performed. Based on the comparison results between the distance between the centroids of the cluster regions in the density map and the preset distance threshold, small clusters with a distance less than the preset distance threshold and continuous density distribution are merged. By calculating the rate of change of the density gradient inside the cluster region, when the rate of change exceeds the preset gradient threshold, it is determined to be a density distribution discontinuity. Large clusters with density distribution discontinuities are split to generate the optimized cluster region. The optimized cluster region is converted into a binary mask image, with the pixel value within the cluster region set to 1 and the background region set to 0, generating a cluster region mask containing complete cluster boundary information.

8. The method according to claim 7, characterized in that, The spatiotemporal correlation matrix is ​​input into a lightweight graph convolutional network to generate trajectory topology features that fuse local and global information, including: The spatiotemporal correlation matrix is ​​converted into a graph structure data representation, where each element of the matrix corresponds to the edge weight in the graph structure, the row and column indices correspond to the graph nodes, and the node feature vector is composed of the corresponding morphology-motion joint encoding sequence. A two-layer graph convolutional network is constructed as the local feature extraction layer. The first graph convolutional layer adopts a fixed-size neighborhood aggregation method to aggregate the direct neighbor features of each node. The second graph convolutional layer adopts an adaptive neighborhood aggregation method to dynamically adjust the neighborhood size according to the node similarity and generate local topological features containing local context information. Local topological features are input into the multi-head self-attention mechanism layer as a global feature aggregation layer. Long-distance dependencies between nodes are calculated through multiple parallel attention heads. Each attention head focuses on different types of trajectory association patterns, generating global association features from multiple perspectives. The output features of multiple attention heads are concatenated, and the concatenated features are mapped to a fixed dimension through a feature transformation layer. Layer normalization is used to stabilize the feature distribution and generate global topological features containing global structural information. Local and global topological features are fused through residual connections. The fusion ratio of the two features is dynamically adjusted through a gating mechanism, which enhances the weight of local features in dense trajectory regions and enhances the weight of global features in sparse trajectory regions. The fused features are subjected to nonlinear transformation, and the feature representation ability is enhanced by activation function, noise feature interference is suppressed, and trajectory topological feature vector is generated.

9. A computer system, characterized in that, include: processor; And a memory, wherein the memory stores computer-readable code that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 8.

Images