A method, apparatus and program product for identifying primate behavior

Abstract

The application relates to the field of behavior recognition, in particular to a method, device and program product for recognizing primate behavior. The method comprises the following steps: acquiring a multi-view image dataset of primate behavior patterns; aligning image frames of multi-view image data of the same behavior pattern of the same primate sample in the multi-view image dataset through timestamps, and then extracting core skeleton key points and behavior-related feature points to obtain a matching feature point coordinate set; performing spatial mapping on the matching feature point coordinate sets of different views to obtain multi-view spatial alignment features; performing similarity calculation on the multi-view spatial alignment features; when the similarity is greater than or equal to a first preset threshold, determining that the feature is an effective feature; otherwise, determining that the feature is an occlusion region; performing feature completion on the data of the occlusion region through the effective features of different views in the same space to obtain a complete feature set, and performing primate behavior recognition to obtain a recognition result. The application has good application value.

Description

Technical Field

[0001] This application relates to the field of behavior recognition, specifically to a method, device, program product, and computer-readable storage medium for recognizing primate behavior. Background Technology

[0002] As primates closely related to humans, monkeys' behavioral analysis is a core component of related research. The four behaviors of "mounting," "grooming," "alertness," and "avoidance" are crucial for revealing an animal's physiological state, social mechanisms, and cognitive abilities. Chinese patent application number 202310369492.0 discloses a method, device, electronic equipment, and medium for analyzing fine motor skills in primates. This method can promptly detect subtle changes, providing a basis for accurately assessing behavioral disorders caused by various neurological diseases. However, this approach focuses on detecting fine motor skills in localized limbs and employs a single-view selection mechanism, making it suitable only for controlled laboratory settings. It cannot meet the practical needs of whole-body monkey behavioral recognition, complex field applications, and cross-individual analysis.

[0003] Furthermore, current monkey behavior recognition technology suffers from three major pain points, which severely restrict research efficiency and accuracy: 1. Data dependency bottleneck: Current behavior recognition methods mostly rely on large-scale labeled data to train models. However, the collection and labeling of monkey behavior data requires a lot of manpower and time, and the labeling process is easily affected by subjective factors, resulting in a scarcity of high-quality labeled data and making it difficult to advance model training. 2. Single-view robustness deficiency: Mainstream visual recognition methods rely only on single-view image / video data, which are easily affected by factors such as monkey limb occlusion, changes in ambient lighting during shooting, and interference from complex backgrounds, resulting in incomplete extraction of behavioral features, unstable recognition results, and insufficient robustness. 3. Poor end-to-end interpretability: Existing mainstream behavior recognition technologies adopt an end-to-end training mode, directly mapping from raw data to behavior categories. The intermediate process lacks interpretable feature associations, making it impossible to clearly define the core basis for behavior recognition. Once the recognition results deviate, it is difficult to trace the root cause of the problem, and it is difficult to meet the requirement of verifiable results in scientific research scenarios. Summary of the Invention

[0004] To overcome the aforementioned technical bottlenecks, this application innovates from three core levels: data, input, and method, proposing a monkey behavior recognition scheme based on multi-view fusion and active learning. This scheme balances model performance, robustness, and interpretability, filling a gap in existing technologies for primate behavior recognition. This invention provides a data processing method for primate behavior recognition, specifically including: Obtain a multi-view image dataset of primate behavioral patterns; The multi-view image data of the same behavioral pattern of the same primate sample in the multi-view image dataset are aligned by timestamp, and then the core skeleton key points and behavioral related feature points are extracted for each aligned image frame to obtain the matching feature point coordinate set. Multi-view spatial alignment features are obtained by spatially mapping the coordinate sets of matching feature points from different perspectives. The similarity of the multi-view spatial alignment features is calculated. When the similarity is greater than or equal to the first preset threshold, it is determined to be a valid feature. Otherwise, it is determined to be an occluded region. The data of the occluded region is supplemented by the valid features of different views in the same space to obtain a complete feature set.

[0005] Optionally, the occlusion region completion calculates the similarity of effective features from different perspectives in the same space, takes the one with the highest similarity as the behavioral feature of the reference perspective, and completes the data of the occlusion region by combining the behavioral feature of the reference perspective with the constraint of primate movement continuity.

[0006] Optionally, the primate motion continuity constraint is that the displacement of feature points in adjacent frames is limited to less than or equal to L pixels, where L is a natural number greater than or equal to 5.

[0007] Optionally, the feature points of the occluded area are selected from the corresponding feature points of the reference viewpoint through primate continuity constraints and then weighted to complete the feature points.

[0008] Optionally, the spatial mapping is performed through a spatial mapping model. The coordinate sets of matching feature points from different perspectives are input into the spatial mapping model to obtain multi-view spatial alignment features. The construction of the spatial mapping model includes: obtaining the coordinate sets of matching feature points from different perspectives, performing feature point matching on the coordinate sets of matching feature points from different perspectives to obtain matching feature point pairs, and constructing a spatial mapping model through the matching feature point pairs. Optionally, the feature point matching includes at least K pairs, where K is a natural number greater than or equal to 4; Optionally, the mapping formula of the spatial mapping model is expressed as:

[0009] in, and Let x and y represent the x and y coordinates from the perspective of m after mapping, k represent the dimension of the mapping, and m and n represent different perspectives. and denoted by , where represents the x and y coordinates of the mapped n-viewpoint, and h represents the elements in the matrix of geometric transformation relationships between planar points under different viewpoints.

[0010] The present invention provides a method for identifying primate behavior, characterized in that it includes: Acquire multi-view image data of the primate to be tested; The multi-view image data is input into the data processing method for primate behavior recognition described above to obtain processed image data; Skeleton detection is performed on the processed image data to obtain the key locations and connections of the primate skeleton; The processed image data is subjected to primate appearance features and temporal features extraction to obtain appearance features and temporal features; The identification results were obtained by identifying primate behavior through the key locations and connections of the primate skeleton, appearance features, and temporal features.

[0011] Optionally, the key locations of the primate skeleton include the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left foot, and right foot.

[0012] Optionally, the primate behaviors include mounting, grooming, alertness, and avoidance.

[0013] Optionally, the skeleton detection is performed by a skeleton detection network model to obtain the location and connection relationship of the skeleton key points. The skeleton detection network model includes a whole skeleton detection module and a primate joint density adaptation module. The image data is processed sequentially by the whole skeleton detection module and the primate joint density adaptation module to perform primate pose estimation and local feature localization, respectively, to obtain the location and connection relationship of the primate skeleton key points.

[0014] Optionally, the overall skeleton detection module estimates primate posture features using a posture detection model, and inputs these estimated features into a primate joint density adaptation module. The primate joint density adaptation module includes N sequential convolutional modules and an average pooling layer, where N is a natural number greater than 1. Local joint features are extracted using the first convolutional module, and these local features are then input into the second convolutional module for structural feature extraction. The structural features are then input into the average pooling layer to obtain the global primate skeleton features. Finally, the primate posture estimation features and the global primate skeleton features are fused to obtain the primate keypoint locations and connections.

[0015] Optionally, the attitude detection model includes any one or more of the following: HRNet-W48 and HRNet-W32.

[0016] Optionally, the overall skeleton detection module further includes a limb proportion perception module, which processes image data to obtain multi-scale general features through a pose detection model, inputs the multi-scale general features to the limb proportion perception module, performs proportional encoding and spatial distribution prior encoding based on the relative lengths of the primate trunk and limbs, generates dynamic spatial offsets, and uses variable convolution to perform proportional adaptive sampling based on the dynamic spatial offsets to obtain primate pose estimation features.

[0017] Optionally, the primate appearance features are obtained by feature extraction using a neural network model, which includes any one or more of the following: convolutional neural network, residual network.

[0018] Optionally, the appearance features include limb proportions, joint angles, limb angles, body curvature, center of gravity offset, limb symmetry, and tail swing amplitude.

[0019] Optionally, the appearance features also include normalization processing, which normalizes the limb proportions, joint angles, and limb angles to obtain normalized appearance features, and uses the normalized appearance features, temporal features, key positions of the primate skeleton, and connection relationships to perform primate behavior recognition to obtain recognition results.

[0020] Optionally, the temporal features are obtained by feature extraction through a temporal network model, which includes any one or more of the following: LSTM, GRU, RNN.

[0021] Optionally, the training of the network model involves: acquiring an unlabeled multi-sample, multi-view image dataset; calculating the uncertainty entropy value of each sample in the unlabeled multi-sample, multi-view image dataset; filtering samples with entropy values ​​greater than a preset value and labeling them to obtain a core training set; training the model using the core training set to obtain an initial model; unlabeling the remaining unlabeled data; selecting pseudo-labeled samples with confidence values ​​greater than a second preset value and adding them to the core training set to obtain a second training set; and training the initial model using the second training set until the model performance stabilizes to obtain the network model.

[0022] The purpose of this invention is to provide a computer program product that includes a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the above-described method for recognizing primate behavior or a data processing method for recognizing primate behavior.

[0023] The purpose of this invention is to provide a computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, wherein the computer program or instructions are executed by the processor to implement the above-described method for recognizing primate behavior or a data processing method for recognizing primate behavior.

[0024] The purpose of this invention is to provide a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the above-described method for recognizing primate behavior or a data processing method for recognizing primate behavior.

[0025] Advantages of this invention: 1. To address the issues of primate (monkey) limb occlusion and changes in ambient lighting in single-view data acquisition, this invention employs a ring-distributed multi-view data acquisition scheme. Through multi-view feature alignment and complementary completion, it eliminates the need to attach positioning tags to monkeys (to prevent them from being removed and to adapt to their natural behavioral scenarios). By leveraging the complementarity of different viewpoints and combining the monkey's skeletal length ratio and angle range constraints, it completes the occlusion key points under a single viewpoint, ensuring the integrity of key points. This effectively solves the shortcomings of single-view methods that are susceptible to occlusion and changes in lighting, significantly improving the robustness and environmental adaptability of behavior recognition, and making it suitable for complex field and laboratory scenarios.

[0026] 2. To address the lack of interpretable feature associations in the mapping process from raw data to behavior categories, this invention introduces a skeleton detection intermediate module into the end-to-end recognition process. This constructs an interpretable link of "multi-view fusion - skeleton extraction - behavior classification," clarifying that the core basis for behavior recognition is the monkey's skeletal posture features. This overcomes the "black box" defect of traditional end-to-end models, making the recognition results traceable and verifiable, and better suited to the application needs of scientific research scenarios. Furthermore, to improve the adaptability of monkey behavior recognition, the recognition model is specifically adjusted based on the characteristics of the monkey's limbs and appearance, providing a solid foundation and high reliability for effective monkey behavior recognition in the wild.

[0027] 3. This invention addresses the problem of limited primate sample data and scarcity of high-quality labeled samples by innovatively adopting an active learning-driven small-sample data collection and labeling strategy. By using uncertainty measurement to screen key samples, only a small portion of core data needs to be labeled to achieve high-performance model training. This breaks through the bottleneck of traditional methods' dependence on large-scale labeled data, significantly reduces data collection and labeling costs, and solves the industry pain point of scarce and difficult labeling of primate behavioral data.

[0028] 4. Outdoor scenes often feature varying lighting and complex backgrounds, impacting image quality. This invention addresses this by adapting infrared cameras to outdoor scenes, combining the CLAHE algorithm to normalize lighting (optimizing the cumulative distribution function of the grayscale histogram to enhance contrast in low-light areas), and using an object detection model to locate primate regions. The bounding box is then expanded by 5% to avoid omissions, effectively removing background interference such as vegetation and cages, thus improving the model's adaptability to complex scenes. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram of the data processing method for primate behavior recognition provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a method for recognizing primate behavior provided in an embodiment of the present invention; Figure 3 A schematic diagram of a computer device is provided for embodiments of the present invention; Figure 4 This is a schematic diagram of multi-view deployment distribution provided in an embodiment of the present invention; Figure 5 A schematic diagram of the core key points provided in the embodiments of the present invention; Figure 6 This is a schematic diagram of monkey skeleton detection provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of monkey behavior recognition results provided in an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0032] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as S101, S102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0033] Figure 1 The data processing diagram for primate behavior recognition provided in this embodiment of the invention specifically includes: S1: Obtain a multi-view image dataset of primate behavioral patterns; In one embodiment, multi-viewpoint imaging involves acquiring images of the primate subject in a circular pattern around its main body, with the angle between the viewpoints ranging from 90° to 120°.

[0034] In one specific embodiment, the active data acquisition and annotation module is implemented, and the camera deployment is as follows: 3-4 cameras are arranged in a ring shape, such as... Figure 4 As shown, the angle between adjacent cameras is 90°-120°, with parameters of 1080P resolution and 30fps frame rate to ensure complete capture of actions. Infrared cameras are configured for outdoor scenes to support day and night shooting.

[0035] In one embodiment, the image further includes data preprocessing, which involves performing illumination normalization, image scaling, and background removal on the multi-view image dataset to obtain a preprocessed image dataset, and aligning the multi-view image data of the same behavioral pattern of the same primate sample in the preprocessed image dataset using timestamps.

[0036] In one specific embodiment, data preprocessing includes: scaling the images from all viewpoints to a uniform 512×512 pixels, optimizing illumination normalization, and eliminating differences in illumination such as day and night and shade; locating the monkey area, cropping the bounding box by 5% to avoid omissions, and removing background interference.

[0037] S2: Align the multi-view image data of the same behavioral pattern of the same primate sample in the multi-view image dataset with the timestamp, and then extract the core skeleton key points and behavioral related feature points of each aligned image frame to obtain the matching feature point coordinate set. In one embodiment, the core skeletal key points include the face, limbs, and torso; the behavioral association feature points include the tip of the nose, the right acromion, the left hip bone, and the tail root. The core skeletal key points are used to match and confirm the subject from different perspectives, while the behavioral association feature points are used to match and confirm actions in consecutive frames.

[0038] In one specific embodiment, synchronization is achieved by aligning the frames of data from each perspective using the camera's built-in timestamps to ensure that the time difference between data from multiple perspectives of the same action is ≤10ms, thereby guaranteeing the accuracy of multi-perspective feature fusion. Multi-view fusion module implementation: Feature alignment: A feature point matching algorithm is used to establish spatial mapping relationships between different viewpoints, achieving coordinate alignment of behavioral features. The core of feature alignment is to establish spatial mapping relationships between multiple viewpoints through precise matching of monkey-specific key feature points, achieving unified coordinate alignment of behavioral features. The specific process and quantitative calculation are as follows: Specific Feature Point Filtering: From the preprocessed images from various perspectives, extract the key points of the monkey's core skeleton (including face, limbs, and trunk) and high-frequency behaviorally related feature points (such as nose tip, right acromion, left hip bone, tail root, etc.) to form a specific matching feature point set. The pixel coordinates of each feature point ; Feature point matching calculation: An improved SIFT algorithm is used for cross-view feature point matching. Descriptors are constructed using the local gradient histograms of feature points, and feature points in viewpoints n and m are calculated. and Descriptor similarity:

[0039] in , Let be the k-th dimension components of the feature point descriptors in viewpoints n and m, respectively. When the value is ≥0.85, it is determined to be a matching feature point pair. .

[0040] S3: Spatial mapping of the coordinate sets of matching feature points from different perspectives yields multi-view spatial alignment features; In one embodiment, the spatial mapping is performed through a spatial mapping model. The coordinate sets of matching feature points from different perspectives are input into the spatial mapping model to obtain multi-view spatial alignment features. The construction of the spatial mapping model includes: obtaining the coordinate sets of matching feature points from different perspectives, performing feature point matching on the coordinate sets of matching feature points from different perspectives to obtain matching feature point pairs, and constructing a spatial mapping model through the matching feature point pairs. Optionally, the feature point matching includes at least K pairs, where K is a natural number greater than or equal to 4; Optionally, the mapping formula of the spatial mapping model is expressed as:

[0041] in, and Let x and y represent the x and y coordinates from the perspective of m after mapping, k represent the dimension of the mapping, and m and n represent different perspectives. and denoted by , where represents the x and y coordinates of the mapped n-viewpoint, and h represents the elements in the matrix of geometric transformation relationships between planar points under different viewpoints.

[0042] In one specific embodiment, spatial mapping relationship modeling is performed: based on at least four pairs of matching feature points, a spatial mapping model between viewpoints is established using a homography matrix. Let the coordinates of the feature points at viewpoint n be... The coordinates of the feature point corresponding to the viewpoint m are homography matrix satisfy:

[0043] in, , Solving for matrix parameters using the least squares method:

[0044] Coordinate alignment is achieved by using the homography matrix H obtained from the solution to map the coordinates of all behavioral feature points in viewpoint n to the coordinate system of viewpoint m, thus completing the spatial alignment of multi-view features. For feature points that are not directly matched... The alignment coordinates of the feature in viewpoint m are obtained through the mapping formula, ensuring that multi-view behavioral features can be directly fused in the same spatial coordinate system.

[0045] S4: Calculate the similarity of the multi-view spatial alignment features. If the similarity is greater than or equal to the first preset threshold, it is determined to be a valid feature. Otherwise, it is determined to be an occluded area. Complete the data of the occluded area by using the valid features of different views in the same space to obtain a complete feature set.

[0046] In one embodiment, the occlusion region completion calculates the similarity of effective features from different perspectives in the same space, takes the highest similarity as the behavioral feature of the reference perspective, and completes the data of the occluded region by combining the behavioral feature of the reference perspective with the monkey's motion continuity constraint. Optionally, the monkey motion continuity constraint is that the displacement of feature points in adjacent frames is limited to less than or equal to L pixels, where L is a natural number greater than or equal to 5; Optionally, the corresponding feature points of the reference viewpoint are selected for the feature points of the occluded area through monkey continuity constraints and then weighted and completed.

[0047] In one specific embodiment, consistency verification and completion are performed as follows: The similarity of the same behavioral feature under different viewpoints is calculated. A similarity ≥ 0.8 is considered a valid feature, while a similarity < 0.3 is considered an occluded region. For occluded regions, feature completion is performed using feature data from valid viewpoints, combined with constraints on the continuity of monkey movement, to form a complete feature set. The behavioral feature similarity is calculated based on the aligned feature vectors: Let the viewpoints be... The behavioral feature vector is Perspective The behavioral feature vector is The similarity is calculated as follows:

[0048] When completing features in occluded regions, based on the monkey motion continuity constraint (the displacement of feature points in adjacent frames does not exceed 5 pixels), the occluded feature points are... ,pass Corresponding feature points of each effective viewpoint Weighted completion:

[0049] in The behavior feature vector is used as a reference perspective to ensure that the completed features are consistent with the actual behavior and posture of the monkey.

[0050] Figure 2 This invention provides a schematic diagram of a method for identifying primate behavior, comprising: S201. Acquire multi-view image data of the primate to be tested; S202. Input the multi-view image data into the above-described data processing method for primate behavior recognition to obtain processed image data; S203. Perform skeleton detection on the processed image data to obtain the key locations and connections of the primate skeleton; In one embodiment, key locations of the primate skeleton include the nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left foot, and right foot.

[0051] Key locations of the monkey skeleton, such as Figure 5 As shown, 17 key sites of the monkey were detected in real time using a skeleton detection network, such as... Figure 6 As shown, this provides the foundation for subsequent behavior recognition.

[0052] In one embodiment, a customized skeleton detection network (overall skeleton detection module) is constructed based on a lightweight convolutional neural network. The input is a feature set fused from multiple perspectives, and the output is the position and connection relationship of key feature points of the monkey skeleton. It does not rely on a fixed number of key point definitions and focuses on the overall representation of the skeleton posture. Model optimization: Focal Loss is used to optimize the positive and negative sample balance problem, and the Adam optimizer is used for model training to ensure the accuracy and robustness of skeleton feature extraction.

[0053] In one embodiment, the skeleton detection is performed by a skeleton detection network model to obtain the location and connection relationship of the skeleton key points. The skeleton detection network model includes a whole skeleton detection module and a primate joint density adaptation module. The image data is processed sequentially by the whole skeleton detection module and the primate joint density adaptation module to perform primate pose estimation and local feature localization, respectively, to obtain the location and connection relationship of the primate skeleton key points.

[0054] In one embodiment, the overall skeleton detection module estimates primate posture features using a posture detection model, and inputs these estimated features to a primate joint density adaptation module. The primate joint density adaptation module includes N sequential convolutional modules and an average pooling layer, where N is a natural number greater than 1. Local joint features are extracted using a first convolutional module, and these local features are then input to a second convolutional module for structural feature extraction. The structural features are then input to the average pooling layer to obtain global primate skeletal features. Finally, the primate posture estimation features and the global primate skeletal features are fused to obtain the primate keypoint locations and connections. Optionally, the attitude detection model includes any one or more of the following: HRNet-W48 and HRNet-W32.

[0055] In one embodiment, the overall skeleton detection module further includes a limb proportion perception module. The module processes image data to obtain multi-scale general features through a pose detection model, and inputs the multi-scale general features into the limb proportion perception module. The module performs proportional encoding and spatial distribution prior encoding based on the relative lengths of the primate trunk and limbs to generate dynamic spatial offsets. Variable convolution performs proportional adaptive sampling based on the dynamic spatial offsets to obtain primate pose estimation features.

[0056] In one specific embodiment, by using multi-view information complementarity and completion, outlier removal (based on monkey skeletal structure constraints), and linear interpolation completion of adjacent key points, it is ensured that each image / video frame can output all 17 core key points without omission; Necessity and Minimum Number of 17 Keypoints: 17 keypoints represent the optimal and minimum effective set that balances behavior recognition accuracy and computational efficiency. This number of keypoints comprehensively covers the monkey's face, limbs, and core trunk, fully representing the key postural features of the four target behaviors. For example, the absence of the left wrist keypoint leads to the loss of the reciprocating motion features of the grooming behavior, while the absence of the right hip keypoint affects the relative position judgment of the mounting behavior. Experiments have verified that fewer than 17 keypoints significantly reduce the accuracy of behavior recognition, failing to meet the requirements for precise recognition.

[0057] In one specific embodiment, the monkey joint density adaptation branch and limb proportion perception module are constructed as monkey-specific feature extraction modules, used to specifically learn species-specific skeletal morphology and proportion features such as flat faces, long limbs, and flexible movement postures. The monkey joint density adaptation branch structure is a serial "two 3×3 convolutional layers + one global average pooling layer": the first convolutional layer (64 convolutional kernels, stride 1, padding=1, GELU activation) extracts local joint features, the second convolutional layer (128 convolutional kernels, stride 1, padding=1, GELU activation) fuses structural features, and the global average pooling layer outputs a 64-dimensional monkey-specific global skeletal feature vector. This branch is concatenated and fused along the channel dimension with the 1 / 4 resolution feature map of the overall skeleton detection module HRNet-W48, enabling the model to combine general skeletal features with monkey-specific features simultaneously, significantly reducing the keypoint false negative rate and localization bias, and improving the species adaptability of feature extraction. The overall skeleton detection module performs general skeletal feature extraction (pose estimation features). In another embodiment, the overall skeleton detection module further includes a limb proportion perception module. The posture estimation features are resampled by the limb proportion perception module to obtain monkey posture estimation features. The monkey posture estimation features are fused with monkey-specific features (global skeleton features) to obtain the monkey key point positions and connections.

[0058] In HRNet-W48, the weights are pre-trained on human subjects. The first three layers of the backbone network are frozen, and only the adaptation branch and the last layer are trained. When fine-tuning on the monkey dataset, the learning rate is set to 1e-4 (1 / 10 of the human training rate) to avoid overfitting.

[0059] In one specific embodiment, the core proportional differences between monkeys and humans are reflected in the following: the trunk accounts for approximately 20%-30% (humans approximately 40%), the limbs account for approximately 60%-70% (humans approximately 50%), and the tail accounts for approximately 10%-20% (humans have none). Furthermore, the limb joints have wider spacing and a more dispersed spatial distribution. Traditional human posture models rely on pre-defined human proportions for their receptive fields and feature encoding. Direct transfer of these proportions can lead to limb joint positioning shifts and over-extraction of trunk features. Therefore, the core of the limb proportion perception module is to allow the network to autonomously learn the prior proportions of the monkey's limbs, rather than manually setting rules. Through an end-to-end approach of "proportion encoding → offset generation → receptive field adaptation," the spatial range and weight distribution of feature extraction are dynamically adjusted to match the spatial resolution and receptive field size of the feature map with the actual proportions of the monkey's limbs. Specifically: The core of the proportional encoding branch is to extract global proportional features from high-resolution feature maps and transform the relative length and spatial distribution of monkey limbs into learnable numerical parameters: through local feature aggregation + global pooling + dimension mapping, the features of four key regions, namely the trunk, upper limbs, lower limbs and tail, are first locally encoded, and then integrated into global proportional parameters through global average pooling. Finally, a 4-dimensional vector is output (corresponding to the relative length ratio of the trunk / upper limb / lower limb / tail). This vector is the "numerical prior" of the monkey's limb proportion.

[0060] The module's input data undergoes local feature aggregation via four-channel convolution, outputting attention maps for the torso, upper limbs, lower limbs, and tail. This allows the network to automatically focus on the corresponding limb regions, avoiding background interference. Global average pooling then transforms the spatial features of each region into a single numerical value, mapping the pooled data directly to the length proportion of that region. Simultaneously, the module's input data undergoes spatial distribution prior encoding (two-dimensional convolution + activation + pooling + linear layers) to supplement the spatial position features of the limbs (e.g., monkeys have more forward upper limbs and longer lower limbs), avoiding focusing solely on length while ignoring the proportional differences in spatial position.

[0061] Furthermore, dynamic spatial offsets are calculated based on the spatial proportions and the prior proportions of the monkey's limbs. The generated dynamic offset vectors are applied to each pixel of the feature map, and spatial position adjustments are achieved through deformable convolutions. The deformable convolutions change the sampling position of the convolution kernel according to the offset vector: in the long limb regions of the monkey, the sampling range of the convolution kernel is expanded (to adapt to the spatial distribution of long limbs); in the short trunk regions, the sampling range is reduced (to avoid redundant features), achieving proportionally adaptive feature extraction. When the monkey's limb proportion is large, the network uses more 5×5 convolutions with large receptive fields to adapt to the large-scale feature extraction of long limbs; when the trunk proportion is small, the network uses more 1×1 convolutions with small receptive fields to adapt to the local feature extraction of short trunks, always retaining 3×3 convolutions as the foundation to ensure the stability of feature extraction.

[0062] S204. Extract primate appearance features and temporal features from the processed image data to obtain appearance features and temporal features; In one embodiment, the primate appearance features are obtained by feature extraction using a neural network model, which includes any one or more of the following: convolutional neural network, residual network; Optionally, the appearance features include limb proportions, joint angles, limb angles, body curvature, center of gravity offset, limb symmetry, and tail swing amplitude.

[0063] Limb proportions (distance features): Calculate the distance between facial key points (e.g., the distance from the nose to the left / right eye) and the distance between limb key points (e.g., the distance from the left shoulder to the left elbow, and the distance from the left hip to the left knee) to characterize the degree of limb extension; Angle features (joint angles, limb angles): Calculate the vector angle formed by adjacent key points (e.g., the bending angle of the left shoulder-left elbow-left wrist, and the bending angle of the left hip-left knee-left foot) to characterize joint posture.

[0064] In one embodiment, the appearance features further include normalization processing, which normalizes the limb proportions, joint angles, and limb angles to obtain normalized appearance features, and then uses the normalized appearance features, temporal features, key positions of the primate skeleton, and connection relationships to perform primate behavior recognition to obtain recognition results.

[0065] In another embodiment, behavior recognition (temporal features, appearance features, skeleton detection key point positions): For the coordinates of 17 key points in 5 consecutive frames, calculate the inter-frame displacement vector (such as the reciprocating displacement of the right wrist (adapting to grooming behavior), the rotation speed of the head (nose + eyes) (adapting to alert behavior), and the change in the degree of curling of the limbs (adapting to avoidance behavior)) to capture the dynamic features of the behavior.

[0066] In one embodiment, the temporal features are obtained by feature extraction using a temporal network model, which includes any one or more of the following: LSTM, GRU, and RNN.

[0067] Temporal features are used to ensure consistency across multiple frames, improve accuracy, and avoid missed or false detections.

[0068] S205. The recognition result is obtained by recognizing the behavior of primates through the key positions and connections of the primate skeleton, appearance features and temporal features.

[0069] In one embodiment, the monkey behaviors include mounting, stroking, alertness, and avoidance. Monkey behavior recognition types include... Figure 7 As shown, the behavior of the monkeys under test is used for identification and classification, and the behavior identification results are displayed.

[0070] Straddling behavior: the relative position of the right hip and the other person's torso, and the extension state of the limbs; Stroking behavior: the distance between the right wrist and the body parts of oneself / companion, and the characteristics of the reciprocating movement of the wrist; Be alert to the following behaviors: the angle of head (nose + eyes) rotation, and the tension in the limbs; Avoidance behaviors: the degree of trunk curling and the angle of limb contraction.

[0071] In one specific embodiment, the skeleton features are concatenated with the appearance features (extracted by CNN) and temporal features (extracted by LSTM for continuous frame motion information) of the multi-view images to construct a multi-dimensional feature vector; The classification network employs a hybrid structure of "MLP + Transformer encoder" (128-dimensional → 256-dimensional), focusing on capturing feature dependencies strongly associated with the four behaviors. The output behavior recognition result is activated by Softmax. It uses the cross-entropy loss function, the AdamW optimizer, with an initial learning rate of 5e-5, 80 iterations, and the learning rate decays to 1 / 5 of its original value every 16 iterations. Temporal smoothing: A sliding window voting mechanism with a length of 10 frames is adopted to select the behavior category with the highest confidence and the most occurrences in the window as the final result, reducing the false detection rate of a single frame to below 1%.

[0072] Hardware deployment: Each module is deployed on an edge computing device, and the camera is connected to the computing device via an Ethernet interface to transmit data in real time; Real-time performance: Each algorithm module is based on the PyTorch framework for inference, with a single frame image processing time of ≤50ms. It supports real-time behavior recognition and result display, meeting the real-time observation needs of scientific research scenarios.

[0073] In one embodiment, the training of the network model involves: acquiring an unlabeled multi-sample, multi-view image dataset; calculating the uncertainty entropy value of each sample in the unlabeled multi-sample, multi-view image dataset; selecting samples with entropy values ​​greater than a preset value and labeling them to obtain a core training set; training the model using the core training set to obtain an initial model; unlabeling the remaining unlabeled data; selecting pseudo-labeled samples with confidence values ​​greater than a second preset value and adding them to the core training set to obtain a second training set; training the initial model using the second training set until the model performance stabilizes to obtain the network model. The network model includes an osteolysis detection network model, a neural network model for extracting appearance features, and a temporal network model.

[0074] In one embodiment, the extraction of appearance features or normalized appearance features serves to assist in the analysis of the monkey's behavior, helping to distinguish between mounting, grooming, avoidance, and alertness behaviors. Secondly, the large differences in body size among monkeys of different breeds and ages can lead to biases in the calculation of behavioral features. By normalizing the extracted appearance features (distance between key points, joint angles) and normalizing the trunk length (based on the straight-line distance between the left shoulder and the right hip), individual differences among monkeys of different body sizes are eliminated, ensuring the consistency and comparability of behavioral features.

[0075] In one specific embodiment, the active learning process is as follows: Initially, collect 1000-2000 frames of unlabeled multi-view data. Calculate the uncertainty entropy value of each sample using a pre-trained model. Select the top 20% of samples (approximately 200-400 frames) by manually labeling them to construct a core training set. Train the initial model using the core training set. Perform pseudo-labeling (behavioral labels and keypoints estimated by the model) on the remaining unlabeled data. Select pseudo-labeled samples with a confidence level ≥ 0.9 to supplement the training set. After 3-5 iterations, the model performance tends to stabilize, and no further expansion of labeled data is required.

[0076] The present invention also discloses a computer program product or system, including a computer program that, when executed by a processor, implements the above-described method steps.

[0077] This invention provides a data processing system for primate behavior recognition, comprising: Acquisition Unit: Acquire a multi-view image dataset of primate behavioral patterns; Matching unit: The multi-view image data of the same behavioral pattern of the same primate sample in the multi-view image dataset are aligned by timestamp, and then the core skeleton key points and behavioral related feature points are extracted for each aligned image frame to obtain the matching feature point coordinate set. Alignment unit: Spatial mapping of the coordinate sets of matching feature points from different perspectives to obtain multi-view spatial alignment features; The completion unit performs similarity calculation on the multi-view spatial alignment features. When the similarity is greater than or equal to the first preset threshold, it is determined to be a valid feature; otherwise, it is determined to be an occluded region. The data of the occluded region is completed by using valid features from different perspectives in the same space to obtain a complete feature set.

[0078] The system for recognizing primate behavior provided in this embodiment of the invention specifically includes: Acquisition Unit: Acquires multi-view image data of the primate to be tested; Processing unit: Inputs the multi-view image data into the above-mentioned data processing method for primate behavior recognition to obtain processed image data; Skeleton detection unit: performs skeleton detection on the processed image data to obtain the key locations and connections of the primate skeleton; Feature unit: Extracting primate appearance features and temporal features from the processed image data to obtain appearance features and temporal features; Recognition Unit: The recognition result is obtained by recognizing the behavior of primates through the key positions and connections of the primate skeleton, appearance features, and temporal features.

[0079] Figure 3 An embodiment of the present invention provides a schematic diagram of a computer device, specifically including: A memory and a processor; the memory is used to store program instructions; the processor is used to invoke the program instructions, when the program instructions are executed any of the above-described methods for recognizing primate behavior or data processing methods for recognizing primate behavior.

[0080] The present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, is any of the above-described data processing methods for recognizing primate behavior or for recognizing primate behavior.

[0081] The verification results of this verification embodiment show that assigning inherent weights to indications can improve the performance of this method compared to the default settings. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated; the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of this embodiment. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0082] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0083] The computer device provided by the present invention has been described in detail above. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the embodiments of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data processing method for primate behavior recognition, characterized in that, include: Obtain a multi-view image dataset of primate behavioral patterns; The multi-view image data of the same behavioral pattern of the same primate sample in the multi-view image dataset are aligned by timestamp, and then the core skeleton key points and behavioral related feature points are extracted for each aligned image frame to obtain the matching feature point coordinate set. Multi-view spatial alignment features are obtained by spatially mapping the coordinate sets of matching feature points from different perspectives. The similarity of the multi-view spatial alignment features is calculated. When the similarity is greater than or equal to the first preset threshold, it is determined to be a valid feature. Otherwise, it is determined to be an occluded region. The data of the occluded region is supplemented by the valid features of different views in the same space to obtain a complete feature set.

2. The data processing method for primate behavior recognition according to claim 1, characterized in that, The occlusion region completion calculates the similarity of effective features from different perspectives in the same space, takes the one with the highest similarity as the behavioral feature of the reference perspective, and completes the data of the occlusion region by combining the behavioral feature of the reference perspective with the constraint of primate movement continuity. The primate motion continuity constraint is that the displacement of feature points in adjacent frames is limited to less than or equal to L pixels, where L is a natural number greater than or equal to 5.

3. The data processing method for primate behavior recognition according to claim 1, characterized in that, The spatial mapping is performed through a spatial mapping model. The coordinate sets of matching feature points from different perspectives are input into the spatial mapping model to obtain multi-view spatial alignment features. The construction of the spatial mapping model includes: obtaining the coordinate sets of matching feature points from different perspectives, performing feature point matching on the coordinate sets of matching feature points from different perspectives to obtain matching feature point pairs, and constructing the spatial mapping model through the matching feature point pairs.

4. A method for identifying primate behavior, characterized in that, include: Acquire multi-view image data of the primate to be tested; The multi-view image data is input into the data processing method for primate behavior recognition as described in any one of claims 1-3 to obtain processed image data; Skeleton detection is performed on the processed image data to obtain the key locations and connections of the primate skeleton; The processed image data is subjected to primate appearance features and temporal features extraction to obtain appearance features and temporal features; The identification results were obtained by identifying primate behavior through the key locations and connections of the primate skeleton, appearance features, and temporal features.

5. The method for identifying primate behavior according to claim 4, characterized in that, The skeleton detection is performed using a skeleton detection network model to obtain the positions and connections of key points in the skeleton. The skeleton detection network model includes a global skeleton detection module and a primate joint density adaptation module. The global skeleton detection module estimates the primate posture features using a posture detection model, and then inputs these estimated features into the primate joint density adaptation module. The primate joint density adaptation module includes N sequential convolutional modules and an average pooling layer, where N is a natural number greater than 1. Local joint features are extracted through the first convolutional module, and then structural features are extracted through the second convolutional module. The structural features are then input into the average pooling layer to obtain the global primate skeletal features. The primate posture estimation features and the global primate skeletal features are fused to obtain the positions and connections of key points in the primate.

6. The method for identifying primate behavior according to claim 5, characterized in that, The overall skeleton detection module also includes a limb proportion perception module. It processes image data to obtain multi-scale general features through a pose detection model, and inputs the multi-scale general features into the limb proportion perception module. It performs proportional encoding and spatial distribution prior encoding based on the relative lengths of the primate trunk and limbs to generate dynamic spatial offsets. Variable convolution performs proportional adaptive sampling through dynamic spatial offsets to obtain primate pose estimation features.

7. The method for identifying primate behavior according to claim 5, characterized in that, Network model training: Obtain an unlabeled multi-sample multi-view image dataset, calculate the uncertainty entropy value of each sample in the unlabeled multi-sample multi-view image dataset, filter samples with entropy values ​​greater than a preset value, label them to obtain a core training set, train the model using the core training set to obtain an initial model, unlabel the remaining unlabeled data, select pseudo-labeled samples with confidence values ​​greater than a second preset value and add them to the core training set to obtain a second training set, train the initial model using the second training set until the model performance tends to stabilize, and obtain the network model.

8. A computer program product comprising a computer program or instructions, characterized in that, The computer program or instructions are executed by the processor to implement the data processing method for primate behavior recognition as described in any one of claims 1-3 or the method for recognizing primate behavior as described in any one of claims 4-7.

9. A computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, characterized in that, The computer program or instructions are executed by the processor to implement the data processing method for primate behavior recognition as described in any one of claims 1-3 or the method for recognizing primate behavior as described in any one of claims 4-7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instructions are executed by the processor to implement the data processing method for primate behavior recognition as described in any one of claims 1-3 or the method for recognizing primate behavior as described in any one of claims 4-7.

Images