A giant panda delivery period prediction method based on multi-level behavior analysis
By employing multi-level behavioral analysis and multi-task neural networks, the accuracy and cost issues in analyzing giant panda pregnancy were addressed, enabling efficient and automated pregnancy prediction and improving the accuracy of behavioral recognition and the applicability of the model.
Patent Information
- Application Number
- CN202211653700.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-22
- Publication Date
- 2026-01-13
- Estimated Expiration
- 2042-12-22
AI Technical Summary
Existing technologies for analyzing giant panda pregnancy suffer from insufficient accuracy, high cost, and limited applicability, making large-scale application difficult.
A multi-level behavior analysis method was adopted, which defined and labeled the giant panda behavior in layers through a multi-task behavior recognition network. The behavior time series data was analyzed by combining LSTM feature extraction and ECA attention mechanism to predict the pregnancy status of giant pandas.
It has achieved high-precision prediction of giant panda pregnancy, reduced the need for human intervention, improved the accuracy of behavior recognition, and supports automatic operation around the clock. The model's analysis capabilities continue to improve as data accumulates.
Smart Images

Figure CN116091962B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of endangered animal protection, and in particular relates to a method for predicting the birthing period of giant pandas based on multi-level behavioral analysis. Background Technology
[0002] Giant pandas are characterized by difficulty conceiving and a high rate of miscarriage, resulting in a relatively small population. To protect the giant panda species and increase its population, panda breeding has always been a key research area in giant panda conservation efforts. Timely and accurate analysis of panda pregnancy status has also been crucial in panda breeding. Early determination of pregnancy and its stage allows for targeted feeding and management, better ensuring the pandas' health while reducing the workload of keepers.
[0003] Traditional analyses of giant panda pregnancy or studies of gestational activity rhythms heavily rely on hormone detection or human observation. Existing technology one proposes predicting the due date of a giant panda by detecting estradiol and progesterone levels in its urine. A regression equation is constructed based on the interval between the true peak of estradiol concentration and the end of the first rise in progesterone concentration to determine the due date. This method requires continuous human monitoring and a rapid response after urination, heavily relying on the timing of defecation and making timed data collection difficult. Existing technology two proposes analyzing behavioral changes during pregnancy to determine the stage of gestation. It uses instantaneous scanning and full-event recording methods to statistically analyze panda behavior, thereby determining whether a panda is experiencing pseudopregnancy and the stage of pregnancy. This method has advantages such as minimal impact on the panda and ease of operation, but it heavily relies on the experience of keepers and consumes significant human and material resources, making large-scale implementation difficult. Summary of the Invention
[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a method for predicting the birthing period of giant pandas based on multi-level behavioral analysis. This method divides giant panda behavior into four levels and defines typical behaviors for each level. By acquiring effective features of each level and temporal descriptions of giant panda behavioral activities, it solves the problems of inaccurate analysis of giant panda pregnancy, high cost, and insufficient universality.
[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:
[0006] This invention provides a method for predicting the birthing date of giant pandas based on multi-level behavioral analysis, comprising the following steps:
[0007] S1. Define giant panda behavior at multiple levels and label giant panda behavior activity videos to obtain multi-level behavior recognition training data;
[0008] S2. Construct a multi-task behavior recognition network;
[0009] S3. Input the multi-level behavior recognition training data into the multi-task behavior recognition network for training to obtain the multi-level behavior recognition model.
[0010] S4. Use a multi-level behavior recognition model to perform multi-level behavior recognition on the giant panda video to be analyzed, and obtain giant panda behavior time series data.
[0011] S5. Analyze the time-series data of giant panda behavior using the pregnancy analysis module to obtain the prediction results of giant panda birthing period.
[0012] Further, step S1 includes the following steps:
[0013] S11. By defining giant panda behavior at multiple levels, the definition results of giant panda behavior are obtained. Among them, the definition results of giant panda behavior include posture-level behavior, action-level behavior, ordinary behavior-level behavior, and pregnancy-level behavior.
[0014] S12. Based on the definition of giant panda behavior, each frame of the giant panda behavior activity video is labeled, and the position of the giant panda in each frame is marked using bounding boxes to obtain multi-level behavior recognition training data. Each frame has annotation information at the posture level, action level, ordinary behavior level, and pregnancy level.
[0015] Furthermore, the multi-task behavior recognition network includes:
[0016] The Fast channel is used to obtain high-speed motion information features in videos of giant panda behavior and activities through high-frequency sampling.
[0017] The Slow channel is used to obtain background information features in giant panda behavior videos through low-frequency sampling, and then fuses the background information features with high-speed motion information features multiple times to obtain behavioral features.
[0018] The fully connected layer is used to fuse high-speed motion information features and behavioral features, and outputs the feature fusion results to the pose prediction head, action prediction head, ordinary behavior prediction head and pregnancy behavior prediction head for behavior recognition and prediction.
[0019] Pose prediction head, used to identify pose-level behaviors based on feature fusion results;
[0020] Action prediction head, used to identify hierarchical behavior based on feature fusion results;
[0021] A general behavior prediction head is used to identify general behavior hierarchical behaviors based on feature fusion results.
[0022] A pregnancy behavior prediction head is used to identify hierarchical pregnancy behaviors based on feature fusion results.
[0023] Further, step S4 includes the following steps:
[0024] S41. Set a fixed time window and divide the panda video to be analyzed according to the length of the fixed time window to obtain several video segments to be analyzed.
[0025] S42. Input each video segment to be analyzed into the multi-level behavior recognition model, and use a stride sliding time window with the same length as the fixed time window to perform recognition, and obtain the multi-level behavior recognition results.
[0026] S43. Based on the results of multi-level behavior recognition, the proportion of the duration of various behaviors occurring in each natural day to the total duration of the giant panda videos to be analyzed is calculated to obtain the time series data of giant panda behavior.
[0027] Furthermore, the calculation expression for the giant panda behavior time-series data is as follows:
[0028] V = {v1, v2, ..., v} t ,…,v T}
[0029] v t ∈R 1×N
[0030] Where V represents the statistical measure of the multi-level behavioral characteristics of giant pandas, v t R represents the probability distribution of various behaviors on day t, where T represents the maximum number of days of observation of the giant panda videos to be analyzed. 1×N Let N represent the probability distribution array of each behavior in the posture level behavior, action level behavior, ordinary behavior level behavior and pregnancy behavior level behavior, and let N represent the total number of behaviors in the posture level behavior, action level behavior, ordinary behavior level behavior and pregnancy behavior level behavior, where t = 1, 2, ..., T.
[0031] Furthermore, the pregnancy status analysis module includes:
[0032] The LSTM feature extraction module is used to obtain a fixed-length feature embedding vector based on the ever-growing time-series data of giant panda behavior.
[0033] The ECA attention mechanism module is used to perform global pooling on the feature embedding vector to obtain the ECA attention-weighted feature vector;
[0034] The classification prediction head module is used to classify pregnant giant pandas based on ECA attention-weighted feature vectors, and to predict the birth date of pregnant giant pandas using the prediction head, thus obtaining the giant panda birth date prediction results.
[0035] Further, step S5 includes the following steps:
[0036] S51. Input the time series data of giant panda behavior into several LSTM units in the LSTM feature extraction module respectively;
[0037] S52. Update the memory state and hidden layer nonlinearly using the forget gate, input gate and output gate of each LSTM unit;
[0038] S53. Concatenate the last layer of the hidden layer output of the LSTM units corresponding to various behaviors to obtain the feature embedding vector;
[0039] S54. Based on the ECA attention mechanism module, the feature embedding vector is globally pooled to obtain a vector composed of the global pooling results of each action:
[0040] g = GAP(H)
[0041] Where GAP(·) represents global pooling, and g represents a vector composed of the global pooling results of each behavior;
[0042] S55. Perform a one-dimensional convolution on the vector composed of the global pooling results of each behavior to obtain the attention coefficients of each behavior:
[0043] α=Conv1d(g)
[0044] Where α represents the attention coefficient of each behavior, and Conv1d(·) represents one-dimensional convolution;
[0045] S56. Based on the attention coefficients of each behavior, obtain the ECA attention-weighted feature vector:
[0046] E = α × H
[0047] Where E represents the ECA attention-weighted feature vector;
[0048] S57. Using the classification head based on the ECA attention-weighted feature vector, obtain the pregnant giant panda;
[0049] S58. Using the prediction head, the birth date of pregnant giant pandas is predicted, and the birth date prediction results of giant pandas are obtained.
[0050] Furthermore, the calculation expression for the LSTM unit is as follows:
[0051]
[0052]
[0053]
[0054]
[0055]
[0056] Among them, f t i Let σ(·) represent the forget gate of the i-th LSTM unit, and let σ(·) represent the sigmoid function. Let represent the forget gate weight matrix of the i-th LSTM unit, and · denote matrix multiplication. This represents the previous hidden layer of the i-th LSTM unit. This represents the value corresponding to the i-th row on day t. This represents the forget gate bias vector of the i-th LSTM unit. This represents the input gate of the i-th LSTM unit. This represents the input gate weight matrix of the i-th LSTM unit. Let represent the input gate bias vector of the i-th LSTM unit. This represents the output gate of the i-th LSTM unit. This represents the output gate weight matrix of the i-th LSTM unit. This represents the output gate bias vector of the i-th LSTM unit. This represents the memory state of the i-th LSTM cell. Let represent the previous memory state of the i-th LSTM cell, and tanh(·) denote the hyperbolic tangent function. This represents the memory state weight matrix of the i-th LSTM unit. This represents the memory state bias vector of the i-th LSTM cell. Let represent the hidden layer of the i-th LSTM unit, and * denote the Hadamard product, where i is a non-zero natural number.
[0057] Furthermore, the calculation expression for the feature embedding vector is as follows:
[0058]
[0059]
[0060] Where H represents the feature embedding vector. This represents the i-th action. d represents the set of behavioral feature vectors. h This represents the dimension of the hidden layer of an LSTM unit.
[0061] The beneficial effects of this invention are as follows: This invention provides a method for predicting the birthing period of giant pandas based on multi-level behavioral analysis. It describes the behavior of giant pandas from multiple perspectives, including both shallow posture information and deep state information, which can better reveal the behavioral patterns of giant pandas at different stages of pregnancy. This invention provides a multi-task neural network model that can efficiently detect multi-level behaviors of giant pandas. This model achieves good simultaneous capture of background and action information through fast and slow dual channels, improving the accuracy of behavior recognition. Furthermore, by simultaneously inputting several levels of behavior labels and bounding boxes of giant pandas for multi-task learning, and by jointly training the four levels of behavior labels through multi-task training, it can not only share weights and network skeletons, reducing the amount of computation and model parameters, but also make full use of the labeling information of different behavior levels to improve the recognition performance of each level of behavior. This invention is based on video analysis and prediction, which can achieve all-weather, fully automatic operation with only minimal human intervention. As the system is used, the continuous accumulation of relevant data can improve the model's analytical capabilities. Attached Figure Description
[0062] Figure 1 This is a flowchart illustrating the steps of a method for predicting the birthing date of giant pandas based on multi-level behavioral analysis, as described in an embodiment of the present invention.
[0063] Figure 2 This is a schematic diagram of a multi-task behavior recognition network in an embodiment of the present invention.
[0064] Figure 3 This is a schematic diagram of the LSTM unit obtaining feature embedding vectors from behavioral time-series data in an embodiment of the present invention.
[0065] Figure 4 This is a schematic diagram illustrating the pregnancy analysis module's analysis of giant panda birthing period in an embodiment of the present invention. Detailed Implementation
[0066] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0067] like Figure 1 As shown, in one embodiment of the present invention, the present invention provides a method for predicting the birthing date of giant pandas based on multi-level behavioral analysis, comprising the following steps:
[0068] S1. Define giant panda behavior at multiple levels and label giant panda behavior activity videos to obtain multi-level behavior recognition training data;
[0069] Step S1 includes the following steps:
[0070] S11. By defining giant panda behavior at multiple levels, the definition results of giant panda behavior are obtained. Among them, the definition results of giant panda behavior include posture-level behavior, action-level behavior, ordinary behavior-level behavior, and pregnancy-level behavior.
[0071] S12. Based on the definition of giant panda behavior, each frame of the giant panda behavior activity video is labeled, and the position of the giant panda in each frame is marked by bounding boxes to obtain multi-level behavior recognition training data. Each frame has annotation information at the posture level, action level, ordinary behavior level and pregnancy level.
[0072] The section on the classification and construction of multi-level behaviors defines 46 typical behaviors of giant pandas across four levels: posture, action, ordinary behavior, and pregnancy behavior. Posture is a behavior that can be judged from a single frame image, representing an instantaneous (shallow) state; action can be judged from short video clips, representing a short-term dependency; ordinary behavior represents a longer-term dependency, indicating deeper state information; pregnancy behavior, in addition to possessing the characteristics of ordinary behavior, is also a behavior that occurs more frequently during pregnancy and rarely during other periods, and has different distribution characteristics at different stages of pregnancy. The behavioral definitions at the posture, action, ordinary behavior, and pregnancy levels correspond one-to-one as shown in Tables 1, 2, 3, and 4.
[0073] Table 1 Definition of Postural Hierarchy Behavior
[0074]
[0075]
[0076] Table 2 Definition of Action Hierarchy Behavior
[0077]
[0078]
[0079] Table 3 Definition of General Behavior Level
[0080] name describe drink Drinking liquid by sucking it from the water bowl (jar). feeding Use their mouth and paws to handle and swallow food. sports Moving within the area, including walking, running, and climbing. rest In one place, either lying down or sitting, occasionally changing position. Explore Explore objects with your mouth, nose, or paws; stand and gaze. play Without a clear purpose, such as splashing in water, rolling around, playing games, and playing with various objects. mark Apply to perianal glands, vulva, feces, urine, or other objects. Elimination Excretion of feces or urine Modification Scratching with paws or licking its fur with its mouth, scratching itself, etc.
[0081] Table 4 Definitions of Pregnancy Behavior Levels
[0082]
[0083]
[0084] Multi-level behavior definitions were used to annotate videos of giant pandas during their pregnancy. At each moment, there were four levels of annotation information: posture, action, normal behavior, and pregnancy behavior. In addition, bounding boxes were used to mark the position of the giant panda in the video. This not only allows for the sharing of weights and network skeletons, reducing computation and model parameters, but also makes full use of the annotation information of different behavior levels to improve the recognition performance of each level of behavior.
[0085] S2. Construct a multi-task behavior recognition network;
[0086] The main framework of the multi-task action recognition network is the SlowFast framework. SlowFast has two channels, Slow and Fast. The Slow channel samples the video at a low frequency of 2Hz to capture the background information of the video, while the Fast channel samples the video at a high frequency of 15Hz to capture the high-speed motion information in the video. At the same time, the two channels are fused multiple times to capture richer action features. After fusion in the fully connected layer, the network is connected to multiple prediction heads to recognize and predict actions at different levels. This network shares the main framework, which can capture information at other levels while reducing the number of network parameters and computational cost.
[0087] like Figure 2 As shown, the multi-task behavior recognition network includes:
[0088] The Fast channel is used to obtain high-speed motion information features in videos of giant panda behavior and activities through high-frequency sampling.
[0089] The Slow channel is used to obtain background information features in giant panda behavior videos through low-frequency sampling, and then fuses the background information features with high-speed motion information features multiple times to obtain behavioral features.
[0090] The fully connected layer is used to fuse high-speed motion information features and behavioral features, and outputs the feature fusion results to the pose prediction head, action prediction head, ordinary behavior prediction head and pregnancy behavior prediction head for behavior recognition and prediction.
[0091] Pose prediction head, used to identify pose-level behaviors based on feature fusion results;
[0092] Action prediction head, used to identify hierarchical behavior based on feature fusion results;
[0093] A general behavior prediction head is used to identify general behavior hierarchical behaviors based on feature fusion results.
[0094] A pregnancy behavior prediction head is used to identify hierarchical pregnancy behaviors based on feature fusion results.
[0095] S3. Input the multi-level behavior recognition training data into the multi-task behavior recognition network for training to obtain the multi-level behavior recognition model.
[0096] S4. Use a multi-level behavior recognition model to perform multi-level behavior recognition on the giant panda video to be analyzed, and obtain giant panda behavior time series data.
[0097] Step S4 includes the following steps:
[0098] S41. Set a fixed time window and divide the panda video to be analyzed according to the length of the fixed time window to obtain several video segments to be analyzed.
[0099] S42. Input each video segment to be analyzed into the multi-level behavior recognition model, and use a stride sliding time window with the same length as the fixed time window to perform recognition, and obtain the multi-level behavior recognition results.
[0100] S43. Based on the results of multi-level behavior recognition, the proportion of the duration of various behaviors occurring in each natural day to the total duration of the giant panda videos to be analyzed is statistically analyzed to obtain giant panda behavior time series data.
[0101] The calculation expression for the giant panda behavior time series data is as follows:
[0102] V = {v1, v2, ..., v} t ,…,v T}
[0103] v t ∈R 1×N
[0104] Where V represents the statistical measure of the multi-level behavioral characteristics of giant pandas, v t R represents the probability distribution of various behaviors on day t, where T represents the maximum number of days of observation of the giant panda videos to be analyzed. 1×N Let N represent the probability distribution array of each behavior in the posture-level behavior, action-level behavior, ordinary behavior-level behavior, and pregnancy behavior-level behavior, where N represents the total number of behaviors in the posture-level behavior, action-level behavior, ordinary behavior-level behavior, and pregnancy behavior-level behavior, and t = 1, 2, ..., T;
[0105] S5. Analyze the time-series data of giant panda behavior using the pregnancy analysis module to obtain the prediction results of giant panda birthing period;
[0106] The pregnancy analysis module includes:
[0107] The LSTM feature extraction module is used to obtain a fixed-length feature embedding vector based on the ever-growing time-series data of giant panda behavior.
[0108] The ECA attention mechanism module is used to perform global pooling on the feature embedding vector to obtain the ECA attention-weighted feature vector;
[0109] The classification and prediction head module is used to classify pregnant giant pandas based on ECA attention-weighted feature vectors, and to predict the birth date of pregnant giant pandas using the prediction head, thus obtaining the birth date prediction results. Both the classification head and the prediction head are composed of two fully connected layers, with the ReLU activation function used between the fully connected layers.
[0110] Step S5 includes the following steps:
[0111] like Figure 3 As shown in S51, input the time series data of giant panda behavior into several LSTM units in the LSTM feature extraction module respectively;
[0112] The calculation expression for the LSTM unit is as follows:
[0113]
[0114]
[0115]
[0116]
[0117]
[0118] Among them, f t i Let σ(·) represent the forget gate of the i-th LSTM unit, and let σ(·) represent the sigmoid function. Let represent the forget gate weight matrix of the i-th LSTM unit, and · denote matrix multiplication. This represents the previous hidden layer of the i-th LSTM unit. This represents the value corresponding to the i-th row on day t. This represents the forget gate bias vector of the i-th LSTM unit. This represents the input gate of the i-th LSTM unit. This represents the input gate weight matrix of the i-th LSTM unit. Let represent the input gate bias vector of the i-th LSTM unit. This represents the output gate of the i-th LSTM unit. This represents the output gate weight matrix of the i-th LSTM unit. This represents the output gate bias vector of the i-th LSTM unit. This represents the memory state of the i-th LSTM cell. Let represent the previous memory state of the i-th LSTM cell, and tanh(·) denote the hyperbolic tangent function. This represents the memory state weight matrix of the i-th LSTM unit. This represents the memory state bias vector of the i-th LSTM cell. denoted as the hidden layer of the i-th LSTM unit, and * denotes the Hadamard product, where i is a non-zero natural number;
[0119] S52. Update the memory state and hidden layer nonlinearly using the forget gate, input gate and output gate of each LSTM unit;
[0120] S53. Concatenate the last layer of the hidden layer output of the LSTM units corresponding to various behaviors to obtain the feature embedding vector;
[0121] The expression for calculating the feature embedding vector is as follows:
[0122]
[0123]
[0124] Where H represents the feature embedding vector. This represents the i-th action. d represents the set of behavioral feature vectors. h Indicates the dimension of the hidden layer of an LSTM unit;
[0125] like Figure 4 As shown, S54, based on the ECA attention mechanism module, globally pools the feature embedding vector to obtain a vector composed of the global pooling results of each action:
[0126] g = GAP(H)
[0127] Where GAP(·) represents global pooling, and g represents a vector composed of the global pooling results of each behavior;
[0128] S55. Perform a one-dimensional convolution on the vector composed of the global pooling results of each behavior to obtain the attention coefficients of each behavior:
[0129] α=Conv1d(g)
[0130] Where α represents the attention coefficient of each behavior, and Conv1d(·) represents one-dimensional convolution;
[0131] S56. Based on the attention coefficients of each behavior, obtain the ECA attention-weighted feature vector:
[0132] E = α × H
[0133] Where E represents the ECA attention-weighted feature vector;
[0134] S57. Using the classification head based on the ECA attention-weighted feature vector, obtain the pregnant giant panda;
[0135] S58. Using the prediction head, the birth date of pregnant giant pandas is predicted, and the birth date prediction results of giant pandas are obtained.
[0136] This invention describes giant panda behavior from multiple perspectives, including both superficial posture information and deep state information, enabling a better understanding of behavioral patterns during different stages of gestation. It provides a multi-task neural network model capable of efficiently detecting multi-level giant panda behavior. This model achieves good simultaneous capture of background and motion information through fast and slow dual-channel processing, improving the accuracy of behavior recognition. Furthermore, by simultaneously inputting several levels of giant panda behavior labels and bounding boxes for multi-task learning, and jointly training the four levels of behavior annotations through multi-task collaboration, it not only shares weights and network skeletons, reducing computational load and model parameters, but also fully utilizes the annotation information from different behavioral levels to improve the recognition performance of each level of behavior. Based on video analysis and prediction, this invention can operate 24 / 7 and fully automatically, requiring only minimal human intervention. As the system is used, the accumulated data continuously enhances the model's analytical capabilities.
Claims
1. A method for predicting the birthing date of giant pandas based on multi-level behavioral analysis, characterized in that, Includes the following steps: S1. Define giant panda behavior at multiple levels and label giant panda behavior activity videos to obtain multi-level behavior recognition training data; S2. Construct a multi-task behavior recognition network; S3. Input the multi-level behavior recognition training data into the multi-task behavior recognition network for training to obtain the multi-level behavior recognition model. S4. Use a multi-level behavior recognition model to perform multi-level behavior recognition on the giant panda video to be analyzed, and obtain giant panda behavior time series data. S5. Analyze the time-series data of giant panda behavior using the pregnancy analysis module to obtain the prediction results of giant panda birthing period; Step S1 includes the following steps: S11. By defining giant panda behavior at multiple levels, the definition results of giant panda behavior are obtained. Among them, the definition results of giant panda behavior include posture-level behavior, action-level behavior, ordinary behavior-level behavior, and pregnancy-level behavior. S12. Based on the definition of giant panda behavior, each frame of the giant panda behavior activity video is labeled, and the position of the giant panda in each frame is marked by bounding boxes to obtain multi-level behavior recognition training data. Each frame has annotation information at the posture level, action level, ordinary behavior level and pregnancy level. The multi-task behavior recognition network includes: The Fast channel is used to obtain high-speed motion information features in videos of giant panda behavior and activities through high-frequency sampling. The Slow channel is used to obtain background information features in giant panda behavior videos through low-frequency sampling, and then fuses the background information features with high-speed motion information features multiple times to obtain behavioral features. The fully connected layer is used to fuse high-speed motion information features and behavioral features, and outputs the feature fusion results to the pose prediction head, action prediction head, ordinary behavior prediction head and pregnancy behavior prediction head for behavior recognition and prediction. Pose prediction head, used to identify pose-level behaviors based on feature fusion results; Action prediction head, used to identify hierarchical behavior based on feature fusion results; A general behavior prediction head is used to identify general behavior hierarchical behaviors based on feature fusion results. A pregnancy behavior prediction head is used to identify hierarchical pregnancy behaviors based on feature fusion results. The pregnancy analysis module includes: The LSTM feature extraction module is used to obtain a fixed-length feature embedding vector based on the ever-growing time-series data of giant panda behavior. The ECA attention mechanism module is used to perform global pooling on the feature embedding vector to obtain the ECA attention-weighted feature vector; The classification prediction head module is used to classify pregnant giant pandas based on ECA attention-weighted feature vectors, and to predict the birth date of pregnant giant pandas using the prediction head, thus obtaining the giant panda birth date prediction results.
2. The giant panda birthing date prediction method based on multi-level behavioral analysis according to claim 1, characterized in that, Step S4 includes the following steps: S41. Set a fixed time window and divide the panda video to be analyzed according to the length of the fixed time window to obtain several video segments to be analyzed. S42. Input each video segment to be analyzed into the multi-level behavior recognition model, and use a stride sliding time window with the same length as the fixed time window to perform recognition, and obtain the multi-level behavior recognition results. S43. Based on the results of multi-level behavior recognition, the proportion of the duration of various behaviors occurring in each natural day to the total duration of the giant panda videos to be analyzed is calculated to obtain the time series data of giant panda behavior.
3. The giant panda birthing date prediction method based on multi-level behavioral analysis according to claim 2, characterized in that, The calculation expression for the giant panda behavior time series data is as follows: in, This represents the statistical measures of multi-level behavioral characteristics of giant pandas. Indicates the first t The probability distribution of various behaviors in a day. T This indicates the maximum number of days for which giant panda video observations are to be analyzed. This represents the probability distribution array of each behavior in the posture-level behavior, action-level behavior, ordinary behavior-level behavior, and pregnancy behavior-level behavior. N This represents the total number of behaviors across the posture-level, action-level, general behavior-level, and pregnancy-level behavior-level categories. t =1,2,…, T .
4. The method for predicting the birthing date of giant pandas based on multi-level behavioral analysis according to claim 3, characterized in that, Step S5 includes the following steps: S51. Input the time series data of giant panda behavior into several LSTM units in the LSTM feature extraction module respectively; S52. Update the memory state and hidden layer nonlinearly using the forget gate, input gate and output gate of each LSTM unit; S53. Concatenate the last layer of the hidden layer output of the LSTM units corresponding to various behaviors to obtain the feature embedding vector; S54. Based on the ECA attention mechanism module, the feature embedding vector is globally pooled to obtain a vector composed of the global pooling results of each action: in, Indicates global pooling. A vector representing the global pooling results of each behavior; S55. Perform a one-dimensional convolution on the vector composed of the global pooling results of each behavior to obtain the attention coefficients of each behavior: in, This represents the attention coefficient for each behavior. This represents one-dimensional convolution; S56. Based on the attention coefficients of each behavior, obtain the ECA attention-weighted feature vector: in, This represents the ECA attention-weighted feature vector; S57. Using the classification head based on the ECA attention-weighted feature vector, obtain the pregnant giant panda; S58. Using the prediction head, the birth date of pregnant giant pandas is predicted, and the birth date prediction results of giant pandas are obtained.
5. The giant panda birthing date prediction method based on multi-level behavioral analysis according to claim 4, characterized in that, The calculation expression for the LSTM unit is as follows: in, Indicates the first i Forget gates for LSTM units This represents the sigmoid function. Indicates the first i Forget gate weight matrix of LSTM units Represents matrix multiplication. Indicates the first i The hidden layer above each LSTM unit, Indicates the first t The timing of the day i The value corresponding to each behavior Indicates the first i Forget gate bias vector of each LSTM unit Indicates the first i The input gates of an LSTM unit, Indicates the first i The input gate weight matrix of each LSTM unit. Indicates the first i The input gate bias vector of each LSTM unit Indicates the first i The output gates of the LSTM unit Indicates the first i The output gate weight matrix of each LSTM unit Indicates the first i The output gate bias vector of each LSTM unit Indicates the first i The memory state of each LSTM cell, Indicates the first i The previous memory state of each LSTM cell. Represents the hyperbolic tangent function. Indicates the first i The memory state weight matrix of each LSTM cell. Indicates the first i The memory state bias vector of each LSTM unit Indicates the first i Hidden layers of LSTM units, This represents the Hadamarda product, in which, i It is a non-zero natural number.
6. The method for predicting the birthing date of giant pandas based on multi-level behavioral analysis according to claim 5, characterized in that, The expression for calculating the feature embedding vector is as follows: in, Represents the feature embedding vector. Indicates the first i One behavior, Represents a set of behavioral feature vectors. This represents the dimension of the hidden layer of an LSTM unit.
Citation Information
Patent Citations
Double-flow network behavior recognition method based on multi-level spatial-temporal feature fusion enhancement
CN111709306A
Panda behavior recognition method and system based on deep learning, terminal and medium
CN112016527A