Animal behavior recognition method and system based on deep learning

Abstract

The present invention discloses an animal behavior recognition method and system based on deep learning, and the method includes data collection, data reconstruction, individual behavior classification, group association analysis and animal behavior recognition. The present invention relates to the field of animal behavior recognition technology, and specifically refers to an animal behavior recognition method and system based on deep learning. The present invention adopts a comprehensive animal behavior recognition method combining individual analysis and group analysis, identifies specific behavior types through individual behavior analysis, and combines multiple animals for comprehensive analysis and clustering through group association analysis, thereby improving the recognition depth and comprehensive usability of animal behavior recognition; adopts a residual convolutional recurrent neural network with structural and attention improvements to classify individual behaviors; and adopts a dynamic and static graph neural integration network combined with reinforcement learning to perform group association analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of animal behavior recognition, and specifically to an animal behavior recognition method and system based on deep learning. Background Art

[0002] The animal behavior recognition method based on deep learning is an intelligent system that uses deep learning technology to analyze and identify the behavior of domestic pets (such as cats and dogs). This method processes the image or video data of pets, extracts their behavioral characteristics, such as eating, resting, running, playing, etc., and recognizes and classifies these behaviors in real time. Its role is to provide efficient and accurate behavior monitoring, which is applied to pet health management, behavior analysis, smart home control and other fields, helping pet owners better understand the status and needs of their pets and improve pet care and interactive experience.

[0003] However, among the existing animal behavior recognition methods, there are many existing solutions for individual animal behavior recognition, but there are few technical contents for integrated behavior analysis and recognition of individuals and groups, and it is even more difficult to reflect practical value when applied to domestic animals. Therefore, there is a lack of a technical problem of a method that can analyze the individual behavior of domestic animals and recognize group-related animal behavior;

[0004] Among the existing individual behavior classifications, there are traditional animal behavior classification methods that have poor recognition capabilities when dealing with complex backgrounds and local action details. At the same time, since domestic animals are not in a static state most of the time, they are more dynamic, which is reflected not only in the changes in action behaviors, but also in the changes in individual states. Therefore, the ability of the previous method to capture the details of animal behavior, the ability to perceive changes, and the ability to analyze dynamics need to be improved.

[0005] In the existing animal group association analysis, there are existing methods that focus on single-stage behavior analysis for animal group behavior analysis. However, in fact, the essence of an animal group is a complex interaction system of multiple individual animals. Single-stage group analysis is not only difficult to effectively understand the animal's behavior, emotional state, and interaction information, but also has a high probability of causing misinterpretation and misclassification, which in turn has a significant negative impact on the technical problem of animal group association analysis. Summary of the invention

[0006] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an animal behavior recognition method and system based on deep learning. In the existing animal behavior recognition methods, there are many solutions for the individual behavior recognition of animals in the prior art, but there are few technical contents for integrated behavior analysis and recognition of individuals and groups, and it is even more difficult to reflect practical value when applied to domestic animals. Therefore, there is a lack of a method that can analyze the individual behavior of domestic animals and identify the group-related animal behavior. The present solution creatively adopts a comprehensive animal behavior recognition method that combines individual analysis and group analysis, identifies specific behavior types through individual behavior analysis, and Through group association analysis combined with multiple animals for comprehensive analysis and clustering, the recognition depth and comprehensive usability of animal behavior recognition are improved; in the existing individual behavior classification, there are traditional animal behavior classification methods, which have poor recognition ability when dealing with complex backgrounds and local action details. At the same time, because domestic animals are not in a static state most of the time, they are more dynamic, which is reflected not only in the changes of action behavior, but also in the changes of individual state. Therefore, the previous method has the technical problem that the ability to capture the details of animal behavior, the ability to perceive changes and the ability to analyze dynamics need to be improved. This solution creatively adopts a residual convolutional recurrent neural network that combines structure and attention improvements to carry out By improving the traditional ResNet network structure and introducing two attention mechanisms, the ability to capture details and perceive changes in animal behavior is improved. At the same time, by integrating the outputs of the two recurrent networks, the dynamic analysis capabilities required for the classification of individual animal behaviors can be more comprehensively addressed, which improves the overall versatility of the method and provides a good technical prerequisite for group analysis. In the existing animal group association analysis, the existing methods focus on single-stage behavior analysis for animal group behavior analysis. However, in fact, the essence of an animal group is a complex interactive system of multiple individual animals. Single-stage group analysis is not only difficult to effectively understand the behavior and emotional state of animals, but also difficult to effectively understand the behavior and emotional state of animals. The dynamic and static causal analysis of the interaction relationship between individual animals and groups is improved through the integration of dynamic and static graph neural networks. At the same time, by introducing reinforcement learning methods, it can better output the optimal interpretation method for the effect of individual animal behavior on the group. Finally, the clustering method is used to realize the correlation analysis of animal groups, which improves the comprehensive usability of animal behavior recognition and provides certain practical experience and technical support for the field of animal research.

[0007] The technical solution adopted by the present invention is as follows: The animal behavior recognition method based on deep learning provided by the present invention comprises the following steps:

[0008] Step S1: data collection;

[0009] Step S2: data reconstruction;

[0010] Step S3: individual behavior classification;

[0011] Step S4: group association analysis;

[0012] Step S5: Animal behavior recognition.

[0013] Furthermore, in step S1, the data collection is used to collect raw data required for animal behavior analysis, specifically, to capture animal behavior data through multi-source data collection to obtain an original data set for animal behavior recognition;

[0014] The animal behavior recognition raw data set specifically includes video data, label data, sensor data, sound data, environmental data, spatial data, temporal data, information data and health data;

[0015] The animals specifically refer to domestic pet animals, including domestic cats and dogs.

[0016] Further, in step S2, the data reconstruction is used to construct a data set for training individual and group behavior analysis sub-item models and perform data enhancement and optimization processing, specifically, through data enhancement and optimization operations, the original animal behavior recognition data set is reconstructed into a format suitable for the training model to obtain an animal behavior recognition optimized data set, which specifically includes the following steps:

[0017] Step S21: data enhancement, specifically performing video data enhancement, sensor data enhancement and sound data enhancement on the original animal behavior recognition data set in sequence to obtain an enhanced data set;

[0018] Step S22: data preprocessing, specifically, performing video preprocessing, sensor data preprocessing and sound data preprocessing on the enhanced data set in sequence, and performing artificial feature extraction to obtain an optimized feature data set;

[0019] Step S23: data conversion and fusion, specifically, converting the original data and the corresponding data labels into a unified format, and aligning them according to the timestamps, and then performing feature fusion on the feature data in the optimized feature data set to obtain an integrated feature data set;

[0020] Step S24: data division, specifically, dividing the original data and the integrated feature data set into a training set, a validation set and a test set, and performing category balance detection and adjustment to obtain a divided data set;

[0021] Step S25: Data reconstruction, specifically, performing data reconstruction through the data enhancement, the data preprocessing, the data conversion and fusion, and the data partitioning to obtain an animal behavior recognition optimized data set, wherein the animal behavior recognition optimized data set specifically includes a partitioned data set and an integrated feature data set.

[0022] Further, in step S3, the individual behavior classification is used to identify and classify the individual behavior types of animals and provide detailed behavior classification results, specifically, based on the animal behavior recognition optimization data set, using a residual convolutional recurrent neural network with combined structure and attention improvement to perform individual behavior classification and obtain detailed animal individual behavior classification data;

[0023] The residual convolutional recurrent neural network with combined structure and attention improvement specifically includes an improved residual convolutional subnet, a visual detection subnet and a recurrent integration subnet;

[0024] The step of using the residual convolutional recurrent neural network with combined structure and attention improvement to classify individual behaviors and obtain detailed classification data of individual animal behaviors includes:

[0025] Step S31: constructing an improved residual convolutional subnet, specifically, constructing an improved residual convolutional subnet by improving the structure and attention of the standard residual convolutional network;

[0026] The structural improvement and attention improvement specifically include structural improvement and attention improvement;

[0027] The structural improvement specifically refers to replacing the standard convolution layer with an integrated convolution after being processed by a temporal convolution layer and a depth-separable convolution layer;

[0028] The attention improvement specifically refers to the application of spatial attention and channel attention mechanisms to improve attention;

[0029] Step S32: constructing a visual detection subnet, specifically using a pre-trained computer vision target detection model to perform target detection and obtain target detection positioning features;

[0030] Step S33: constructing a recurrent integration subnetwork, specifically constructing a standard long short-term memory network and a standard gated recurrent unit, and performing weighted integration of temporal features to construct the recurrent integration subnetwork;

[0031] The standard long short-term memory network is used to optimize the feature extraction of long-term individual animal behavior;

[0032] The standard gated recurrent unit is used to optimize the feature extraction of short-term behavior changes;

[0033] Step S34: Individual behavior classification model training, specifically, through the improved residual convolution subnet, the visual detection subnet and the recurrent integration subnet, and constructing a combined classification output layer of the standard softmax model and the regression model, the individual behavior classification model training is performed to obtain the individual behavior classification model Model OA ;

[0034] Step S35: Individual behavior classification, specifically, using the individual behavior classification model Model according to the animal behavior recognition optimization data set OA , classify individual behaviors and obtain detailed data of animal individual behavior classification;

[0035] The animal individual behavior classification detailed data specifically includes image detection annotation data, behavior category classification data, behavior probability quantification reference data and timestamp data.

[0036] Further, in step S4, the group association analysis is used to analyze the association between individual behavior and group behavior for group animal behavior and provide classification details, specifically, based on the animal behavior recognition optimization data set and the animal individual behavior classification detail data, a dynamic and static graph neural integration network combined with reinforcement learning is used to perform group association analysis to obtain animal group behavior association analysis detail data;

[0037] The dynamic and static graph neural integration network combined with reinforcement learning specifically includes a reinforcement learning sub-model, a static graph neural sub-network and a dynamic graph neural sub-network;

[0038] The reinforcement learning sub-model is used to analyze the optimal possibility of individual animal behavior by optimizing individual behavior strategies;

[0039] The static graph neural subnet is used to capture the static relationship characteristics between individual animals;

[0040] The dynamic graph neural subnetwork is used to capture the dynamic relationship characteristics of individual animals that change over time;

[0041] The step of using a dynamic and static graph neural integration network combined with reinforcement learning to perform group association analysis and obtain detailed data on animal group behavior association analysis includes:

[0042] Step S41: constructing a static graph neural subnet, specifically constructing an initialization graph structure, and constructing a standard graph neural network as a static graph neural subnet;

[0043] The initialization graph structure specifically includes node features and edge features;

[0044] The node features are used to represent individual animals and their behavioral states, and are specifically obtained from the detailed classification data of the individual animal behaviors;

[0045] The edge features are used to represent the interaction intensity and similarity between animals, and are specifically obtained by combining the animal behavior recognition optimization dataset and the animal individual behavior classification detailed data;

[0046] Step S42: constructing a dynamic graph neural subnet, specifically, introducing a time variable dimension on the basis of a standard graph neural network according to the initialization graph structure, and constructing a dynamic graph neural subnet;

[0047] Step S43: constructing a reinforcement learning sub-model, specifically, by constructing a reinforcement learning agent model and performing iterative training of a deep Q network optimization strategy to obtain the optimal individual behavior type of group association;

[0048] In the group-related optimal individual behavior type, the optimal individual behavior specifically refers to the individual behavior with the strongest group relevance;

[0049] The construction of the reinforcement learning agent model specifically involves constructing individual animal state variables, individual animal behavior variables, and behavior reward variables, and the calculation formula is:

[0050] MA={s,a,r};

[0051] Wherein, MA is the reinforcement learning agent model, s is the animal individual state variable, which is used to represent the current individual behavior of the animal, a is the animal individual behavior variable, which is used to represent the behavior that the animal can currently perform, and r is the behavior reward variable, which is used to represent the reward parameter calculated based on the impact of individual behavior on group behavior or interactive behavior;

[0052] Step S44: constructing a cluster analysis output layer, specifically by integrating the output of the static graph neural subnetwork, the output of the dynamic graph neural subnetwork and the output of the reinforcement learning submodel, and constructing a cluster analysis layer for output, so as to obtain a group association analysis output;

[0053] The calculation formula of the group association analysis output is:

[0054] ;

[0055] In the formula, C g is the clustering output of group association analysis, f Kmeans (·) is the K-means clustering calculation function, is the reinforcement learning output weight, Q is the output of the reinforcement learning sub-model, is the weight of the static graph neural subnetwork, h k is the output of the static graph neural subnetwork, is the output of the dynamic graph neural subnetwork, h l (t) is the output of the dynamic graph neural subnetwork;

[0056] Step S45: group association analysis model training, specifically, training the group association analysis model through the static graph neural subnet, the dynamic graph neural subnet, the reinforcement learning submodel and the cluster analysis output layer to obtain the group association analysis model Model FA ;

[0057] Step S46: Group association analysis, specifically, using the group association analysis model Model based on the animal behavior recognition optimization data set and the animal individual behavior classification detailed data FA , conduct group association analysis and obtain detailed data on animal group behavior association analysis;

[0058] The animal group behavior association analysis detailed data specifically includes group behavior type, group behavior time pattern and group behavior associated animal emotion type.

[0059] Furthermore, in step S5, the animal behavior recognition is used to conduct a comprehensive analysis of animal behavior by combining individual and group behavior analysis, specifically combining the individual animal behavior classification detail data and the group animal behavior association analysis detail data to conduct a comprehensive animal behavior recognition and obtain animal behavior recognition reference data.

[0060] The present invention provides an animal behavior recognition system based on deep learning, comprising a data collection module, a data reconstruction module, an individual behavior classification module, a group association analysis module and an animal behavior recognition module;

[0061] The data collection module is used for data collection, and obtains an original data set for animal behavior recognition through data collection, and sends the original data set for animal behavior recognition to a data reconstruction module;

[0062] The data reconstruction module is used for data reconstruction, and obtains an animal behavior recognition optimization data set through data reconstruction, and sends the animal behavior recognition optimization data set to the individual behavior classification module and the group association analysis module;

[0063] The individual behavior classification module is used for individual behavior classification, and obtains animal individual behavior classification detailed data through individual behavior classification, and sends the animal individual behavior classification detailed data to the group association analysis module and the animal behavior recognition module;

[0064] The group association analysis module is used for group association analysis, and obtains detailed data of animal group behavior association analysis through group association analysis, and sends the detailed data of animal group behavior association analysis to the animal behavior recognition module;

[0065] The animal behavior recognition module is used for animal behavior recognition, and obtains animal behavior recognition reference data through animal behavior recognition.

[0066] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0067] (1) In view of the fact that there are many existing solutions for the individual behavior recognition of animals in the existing technologies, there are relatively few technical contents for the integrated behavior analysis and recognition of individuals and groups, and it is even more difficult to reflect the practical value when applied to domestic animals. Therefore, there is a lack of a technical problem of a method that can analyze the individual behavior of domestic animals and recognize the group-related animal behavior. This solution creatively adopts a comprehensive animal behavior recognition method that combines individual analysis and group analysis, identifies specific behavior types through individual behavior analysis, and combines multiple animals for comprehensive analysis and clustering through group association analysis, thereby improving the recognition depth and comprehensive usability of animal behavior recognition;

[0068] (2) In the existing individual behavior classification, there are traditional animal behavior classification methods that have poor recognition capabilities when dealing with complex backgrounds and local action details. At the same time, since domestic animals are not in a static state most of the time, they are often dynamic, which is reflected not only in the changes in action behavior but also in the changes in individual state. Therefore, the technical problem that the previous method has to improve its ability to capture the details of animal behavior, perceive changes, and analyze dynamics is that this method needs to improve its ability. This solution creatively uses a residual convolutional recurrent neural network with improved structure and attention to classify individual behaviors. By improving the traditional ResNet network structure and introducing two attention mechanisms, it improves the ability to capture the details of animal behavior and perceive changes. At the same time, by integrating the outputs of the two recurrent networks, it can more comprehensively cope with the dynamic analysis capabilities required for the classification of individual animal behaviors, improve the overall versatility of the method, and provide a good technical prerequisite for group analysis.

[0069] (3) In the existing animal group association analysis, there are existing methods that focus on single-stage behavior analysis for animal group behavior analysis. However, in fact, the essence of an animal group is a complex interactive system of multiple individual animals. Single-stage group analysis is not only difficult to effectively understand the behavior, emotional state and interaction information of animals, but also has a high probability of misinterpretation and misclassification, which in turn has a significant negative impact on the technical problem of animal group association analysis. This scheme creatively uses a dynamic and static graph neural integration network combined with reinforcement learning to conduct group association analysis. Through the integration of dynamic and static graph neural networks, the dynamic and static causal analysis of the interaction between individual animals and groups is improved. At the same time, by introducing reinforcement learning methods, the optimal interpretation method for the effect of individual animal behavior on the group can be better output. Finally, the clustering method is used to realize the animal group association analysis, which improves the comprehensive usability of animal behavior recognition and provides certain practical experience and technical support for the field of animal research. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of the process of the animal behavior recognition method based on deep learning provided by the present invention;

[0071] Figure 2 A schematic diagram of an animal behavior recognition system based on deep learning provided by the present invention;

[0072] Figure 3 A schematic diagram of the process of data reconstruction in step S2;

[0073] Figure 4 A schematic diagram of the process of classifying individual behaviors in step S3;

[0074] Figure 5 Schematic diagram of the process of population association analysis in step S4.

[0075] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0077] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0078] Example 1, see Figure 1 The present invention provides an animal behavior recognition method based on deep learning, which comprises the following steps:

[0079] Step S1: data collection;

[0080] Step S2: data reconstruction;

[0081] Step S3: individual behavior classification;

[0082] Step S4: group association analysis;

[0083] Step S5: Animal behavior recognition.

[0084] By performing the above operations, in view of the fact that in the existing animal behavior recognition methods, there are many solutions for the individual behavior recognition of animals in the prior art, but there are fewer technical contents for integrated behavior analysis and recognition of individuals and groups, and it is even more difficult to reflect practical value when applied to domestic animals. Therefore, there is a lack of a method that can analyze the individual behavior of domestic animals and identify the group-related animal behavior. The present solution creatively adopts a comprehensive animal behavior recognition method that combines individual analysis and group analysis, identifies specific behavior types through individual behavior analysis, and combines multiple animals for comprehensive analysis and clustering through group association analysis, thereby improving the recognition depth and comprehensive usability of animal behavior recognition.

[0085] Example 2, see Figure 1 and Figure 2 , This embodiment is based on the above embodiment. In step S1, the data collection is used to collect the original data required for animal behavior analysis, specifically, to capture animal behavior data through multi-source data collection to obtain an original data set for animal behavior recognition;

[0086] The animal behavior recognition raw data set specifically includes video data, label data, sensor data, sound data, environmental data, spatial data, temporal data, information data and health data;

[0087] The animals specifically refer to domestic pet animals, including domestic cats and dogs;

[0088] Table 1 is a schematic table of the original data set for animal behavior recognition. As shown in the table, the label data specifically includes eating, sleeping, walking, running, playing, calling, anxiety, attacking, human interaction, animal interaction and self-cleaning;

[0089] The information data specifically includes breed, age, gender and weight;

[0090] The breeds include domestic cats and dogs; the domestic dogs include golden retrievers, Labrador retrievers, border collies, German shepherds, pit bulls and poodles; the domestic cats include Persian cats, Siamese cats, Maine Coons, British shorthair cats and Bengal cats;

[0091] The age specifically includes childhood, adulthood and old age; the childhood age range is [0,1], in years; the adult age range is [1,6], in years; the old age range is [6,15], in years;

[0092] The said gender specifically includes female and male;

[0093] The body weight specifically includes light weight, medium weight and heavy weight; the weight range of the light weight is [1,10], in kilograms; the weight range of the medium weight is [10,25], in kilograms; the weight range of the heavy weight is [25,50], in kilograms;

[0094] The health data specifically refers to the physiological data of the animal, including body temperature data and blood sugar data;

[0095] The blood sugar data is in mmol / L;

[0096] Table 1 Schematic diagram of the original dataset for animal behavior recognition

[0097]

[0098] Example 3, see Figure 1 , Figure 2 and Figure 3 This embodiment is based on the above embodiment. In step S2, the data reconstruction is used to construct a data set for training individual and group behavior analysis sub-item models and perform data enhancement and optimization processing. Specifically, through data enhancement and optimization operations, the original animal behavior recognition data set is reconstructed into a format suitable for the training model to obtain an animal behavior recognition optimized data set, which specifically includes the following steps:

[0099] Step S21: data enhancement, specifically performing video data enhancement, sensor data enhancement and sound data enhancement on the original animal behavior recognition data set in sequence to obtain an enhanced data set;

[0100] The video data enhancement specifically includes random frame image rotation, cropping and scaling operations, and random noise injection;

[0101] The sensor data enhancement is specifically to perform random window sliding and data translation enhancement;

[0102] The sound data enhancement is specifically performed by random stretching, volume change and background noise superposition enhancement;

[0103] Step S22: data preprocessing, specifically, performing video preprocessing, sensor data preprocessing and sound data preprocessing on the enhanced data set in sequence, and performing artificial feature extraction to obtain an optimized feature data set;

[0104] The video preprocessing specifically refers to frame image conversion and size unification processing, and optical flow information feature extraction;

[0105] The sensor data preprocessing specifically refers to sensor data standardization processing, time series data segmentation and signal statistical feature extraction;

[0106] The sound data processing specifically converts the audio file into spectrogram data and extracts audio features through short-time Fourier transform;

[0107] Step S23: data conversion and fusion, specifically, converting the original data and the corresponding data labels into a unified format, and aligning them according to the timestamps, and then performing feature fusion on the feature data in the optimized feature data set to obtain an integrated feature data set;

[0108] Step S24: data division, specifically, dividing the original data and the integrated feature data set into a training set, a validation set and a test set, and performing category balance detection and adjustment to obtain a divided data set;

[0109] Step S25: Data reconstruction, specifically, performing data reconstruction through the data enhancement, the data preprocessing, the data conversion and fusion, and the data partitioning to obtain an animal behavior recognition optimized data set, wherein the animal behavior recognition optimized data set specifically includes a partitioned data set and an integrated feature data set.

[0110] Example 4, see Figure 1 , Figure 2 and Figure 4 This embodiment is based on the above embodiment. In step S3, the individual behavior classification is used to identify and classify the individual behavior types of animals and give detailed behavior classification results. Specifically, based on the animal behavior recognition optimization data set, a residual convolutional recurrent neural network with combined structure and attention improvement is used to perform individual behavior classification to obtain detailed animal individual behavior classification data;

[0111] The residual convolutional recurrent neural network with combined structure and attention improvement specifically includes an improved residual convolutional subnet, a visual detection subnet and a recurrent integration subnet;

[0112] The step of using the residual convolutional recurrent neural network with combined structure and attention improvement to classify individual behaviors and obtain detailed classification data of individual animal behaviors includes:

[0113] Step S31: constructing an improved residual convolutional subnet, specifically, constructing an improved residual convolutional subnet by improving the structure and attention of the standard residual convolutional network;

[0114] The standard residual convolutional network specifically refers to the ResNet-50 network;

[0115] The structural improvement and attention improvement specifically include structural improvement and attention improvement;

[0116] The structural improvement specifically refers to replacing the standard convolution layer with an integrated convolution after being processed by a temporal convolution layer and a depth-separable convolution layer;

[0117] The attention improvement specifically refers to the application of spatial attention and channel attention mechanisms to improve attention; the calculation formula for the structural improvement is:

[0118] X output =TCN(DSC(ResNet(X)));

[0119] Where, X output is the residual convolution feature output after structural improvement, which is used to represent the individual behavior classification feature data after time series correlation feature optimization and computational efficiency optimization. TCN(·) is the standard time series convolution operation function, DSC(·) is the standard depth separable convolution operation function, ResNet(·) is the standard ResNet-50 network function, and X is the original data input, which is used to represent the feature data in the animal behavior recognition optimization dataset.

[0120] Step S32: constructing a visual detection subnet, specifically using a pre-trained computer vision target detection model to perform target detection and obtain target detection positioning features;

[0121] The pre-trained computer vision target detection model specifically refers to the YOLOv8n model;

[0122] Step S33: constructing a recurrent integration subnetwork, specifically constructing a standard long short-term memory network and a standard gated recurrent unit, and performing weighted integration of temporal features to construct the recurrent integration subnetwork;

[0123] The standard long short-term memory network is used to optimize the feature extraction of long-term individual animal behavior, and specifically includes an input gate, an output gate, and a forget gate;

[0124] The standard gated recurrent unit is used to optimize the feature extraction of short-term behavior changes, and specifically includes a reset gate and an update gate;

[0125] The calculation formula of the weighted integration of the time series features is:

[0126] ;

[0127] In the formula, h f It is the hidden state of the weighted integrated output of the time series features, which is used as the total time series feature after weighted fusion. is the cyclic integration weight, the specific value is 0.65, h LSTM is the standard LSTM network output hidden state, h GRU is the standard gated recurrent unit output hidden state;

[0128] Step S34: Individual behavior classification model training, specifically, through the improved residual convolution subnet, the visual detection subnet and the recurrent integration subnet, and constructing a combined classification output layer of the standard softmax model and the regression model, the individual behavior classification model training is performed to obtain the individual behavior classification model Model OA ;

[0129] Step S35: Individual behavior classification, specifically, using the individual behavior classification model Model according to the animal behavior recognition optimization data set OA , classify individual behaviors and obtain detailed data of animal individual behavior classification;

[0130] The detailed classification data of individual animal behaviors specifically includes image detection annotation data, behavior category classification data, behavior probability quantification reference data and timestamp data;

[0131] The image detection annotation data specifically refers to the labeled image after target detection by the visual detection subnet;

[0132] The behavior category classification data specifically refers to the specific identification and classification categories of individual animal behaviors;

[0133] The behavior probability quantification reference data specifically refers to the predicted probability corresponding to the behavior category classification data;

[0134] The timestamp data is used to indicate the start and end time of an individual behavior and is used to assist in behavior identification and analysis.

[0135] By performing the above operations, in view of the fact that in the existing individual behavior classification, there are traditional animal behavior classification methods, which have poor recognition ability when dealing with complex backgrounds and local action details. At the same time, because domestic animals are not in a static state most of the time, they are mostly dynamic, which is not only reflected in the changes in action behavior, but also in the changes in individual state. Therefore, the technical problem that the method used previously has the ability to capture the details of animal behavior, perceive changes, and analyze dynamics needs to be improved. This solution creatively adopts a residual convolutional recurrent neural network with structural and attention improvements to perform individual behavior classification. By improving the traditional ResNet network structure and introducing two attention mechanisms, the ability to capture the details of animal behavior and the ability to perceive changes are improved. At the same time, by integrating the outputs of the two recurrent networks, it can more comprehensively cope with the dynamic analysis capabilities required for the classification of individual animal behaviors, improve the overall versatility of the method, and provide a good technical prerequisite for group analysis.

[0136] Example 5, see Figure 1 , Figure 2 and Figure 5 This embodiment is based on the above embodiment. In step S4, the group association analysis is used to analyze the association between individual behavior and group behavior for group animal behavior and provide classification details. Specifically, based on the animal behavior recognition optimization data set and the animal individual behavior classification detail data, a dynamic and static graph neural integration network combined with reinforcement learning is used to perform group association analysis to obtain animal group behavior association analysis detail data;

[0137] The dynamic and static graph neural integration network combined with reinforcement learning specifically includes a reinforcement learning sub-model, a static graph neural sub-network and a dynamic graph neural sub-network;

[0138] The reinforcement learning sub-model is used to analyze the optimal possibility of individual animal behavior by optimizing individual behavior strategies;

[0139] The static graph neural subnet is used to capture the static relationship characteristics between individual animals;

[0140] The dynamic graph neural subnetwork is used to capture the dynamic relationship characteristics of individual animals that change over time;

[0141] The step of using a dynamic and static graph neural integration network combined with reinforcement learning to perform group association analysis and obtain detailed data on animal group behavior association analysis includes:

[0142] Step S41: constructing a static graph neural subnet, specifically constructing an initialization graph structure, and constructing a standard graph neural network as a static graph neural subnet;

[0143] The initialization graph structure specifically includes node features and edge features;

[0144] The node features are used to represent individual animals and their behavioral states, and are specifically obtained from the detailed classification data of the individual animal behaviors;

[0145] The edge features are used to represent the interaction intensity and similarity between animals, and are specifically obtained by combining the animal behavior recognition optimization dataset and the animal individual behavior classification detailed data;

[0146] The calculation formula of the static graph neural subnet is:

[0147] ;

[0148] In the formula, is the static graph feature of the i-th node in the k+1-th static graph convolution layer, which is used to represent the static relationship features between individual animals. ReLU(·) is a nonlinear activation function. N is the total number of neighbor nodes of the node feature. j is the neighbor node index. i is the node feature index, which is used to represent the specific individual animal index. k is the graph static convolution level index of the static graph neural subnet. c ij is the normalization coefficient, W k is the learning weight matrix corresponding to the static convolutional layer of the kth graph, b k is the bias term corresponding to the static convolutional layer of the kth graph;

[0149] Step S42: constructing a dynamic graph neural subnet, specifically, introducing a time variable dimension on the basis of a standard graph neural network according to the initialization graph structure, and constructing a dynamic graph neural subnet;

[0150] The calculation formula of the dynamic graph neural subnet is:

[0151] ;

[0152] In the formula, is the dynamic graph feature of the i-th node in the l+1th static graph convolution layer, which is used to represent the dynamic relationship features of individual animals over time. ReLU(·) is a nonlinear activation function. N is the total number of neighbor nodes of the node feature. j is the neighbor node index. i is the node feature index. l is the graph dynamic convolution level index of the dynamic graph neural subnet. c ij is the normalization coefficient, W l is the learning weight matrix corresponding to the lth graph dynamic convolution layer, b l is the bias term corresponding to the lth graph dynamic convolution layer, and t is the time index;

[0153] Step S43: constructing a reinforcement learning sub-model, specifically, by constructing a reinforcement learning agent model and performing iterative training of a deep Q network optimization strategy to obtain the optimal individual behavior type of group association;

[0154] In the group-related optimal individual behavior type, the optimal individual behavior specifically refers to the individual behavior with the strongest group relevance;

[0155] The construction of the reinforcement learning agent model specifically involves constructing individual animal state variables, individual animal behavior variables, and behavior reward variables, and the calculation formula is:

[0156] MA={s,a,r};

[0157] Wherein, MA is the reinforcement learning agent model, s is the animal individual state variable, which is used to represent the current individual behavior of the animal, a is the animal individual behavior variable, which is used to represent the behavior that the animal can currently perform, and r is the behavior reward variable, which is used to represent the reward parameter calculated based on the impact of individual behavior on group behavior or interactive behavior;

[0158] The calculation formula for iterative training of the deep Q network optimization strategy is:

[0159] ;

[0160] In the formula, It is in The state variables of the corresponding animal individuals in the iteration and individual animal behavior variables The Q value is used to represent the behavior variables of individual animals. The expected return to the group when performing an individual action, E[·] is the expected function of the reward parameter, It is The individual animal state variables in the iteration, It is The individual animal behavior variables in the iteration, It is The behavior reward variable in the iteration, is the discount factor, is the individual animal state variable in the next iteration, It is the individual optimal behavior of animals;

[0161] Step S44: constructing a cluster analysis output layer, specifically by integrating the output of the static graph neural subnetwork, the output of the dynamic graph neural subnetwork and the output of the reinforcement learning submodel, and constructing a cluster analysis layer for output, so as to obtain a group association analysis output;

[0162] The calculation formula of the group association analysis output is:

[0163] ;

[0164] Where Cg is the clustering output of group association analysis, fKmeans (·) is the K-means clustering calculation function, is the reinforcement learning output weight, Q is the output of the reinforcement learning sub-model, is the weight of the static graph neural subnetwork, h k is the output of the static graph neural subnetwork, is the output of the dynamic graph neural subnetwork, h l (t) is the output of the dynamic graph neural subnetwork;

[0165] Among them, reinforcement learning output weight The specific value of is 0.4, the weight of the static graph neural subnet The specific value of is 0.25, and the output of the dynamic graph neural subnet The specific value of is 0.35;

[0166] Step S45: group association analysis model training, specifically, training the group association analysis model through the static graph neural subnet, the dynamic graph neural subnet, the reinforcement learning submodel and the cluster analysis output layer to obtain the group association analysis model Model FA ;

[0167] Step S46: Group association analysis, specifically, using the group association analysis model Model based on the animal behavior recognition optimization data set and the animal individual behavior classification detailed data FA , conduct group association analysis and obtain detailed data on animal group behavior association analysis;

[0168] The detailed data of the animal group behavior association analysis specifically includes group behavior type, group behavior time pattern and group behavior associated animal emotion type;

[0169] The group behavior types specifically include collective synchronous behavior, independent behavior, group dormant behavior and group interactive behavior;

[0170] The group behavior time pattern specifically includes the time period of behavior occurrence, the duration of behavior and the frequency of behavior;

[0171] The group behavior is associated with the animal's emotional type, specifically including friendly, aggressive, anxious and dependent.

[0172] By performing the above operations, in the existing animal group association analysis, there is a problem that the existing methods for animal group behavior analysis focus on single-stage behavior analysis. However, in fact, the essence of an animal group is a complex interaction system of multiple individual animals. Single-stage group analysis is not only difficult to effectively understand the behavior, emotional state and interaction information of animals, but also has a great possibility of misinterpretation and misclassification, which in turn has a significant negative impact on the technical problem of animal group association analysis. This solution creatively uses a dynamic and static graph neural integration network combined with reinforcement learning to perform group association analysis. Through the integration of dynamic and static graph neural networks, the dynamic and static causal analysis of the interaction between individual animals and groups is improved. At the same time, by introducing reinforcement learning methods, the optimal interpretation method for the effect of individual animal behavior on the group can be better output. Finally, the clustering method is used to realize the animal group association analysis, which improves the comprehensive usability of animal behavior recognition and provides certain practical experience and technical support for the field of animal research.

[0173] Example 6, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S5, the animal behavior recognition is used to conduct a comprehensive analysis of the animal behavior by combining individual and group behavior analysis. Specifically, the animal individual behavior classification detail data and the animal group behavior association analysis detail data are combined to conduct a comprehensive animal behavior recognition to obtain animal behavior recognition reference data.

[0174] Embodiment 7, see Figure 1 and Figure 2 , this embodiment is based on the above embodiment, and the present invention provides an animal behavior recognition system based on deep learning, including a data collection module, a data reconstruction module, an individual behavior classification module, a group association analysis module and an animal behavior recognition module;

[0175] The data collection module is used for data collection, and obtains an original data set for animal behavior recognition through data collection, and sends the original data set for animal behavior recognition to a data reconstruction module;

[0176] The data reconstruction module is used for data reconstruction, and obtains an animal behavior recognition optimization data set through data reconstruction, and sends the animal behavior recognition optimization data set to the individual behavior classification module and the group association analysis module;

[0177] The individual behavior classification module is used for individual behavior classification, and obtains animal individual behavior classification detailed data through individual behavior classification, and sends the animal individual behavior classification detailed data to the group association analysis module and the animal behavior recognition module;

[0178] The group association analysis module is used for group association analysis, and obtains detailed data of animal group behavior association analysis through group association analysis, and sends the detailed data of animal group behavior association analysis to the animal behavior recognition module;

[0179] The animal behavior recognition module is used for animal behavior recognition, and obtains animal behavior recognition reference data through animal behavior recognition.

[0180] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0181] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0182] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. Animal behavior recognition method based on deep learning, characterized by: The method comprises the following steps: Step S1: Data collection, through multi-source data collection, animal behavior data is captured to obtain an original data set for animal behavior recognition; the animals specifically refer to domestic pet animals, including domestic cats and dogs; Step S2: Data reconstruction to obtain an optimized data set for animal behavior recognition, specifically including dividing the data set and integrating the feature data set; Step S3: individual behavior classification, which is used to identify and classify the individual behavior types of animals and give detailed behavior classification results. Specifically, based on the animal behavior recognition optimization data set, a residual convolutional recurrent neural network with improved structure and attention is used to perform individual behavior classification to obtain detailed animal individual behavior classification data, including the following steps: Step S31: constructing an improved residual convolutional subnet, specifically, constructing an improved residual convolutional subnet by improving the structure and attention of the standard residual convolutional network; The structural improvement and attention improvement specifically include structural improvement and attention improvement; The structural improvement specifically refers to replacing the standard convolution layer with an integrated convolution after being processed by a temporal convolution layer and a depth-separable convolution layer; The attention improvement specifically refers to the application of spatial attention and channel attention mechanisms to improve attention; Step S32: constructing a visual detection subnet, specifically using a pre-trained computer vision target detection model to perform target detection and obtain target detection positioning features; Step S33: constructing a recurrent integration subnetwork, specifically constructing a standard long short-term memory network and a standard gated recurrent unit, and performing weighted integration of temporal features to construct the recurrent integration subnetwork; The standard long short-term memory network is used to optimize the feature extraction of long-term individual animal behavior; The standard gated recurrent unit is used to optimize the feature extraction of short-term behavior changes; Step S34: Individual behavior classification model training, specifically, through the improved residual convolution subnet, the visual detection subnet and the recurrent integration subnet, and constructing a combined classification output layer of the standard softmax model and the regression model, the individual behavior classification model training is performed to obtain the individual behavior classification model Model OA ; Step S35: Individual behavior classification, specifically, using the individual behavior classification model Model according to the animal behavior recognition optimization data set OA , classify individual behaviors and obtain detailed data of animal individual behavior classification; The detailed classification data of individual animal behaviors specifically includes image detection annotation data, behavior category classification data, behavior probability quantification reference data and timestamp data Step S4: Group association analysis, which is used to analyze the association between individual behavior and group behavior for group animal behavior and provide classification details. Specifically, based on the animal behavior recognition optimization data set and the animal individual behavior classification details data, a dynamic and static graph neural integration network combined with reinforcement learning is used to perform group association analysis to obtain animal group behavior association analysis details data, which specifically includes the following steps: Step S41: constructing a static graph neural subnet, specifically constructing an initialization graph structure, and constructing a standard graph neural network as a static graph neural subnet; The initialization graph structure specifically includes node features and edge features; The node features are used to represent individual animals and their behavioral states, and are specifically obtained from the detailed classification data of the individual animal behaviors; The edge features are used to represent the interaction intensity and similarity between animals, and are specifically obtained by combining the animal behavior recognition optimization dataset and the animal individual behavior classification detailed data; Step S42: constructing a dynamic graph neural subnet, specifically, introducing a time variable dimension on the basis of a standard graph neural network according to the initialization graph structure, and constructing a dynamic graph neural subnet; Step S43: constructing a reinforcement learning sub-model, specifically, by constructing a reinforcement learning agent model and performing iterative training of a deep Q network optimization strategy to obtain the optimal individual behavior type of group association; In the group-related optimal individual behavior type, the optimal individual behavior specifically refers to the individual behavior with the strongest group relevance; The construction of the reinforcement learning agent model specifically involves constructing individual animal state variables, individual animal behavior variables, and behavior reward variables, and the calculation formula is: MA={s,a,r}; Wherein, MA is the reinforcement learning agent model, s is the animal individual state variable, which is used to represent the current individual behavior of the animal, a is the animal individual behavior variable, which is used to represent the behavior that the animal can currently perform, and r is the behavior reward variable, which is used to represent the reward parameter calculated based on the impact of individual behavior on group behavior or interactive behavior; Step S44: constructing a cluster analysis output layer, specifically by integrating the output of the static graph neural subnetwork, the output of the dynamic graph neural subnetwork and the output of the reinforcement learning submodel, and constructing a cluster analysis layer for output, so as to obtain a group association analysis output; The calculation formula of the group association analysis output is: C g =f Kmeans (λ1Q+λ2h k +λ3h 1 (t)); In the formula, C g is the clustering output of group association analysis, f Kmeans (·) is the K-means clustering calculation function, λ1 is the reinforcement learning output weight, Q is the output of the reinforcement learning sub-model, λ2 is the static graph neural sub-network weight, and h k is the output of the static graph neural subnet, λ3 is the output of the dynamic graph neural subnet, and h l (t) is the output of the dynamic graph neural subnetwork; Step S45: group association analysis model training, specifically, training the group association analysis model through the static graph neural subnet, the dynamic graph neural subnet, the reinforcement learning submodel and the cluster analysis output layer to obtain the group association analysis model Model FA ; Step S46: Group association analysis, specifically, using the group association analysis model Model based on the animal behavior recognition optimization data set and the animal individual behavior classification detailed data FA , conduct group association analysis and obtain detailed data on animal group behavior association analysis; The detailed data of the animal group behavior association analysis specifically includes group behavior type, group behavior time pattern and group behavior associated animal emotion type; Step S5: Animal behavior identification, performing comprehensive animal behavior identification to obtain animal behavior identification reference data.

2. The animal behavior recognition method based on deep learning according to claim 1, characterized in that: In step S1, the animal behavior recognition original data set specifically includes video data, label data, sensor data, sound data, environmental data, spatial data, time data, information data and health data.

3. The animal behavior recognition method based on deep learning according to claim 2, characterized in that: In step S2, the data reconstruction is used to construct a data set for training individual and group behavior analysis sub-item models and perform data enhancement and optimization processing. Specifically, through data enhancement and optimization operations, the original animal behavior recognition data set is reconstructed into a format suitable for the training model to obtain an animal behavior recognition optimized data set, which specifically includes the following steps: Step S21: data enhancement, specifically performing video data enhancement, sensor data enhancement and sound data enhancement on the original animal behavior recognition data set in sequence to obtain an enhanced data set; Step S22: data preprocessing, specifically, performing video preprocessing, sensor data preprocessing and sound data preprocessing on the enhanced data set in sequence, and performing artificial feature extraction to obtain an optimized feature data set; Step S23: data conversion and fusion, specifically, converting the original data and the corresponding data labels into a unified format, and aligning them according to the timestamps, and then performing feature fusion on the feature data in the optimized feature data set to obtain an integrated feature data set; Step S24: data division, specifically, dividing the original data and the integrated feature data set into a training set, a validation set and a test set, and performing category balance detection and adjustment to obtain a divided data set; Step S25: Data reconstruction, specifically, performing data reconstruction through the data enhancement, the data preprocessing, the data conversion and fusion, and the data partitioning to obtain an animal behavior recognition optimized data set, wherein the animal behavior recognition optimized data set specifically includes a partitioned data set and an integrated feature data set.

4. The animal behavior recognition method based on deep learning according to claim 3, characterized in that: In step S3, the residual convolutional recurrent neural network with combined structure and attention improvement specifically includes an improved residual convolutional subnet, a visual detection subnet and a recurrent integration subnet.

5. The animal behavior recognition method based on deep learning according to claim 4, characterized in that: In step S4, the dynamic and static graph neural integrated network combined with reinforcement learning specifically includes a reinforcement learning sub-model, a static graph neural sub-network and a dynamic graph neural sub-network; The reinforcement learning sub-model is used to analyze the optimal possibility of individual animal behavior by optimizing individual behavior strategies; The static graph neural subnet is used to capture the static relationship characteristics between individual animals; The dynamic graph neural subnet is used to capture the dynamic relationship characteristics of individual animals that change over time.

6. The animal behavior recognition method based on deep learning according to claim 5, characterized in that: In step S5, the animal behavior recognition is used to conduct a comprehensive analysis of animal behavior by combining individual and group behavior analysis, specifically combining the individual animal behavior classification detail data and the group animal behavior association analysis detail data to conduct a comprehensive animal behavior recognition and obtain animal behavior recognition reference data.

7. An animal behavior recognition system based on deep learning, used to implement the animal behavior recognition method based on deep learning as described in any one of claims 1 to 6, characterized in that: It includes a data collection module, a data reconstruction module, an individual behavior classification module, a group association analysis module and an animal behavior recognition module.

8. The animal behavior recognition system based on deep learning according to claim 7, characterized in that: The data collection module is used for data collection, and obtains an original data set for animal behavior recognition through data collection, and sends the original data set for animal behavior recognition to a data reconstruction module; The data reconstruction module is used for data reconstruction, and obtains an animal behavior recognition optimization data set through data reconstruction, and sends the animal behavior recognition optimization data set to the individual behavior classification module and the group association analysis module; The individual behavior classification module is used for individual behavior classification, and obtains animal individual behavior classification detailed data through individual behavior classification, and sends the animal individual behavior classification detailed data to the group association analysis module and the animal behavior recognition module; The group association analysis module is used for group association analysis, and obtains detailed data of animal group behavior association analysis through group association analysis, and sends the detailed data of animal group behavior association analysis to the animal behavior recognition module; The animal behavior recognition module is used for animal behavior recognition, and obtains animal behavior recognition reference data through animal behavior recognition.

Images